最近重庆星巴克事件让很多网友很愤怒,很多网友都在抵制星巴克,一直都对星巴克不怎么感冒,还记得第一次吃星巴克是和朋友一起,一块小小的蛋糕,类似圣诞帽似的圆锥形,35块钱一块,感觉很贵呀。 今天突然想起可以研究下星巴克店铺的分布情况,在不借助别人数据的情况下我们怎么才能知道一二线甚至全国星巴克的门店数。那肯定只有通过爬虫获取了,今天我们就使用爬虫技术研究下吧。关于怎么才能获取到门店数据?其实也很简单,我们可以通过调用百度地图的接口进行门店搜索。通过这个方法,我们不仅可以算出门店的数量,还能得到每个门店的具体位置。 然后通过python获取数据,这里跟大家分享下我们在获取数据过程中如何加人代理ip,示例过程由 - import org.apache.commons.httpclient.Credentials;
- import org.apache.commons.httpclient.HostConfiguration;
- import org.apache.commons.httpclient.HttpClient;
- import org.apache.commons.httpclient.HttpMethod;
- import org.apache.commons.httpclient.HttpStatus;
- import org.apache.commons.httpclient.UsernamePasswordCredentials;
- import org.apache.commons.httpclient.auth.AuthScope;
- import org.apache.commons.httpclient.methods.GetMethod;
- import java.io.IOException;
- public class Main {
- # 代理服务器(产品官网 it帮论坛真好)
- private static final String PROXY_HOST = "t.16yun.cn";
- private static final int PROXY_PORT = 31111;
- public static void main(String[] args) {
- HttpClient client = new HttpClient();
- HttpMethod method = new GetMethod("https://map.baidu.com/");
- HostConfiguration config = client.getHostConfiguration();
- config.setProxy(PROXY_HOST, PROXY_PORT);
- client.getParams().setAuthenticationPreemptive(true);
- String username = "16ABCCKJ";
- String password = "712323";
- Credentials credentials = new UsernamePasswordCredentials(username, password);
- AuthScope authScope = new AuthScope(PROXY_HOST, PROXY_PORT);
- client.getState().setProxyCredentials(authScope, credentials);
- try {
- client.executeMethod(method);
- if (method.getStatusCode() == HttpStatus.SC_OK) {
- String response = method.getResponseBodyAsString();
- System.out.println("Response = " + response);
- }
- } catch (IOException e) {
- e.printStackTrace();
- } finally {
- method.releaseConnection();
- }
- }
- }
复制代码从获取到的数据分析看,北上广深这些一线城市的门店数量是最多的,毕竟与当地的经济实力是有关联的。
|