starbhhc
- 浏览: 636419 次
- 性别:
- 来自: 深圳
-
文章分类
- 全部博客 (609)
- java (139)
- 数据库 (107)
- 微信 (23)
- IT生活 (5)
- web前端 (74)
- SSH (11)
- 设计模式 (12)
- 重要资料 (11)
- 其他 (15)
- java技巧 (23)
- 服务器 (9)
- 2D/GUI (3)
- JAVA3D (2)
- ANT (5)
- Apache项目 (19)
- 数据类型 (10)
- 报表 (3)
- Collections (6)
- SQL/JDBC (15)
- 开发类 (6)
- EJB (6)
- Email (6)
- 文件读写 (2)
- 游戏 (0)
- Flex (2)
- Generic (2)
- HIbernate (12)
- I18N (5)
- Java EE (9)
- java ME (4)
- JDK 6 (8)
- JNDI/LDAP (5)
- JSP (7)
- JSTL (2)
- 正则表达式 (2)
- 安全 (2)
- Struts2 (12)
- Spring (4)
- Web服务 (10)
- Xml (1)
- JavaScript (30)
- AJAX (7)
- 验证 (4)
- 上传下载 (1)
- office办公软件 (1)
- Android (2)
- IOS (0)
- Dubbo (3)
- memcached/redis (1)
- 小程序 (1)
- 微信公众号 (0)
最新评论
-
wf_wangfeng：
怎么我用第一种方法不行呢 alert(document.rea ...
当jsp页面完全加载完成后执行一个js函数 -
Lori_Liu：
有帮助，至少可以解决了目前所遇到的问题！谢谢..
当jsp页面完全加载完成后执行一个js函数 -
starbhhc：
String actionMessage = new Stri ...
Java读取txt文件乱码 -
starbhhc：
Sev7en_jun 写道GOOD
客气，互相交流。。
javaeye论坛规则小测验（答案）--star -
Sev7en_jun：
GOOD
javaeye论坛规则小测验（答案）--star
地区代码整理在附件中citys.xml
说明：p代表省 n 省名称例如:B 北京,c 城市
返回的json数据示例：
Java 代码
{"weatherinfo":{"city":"上海","city_en":"shanghai","date_y":"2010年5月31日","date":"庚寅年四月十八","week":"星期一","fchh":"08","cityid":"101020100","temp1":"27℃~18℃","temp2":"26℃~18℃","temp3":"27℃~19℃","temp4":"27℃~20℃","temp5":"25℃~20℃","tempF1":"80.6℉~64.4℉","tempF2":"78.8℉~64.4℉","tempF3":"80.6℉~66.2℉","tempF4":"80.6℉~68℉","tempF5":"77℉~68℉","weather1":"多云","weather2":"多云","weather3":"晴转多云","weather4":"多云","weather5":"阴","img1":"1","img2":"99","img3":"1","img4":"99","img5":"0","img6":"1","img7":"1","img8":"99","img9":"2","img10":"99","img_single":"1","img_title1":"多云","img_title2":"多云","img_title3":"多云","img_title4":"多云","img_title5":"晴","img_title6":"多云","img_title7":"多云","img_title8":"多云","img_title9":"阴","img_title10":"阴","img_title_single":"多云","wind1":"东风3-4级","wind2":"东风3-4级","wind3":"东风3-4级","wind4":"东南风4-5级","wind5":"东南风转东风4-5级","fx1":"东风","fx2":"东风","fl1":"3-4级","fl2":"3-4级","fl3":"3-4级","fl4":"4-5级","fl5":"4-5级","index":"暂缺","index_d":"暂缺","index48":"暂缺","index48_d":"暂缺","index_uv":"弱","index48_uv":"弱","index_xc":"适宜","index_tr":"很适宜","index_co":"较舒适","st1":"26","st2":"17","st3":"25","st4":"17","st5":"25","st6":"18"}}
{"weatherinfo":{"city":"上海","city_en":"shanghai","date_y":"2010年5月31日","date":"庚寅年四月十八","week":"星期一","fchh":"08","cityid":"101020100","temp1":"27℃~18℃","temp2":"26℃~18℃","temp3":"27℃~19℃","temp4":"27℃~20℃","temp5":"25℃~20℃","tempF1":"80.6℉~64.4℉","tempF2":"78.8℉~64.4℉","tempF3":"80.6℉~66.2℉","tempF4":"80.6℉~68℉","tempF5":"77℉~68℉","weather1":"多云","weather2":"多云","weather3":"晴转多云","weather4":"多云","weather5":"阴","img1":"1","img2":"99","img3":"1","img4":"99","img5":"0","img6":"1","img7":"1","img8":"99","img9":"2","img10":"99","img_single":"1","img_title1":"多云","img_title2":"多云","img_title3":"多云","img_title4":"多云","img_title5":"晴","img_title6":"多云","img_title7":"多云","img_title8":"多云","img_title9":"阴","img_title10":"阴","img_title_single":"多云","wind1":"东风3-4级","wind2":"东风3-4级","wind3":"东风3-4级","wind4":"东南风4-5级","wind5":"东南风转东风4-5级","fx1":"东风","fx2":"东风","fl1":"3-4级","fl2":"3-4级","fl3":"3-4级","fl4":"4-5级","fl5":"4-5级","index":"暂缺","index_d":"暂缺","index48":"暂缺","index48_d":"暂缺","index_uv":"弱","index48_uv":"弱","index_xc":"适宜","index_tr":"很适宜","index_co":"较舒适","st1":"26","st2":"17","st3":"25","st4":"17","st5":"25","st6":"18"}}
导航189天气抓取示例：
Crawler.java
Java 代码
import java.util.TimerTask;
import org.apache.commons.lang.StringUtils;
import org.htmlparser.Parser;
import org.htmlparser.beans.StringBean;
import org.htmlparser.lexer.Lexer;
public abstract class Crawler extends TimerTask {
public String convert2String(String html) {
if (html != null) {
String str = html;
Lexer l = new Lexer(str);
Parser parser = new Parser(l);
StringBean sb = new StringBean();
try {
parser.visitAllNodesWith(sb);
} catch (Exception e) {
return str;
}
String ret = sb.getStrings();
if (ret != null) {
ret = ret.replace("\r", "");
ret = ret.replace("\t", "");
}
return ret;
}
return "";
}
public String formatString(String str) {
if (StringUtils.isBlank(str)) {
return "";
}
str = convert2String(str);
str = str.replaceAll("&", "&");
str = str.replaceAll("\"", "“").replaceAll("<", "").replaceAll("\"", "“");
str = str.trim();
return str;
}
}
import java.util.TimerTask;
import org.apache.commons.lang.StringUtils;
import org.htmlparser.Parser;
import org.htmlparser.beans.StringBean;
import org.htmlparser.lexer.Lexer;
public abstract class Crawler extends TimerTask {
public String convert2String(String html) {
if (html != null) {
String str = html;
Lexer l = new Lexer(str);
Parser parser = new Parser(l);
StringBean sb = new StringBean();
try {
parser.visitAllNodesWith(sb);
} catch (Exception e) {
return str;
}
String ret = sb.getStrings();
if (ret != null) {
ret = ret.replace("\r", "");
ret = ret.replace("\t", "");
}
return ret;
}
return "";
}
public String formatString(String str) {
if (StringUtils.isBlank(str)) {
return "";
}
str = convert2String(str);
str = str.replaceAll("&", "&");
str = str.replaceAll("\"", "“").replaceAll("<", "").replaceAll("\"", "“");
str = str.trim();
return str;
}
}
WeatherCrawler.java
Java 代码
import java.net.URI;
import java.util.Date;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.HttpConnectionParams;
import org.apache.http.params.HttpProtocolParams;
import org.apache.http.util.EntityUtils;
import org.json.JSONException;
import org.json.JSONObject;
import com.dh189.util.Constants;
import com.dh189.util.Util;
public class WeatherCrawler extends Crawler {
private static final String GETURL = "http://m.weather.com.cn/data/";
DefaultHttpClient httpclient = new DefaultHttpClient();
public void start() {
String[] citys = Constants.WEATHERCITYCODE.split(",");
for (String city : citys) {
String url = GETURL + city + ".html";
String val = getWeather(url);
try {
JSONObject json = new JSONObject(val);
JSONObject j = json.getJSONObject("weatherinfo");
String alt1 = j.getString("weather1") + " " + j.getString("wind1");
String alt2 = j.getString("weather2") + " " + j.getString("wind2");
String html = "<div class=\"w_l\"> <div class=\"w_l_u\"> <table style=\"width: 35px;height: 45px;\"> <tr> <td style=\"color: #F24F00\"><b>"
+ j.getString("city")
+ "</b></td> </tr> </table> < /div> <div class=\"w_l_d\"><a href=\"javascript:dingCity(); \" title=\"定制天气预报\" target=\"_self\">定制</a> </div> < /div> <ul> <li class=\"w_r\" style=\"width: 222px;\"> <table style=\"height: 70px;width: 100%;float: left;\"> <tr valign=\"bottom\"> <td><img src=\"images/weather/b"
+ j.getString("img1") + ".gif\" height=\"30\"/></td> <td><img src=\"images/weather/b" + j.getString("img3")
+ ".gif\" height=\"30\"/></td> </tr> <tr> <td width=\"105\" style=\"line-height: 15px;\"><a href=\"http://www.weather.com.cn/html/weather/" + city + ".shtml\" title=\""
+ alt1 + "\">" + Util.getWeekOfDate(new Date()) + ":" + j.getString("temp1") + "<br/>" + j.getString("weather1")
+ "</a></td> <td width=\"105\" style=\"line-height: 15px;\"><a href=\"http://www.weather.com.cn/html/weather/" + city + ".shtml\" title=\"" + alt2 + "\">"
+ Util.getWeekOfDate(Util.dateAddOrReduce(new Date(), 1)) + ":" + j.getString("temp2") + "<br/>" + j.getString("weather2") + "</a></td></tr></table></li></ul>";
String dir = Constants.WEATHERDIR + city + ".html";
Util.writeSortFile(dir, html, false);
} catch (JSONException e) {
e.printStackTrace();
}
}
}
/**
* 处理GET请求，返回整个页面
*
* @param url
* @return
*/
public String getWeather(String url) {
String content = null;
try {
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpConnectionParams.setConnectionTimeout(httpclient.getParams(), 30 * 1000);
HttpConnectionParams.setSoTimeout(httpclient.getParams(), 30 * 1000);
//伪装成火狐浏览器
HttpProtocolParams.setUserAgent(httpclient.getParams(), "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9");
HttpGet httpget = new HttpGet();
content = "";
httpget.setURI(new URI(url));
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
content = null;
if (entity != null) {
content = EntityUtils.toString(entity);
httpget.abort();
httpclient.getConnectionManager().shutdown();
}
} catch (Exception e) {
e.printStackTrace();
}
return content;
}
@Override
public void run() {
start();
}
}
import java.net.URI;
import java.util.Date;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.HttpConnectionParams;
import org.apache.http.params.HttpProtocolParams;
import org.apache.http.util.EntityUtils;
import org.json.JSONException;
import org.json.JSONObject;
import com.dh189.util.Constants;
import com.dh189.util.Util;
public class WeatherCrawler extends Crawler {
private static final String GETURL = "http://m.weather.com.cn/data/";
DefaultHttpClient httpclient = new DefaultHttpClient();
public void start() {
String[] citys = Constants.WEATHERCITYCODE.split(",");
for (String city : citys) {
String url = GETURL + city + ".html";
String val = getWeather(url);
try {
JSONObject json = new JSONObject(val);
JSONObject j = json.getJSONObject("weatherinfo");
String alt1 = j.getString("weather1") + " " + j.getString("wind1");
String alt2 = j.getString("weather2") + " " + j.getString("wind2");
String html = "<div class=\"w_l\"> <div class=\"w_l_u\"> <table style=\"width: 35px;height: 45px;\"> <tr> <td style=\"color: #F24F00\"><b>"
+ j.getString("city")
+ "</b></td> </tr> </table> </div> <div class=\"w_l_d\"><a href=\"javascript:dingCity();\" title=\"定制天气预报\" target=\"_self\">定制</a> </div> </div> <ul> <li class=\"w_r\" style=\"width: 222px;\"> <table style=\"height: 70px;width: 100%;float: left;\"> <tr valign=\"bottom\"> <td><img src=\"images/weather/b"
+ j.getString("img1") + ".gif\" height=\"30\"/></td> <td><img src=\"images/weather/b" + j.getString("img3")
+ ".gif\" height=\"30\"/></td> </tr> <tr> <td width=\"105\" style=\"line-height: 15px;\"><a href=\"http://www.weather.com.cn/html/weather/" + city + ".shtml\" title=\""
+ alt1 + "\">" + Util.getWeekOfDate(new Date()) + ":" + j.getString("temp1") + "<br/>" + j.getString("weather1")
+ "</a></td> <td width=\"105\" style=\"line-height: 15px;\"><a href=\"http://www.weather.com.cn/html/weather/" + city + ".shtml\" title=\"" + alt2 + "\">"
+ Util.getWeekOfDate(Util.dateAddOrReduce(new Date(), 1)) + ":" + j.getString("temp2") + "<br/>" + j.getString("weather2") + "</a></td></tr></table></li></ul>";
String dir = Constants.WEATHERDIR + city + ".html";
Util.writeSortFile(dir, html, false);
} catch (JSONException e) {
e.printStackTrace();
}
}
}
/**
* 处理GET请求，返回整个页面
*
* @param url
* @return
*/
public String getWeather(String url) {
String content = null;
try {
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpConnectionParams.setConnectionTimeout(httpclient.getParams(), 30 * 1000);
HttpConnectionParams.setSoTimeout(httpclient.getParams(), 30 * 1000);
//伪装成火狐浏览器
HttpProtocolParams.setUserAgent(httpclient.getParams(), "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9");
HttpGet httpget = new HttpGet();
content = "";
httpget.setURI(new URI(url));
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
content = null;
if (entity != null) {
content = EntityUtils.toString(entity);
httpget.abort();
httpclient.getConnectionManager().shutdown();
}
} catch (Exception e) {
e.printStackTrace();
}
return content;
}
@Override
public void run() {
start();
}
}
说明：p代表省 n 省名称例如:B 北京,c 城市
返回的json数据示例：
Java 代码
{"weatherinfo":{"city":"上海","city_en":"shanghai","date_y":"2010年5月31日","date":"庚寅年四月十八","week":"星期一","fchh":"08","cityid":"101020100","temp1":"27℃~18℃","temp2":"26℃~18℃","temp3":"27℃~19℃","temp4":"27℃~20℃","temp5":"25℃~20℃","tempF1":"80.6℉~64.4℉","tempF2":"78.8℉~64.4℉","tempF3":"80.6℉~66.2℉","tempF4":"80.6℉~68℉","tempF5":"77℉~68℉","weather1":"多云","weather2":"多云","weather3":"晴转多云","weather4":"多云","weather5":"阴","img1":"1","img2":"99","img3":"1","img4":"99","img5":"0","img6":"1","img7":"1","img8":"99","img9":"2","img10":"99","img_single":"1","img_title1":"多云","img_title2":"多云","img_title3":"多云","img_title4":"多云","img_title5":"晴","img_title6":"多云","img_title7":"多云","img_title8":"多云","img_title9":"阴","img_title10":"阴","img_title_single":"多云","wind1":"东风3-4级","wind2":"东风3-4级","wind3":"东风3-4级","wind4":"东南风4-5级","wind5":"东南风转东风4-5级","fx1":"东风","fx2":"东风","fl1":"3-4级","fl2":"3-4级","fl3":"3-4级","fl4":"4-5级","fl5":"4-5级","index":"暂缺","index_d":"暂缺","index48":"暂缺","index48_d":"暂缺","index_uv":"弱","index48_uv":"弱","index_xc":"适宜","index_tr":"很适宜","index_co":"较舒适","st1":"26","st2":"17","st3":"25","st4":"17","st5":"25","st6":"18"}}
{"weatherinfo":{"city":"上海","city_en":"shanghai","date_y":"2010年5月31日","date":"庚寅年四月十八","week":"星期一","fchh":"08","cityid":"101020100","temp1":"27℃~18℃","temp2":"26℃~18℃","temp3":"27℃~19℃","temp4":"27℃~20℃","temp5":"25℃~20℃","tempF1":"80.6℉~64.4℉","tempF2":"78.8℉~64.4℉","tempF3":"80.6℉~66.2℉","tempF4":"80.6℉~68℉","tempF5":"77℉~68℉","weather1":"多云","weather2":"多云","weather3":"晴转多云","weather4":"多云","weather5":"阴","img1":"1","img2":"99","img3":"1","img4":"99","img5":"0","img6":"1","img7":"1","img8":"99","img9":"2","img10":"99","img_single":"1","img_title1":"多云","img_title2":"多云","img_title3":"多云","img_title4":"多云","img_title5":"晴","img_title6":"多云","img_title7":"多云","img_title8":"多云","img_title9":"阴","img_title10":"阴","img_title_single":"多云","wind1":"东风3-4级","wind2":"东风3-4级","wind3":"东风3-4级","wind4":"东南风4-5级","wind5":"东南风转东风4-5级","fx1":"东风","fx2":"东风","fl1":"3-4级","fl2":"3-4级","fl3":"3-4级","fl4":"4-5级","fl5":"4-5级","index":"暂缺","index_d":"暂缺","index48":"暂缺","index48_d":"暂缺","index_uv":"弱","index48_uv":"弱","index_xc":"适宜","index_tr":"很适宜","index_co":"较舒适","st1":"26","st2":"17","st3":"25","st4":"17","st5":"25","st6":"18"}}
导航189天气抓取示例：
Crawler.java
Java 代码
import java.util.TimerTask;
import org.apache.commons.lang.StringUtils;
import org.htmlparser.Parser;
import org.htmlparser.beans.StringBean;
import org.htmlparser.lexer.Lexer;
public abstract class Crawler extends TimerTask {
public String convert2String(String html) {
if (html != null) {
String str = html;
Lexer l = new Lexer(str);
Parser parser = new Parser(l);
StringBean sb = new StringBean();
try {
parser.visitAllNodesWith(sb);
} catch (Exception e) {
return str;
}
String ret = sb.getStrings();
if (ret != null) {
ret = ret.replace("\r", "");
ret = ret.replace("\t", "");
}
return ret;
}
return "";
}
public String formatString(String str) {
if (StringUtils.isBlank(str)) {
return "";
}
str = convert2String(str);
str = str.replaceAll("&", "&");
str = str.replaceAll("\"", "“").replaceAll("<", "").replaceAll("\"", "“");
str = str.trim();
return str;
}
}
import java.util.TimerTask;
import org.apache.commons.lang.StringUtils;
import org.htmlparser.Parser;
import org.htmlparser.beans.StringBean;
import org.htmlparser.lexer.Lexer;
public abstract class Crawler extends TimerTask {
public String convert2String(String html) {
if (html != null) {
String str = html;
Lexer l = new Lexer(str);
Parser parser = new Parser(l);
StringBean sb = new StringBean();
try {
parser.visitAllNodesWith(sb);
} catch (Exception e) {
return str;
}
String ret = sb.getStrings();
if (ret != null) {
ret = ret.replace("\r", "");
ret = ret.replace("\t", "");
}
return ret;
}
return "";
}
public String formatString(String str) {
if (StringUtils.isBlank(str)) {
return "";
}
str = convert2String(str);
str = str.replaceAll("&", "&");
str = str.replaceAll("\"", "“").replaceAll("<", "").replaceAll("\"", "“");
str = str.trim();
return str;
}
}
WeatherCrawler.java
Java 代码
import java.net.URI;
import java.util.Date;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.HttpConnectionParams;
import org.apache.http.params.HttpProtocolParams;
import org.apache.http.util.EntityUtils;
import org.json.JSONException;
import org.json.JSONObject;
import com.dh189.util.Constants;
import com.dh189.util.Util;
public class WeatherCrawler extends Crawler {
private static final String GETURL = "http://m.weather.com.cn/data/";
DefaultHttpClient httpclient = new DefaultHttpClient();
public void start() {
String[] citys = Constants.WEATHERCITYCODE.split(",");
for (String city : citys) {
String url = GETURL + city + ".html";
String val = getWeather(url);
try {
JSONObject json = new JSONObject(val);
JSONObject j = json.getJSONObject("weatherinfo");
String alt1 = j.getString("weather1") + " " + j.getString("wind1");
String alt2 = j.getString("weather2") + " " + j.getString("wind2");
String html = "<div class=\"w_l\"> <div class=\"w_l_u\"> <table style=\"width: 35px;height: 45px;\"> <tr> <td style=\"color: #F24F00\"><b>"
+ j.getString("city")
+ "</b></td> </tr> </table> < /div> <div class=\"w_l_d\"><a href=\"javascript:dingCity(); \" title=\"定制天气预报\" target=\"_self\">定制</a> </div> < /div> <ul> <li class=\"w_r\" style=\"width: 222px;\"> <table style=\"height: 70px;width: 100%;float: left;\"> <tr valign=\"bottom\"> <td><img src=\"images/weather/b"
+ j.getString("img1") + ".gif\" height=\"30\"/></td> <td><img src=\"images/weather/b" + j.getString("img3")
+ ".gif\" height=\"30\"/></td> </tr> <tr> <td width=\"105\" style=\"line-height: 15px;\"><a href=\"http://www.weather.com.cn/html/weather/" + city + ".shtml\" title=\""
+ alt1 + "\">" + Util.getWeekOfDate(new Date()) + ":" + j.getString("temp1") + "<br/>" + j.getString("weather1")
+ "</a></td> <td width=\"105\" style=\"line-height: 15px;\"><a href=\"http://www.weather.com.cn/html/weather/" + city + ".shtml\" title=\"" + alt2 + "\">"
+ Util.getWeekOfDate(Util.dateAddOrReduce(new Date(), 1)) + ":" + j.getString("temp2") + "<br/>" + j.getString("weather2") + "</a></td></tr></table></li></ul>";
String dir = Constants.WEATHERDIR + city + ".html";
Util.writeSortFile(dir, html, false);
} catch (JSONException e) {
e.printStackTrace();
}
}
}
/**
* 处理GET请求，返回整个页面
*
* @param url
* @return
*/
public String getWeather(String url) {
String content = null;
try {
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpConnectionParams.setConnectionTimeout(httpclient.getParams(), 30 * 1000);
HttpConnectionParams.setSoTimeout(httpclient.getParams(), 30 * 1000);
//伪装成火狐浏览器
HttpProtocolParams.setUserAgent(httpclient.getParams(), "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9");
HttpGet httpget = new HttpGet();
content = "";
httpget.setURI(new URI(url));
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
content = null;
if (entity != null) {
content = EntityUtils.toString(entity);
httpget.abort();
httpclient.getConnectionManager().shutdown();
}
} catch (Exception e) {
e.printStackTrace();
}
return content;
}
@Override
public void run() {
start();
}
}
import java.net.URI;
import java.util.Date;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.HttpConnectionParams;
import org.apache.http.params.HttpProtocolParams;
import org.apache.http.util.EntityUtils;
import org.json.JSONException;
import org.json.JSONObject;
import com.dh189.util.Constants;
import com.dh189.util.Util;
public class WeatherCrawler extends Crawler {
private static final String GETURL = "http://m.weather.com.cn/data/";
DefaultHttpClient httpclient = new DefaultHttpClient();
public void start() {
String[] citys = Constants.WEATHERCITYCODE.split(",");
for (String city : citys) {
String url = GETURL + city + ".html";
String val = getWeather(url);
try {
JSONObject json = new JSONObject(val);
JSONObject j = json.getJSONObject("weatherinfo");
String alt1 = j.getString("weather1") + " " + j.getString("wind1");
String alt2 = j.getString("weather2") + " " + j.getString("wind2");
String html = "<div class=\"w_l\"> <div class=\"w_l_u\"> <table style=\"width: 35px;height: 45px;\"> <tr> <td style=\"color: #F24F00\"><b>"
+ j.getString("city")
+ "</b></td> </tr> </table> </div> <div class=\"w_l_d\"><a href=\"javascript:dingCity();\" title=\"定制天气预报\" target=\"_self\">定制</a> </div> </div> <ul> <li class=\"w_r\" style=\"width: 222px;\"> <table style=\"height: 70px;width: 100%;float: left;\"> <tr valign=\"bottom\"> <td><img src=\"images/weather/b"
+ j.getString("img1") + ".gif\" height=\"30\"/></td> <td><img src=\"images/weather/b" + j.getString("img3")
+ ".gif\" height=\"30\"/></td> </tr> <tr> <td width=\"105\" style=\"line-height: 15px;\"><a href=\"http://www.weather.com.cn/html/weather/" + city + ".shtml\" title=\""
+ alt1 + "\">" + Util.getWeekOfDate(new Date()) + ":" + j.getString("temp1") + "<br/>" + j.getString("weather1")
+ "</a></td> <td width=\"105\" style=\"line-height: 15px;\"><a href=\"http://www.weather.com.cn/html/weather/" + city + ".shtml\" title=\"" + alt2 + "\">"
+ Util.getWeekOfDate(Util.dateAddOrReduce(new Date(), 1)) + ":" + j.getString("temp2") + "<br/>" + j.getString("weather2") + "</a></td></tr></table></li></ul>";
String dir = Constants.WEATHERDIR + city + ".html";
Util.writeSortFile(dir, html, false);
} catch (JSONException e) {
e.printStackTrace();
}
}
}
/**
* 处理GET请求，返回整个页面
*
* @param url
* @return
*/
public String getWeather(String url) {
String content = null;
try {
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpConnectionParams.setConnectionTimeout(httpclient.getParams(), 30 * 1000);
HttpConnectionParams.setSoTimeout(httpclient.getParams(), 30 * 1000);
//伪装成火狐浏览器
HttpProtocolParams.setUserAgent(httpclient.getParams(), "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9");
HttpGet httpget = new HttpGet();
content = "";
httpget.setURI(new URI(url));
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
content = null;
if (entity != null) {
content = EntityUtils.toString(entity);
httpget.abort();
httpclient.getConnectionManager().shutdown();
}
} catch (Exception e) {
e.printStackTrace();
}
return content;
}
@Override
public void run() {
start();
}
}
分享到：
发表评论
-
npm安装教程
2020-11-23 16:09 611一、使用之前，我们先来掌握3个东西是用来干什么的。 npm ... -
手机上万能返回上一页代替history.go(-1)
2016-05-13 11:25 783最近在做wap2.0的手机wap,写返回上一页的时候,开始 ... -
3.nodejs nodejs的mysql模块
2016-04-12 22:20 520安装nodejs的mysql模块首先要选定一个数据库，我选择 ... -
2.nodejs 使用Express4.x搭建第一个Web应用
2016-04-12 09:36 562Express4.x 是一个开源的node.js项目框架 ... -
1.nodejs windows 下安装nodejs
2016-04-12 09:34 3811.下载： 在nodejs的官方网站下载mis文件。ht ... -
Nodejs学习
2016-04-11 17:21 500将介绍如何利Javascript做为服务端脚本，通过Node ... -
HTML5做手机站页面字体显示很小的解决方法
2016-05-28 23:31 697HTML5做手机站需要加上一部分代码，不然字体会显示很小的 ... -
jquery 校验
2015-07-07 11:16 0EasyUI 验证框使用方法: ... -
JS、jquery实现几分钟前、几小时前、几天前等时间差显示效果的代码实例分享
2015-05-20 09:49 565一、javascript函数实现: //JavaS ... -
//from提交表单 使用ajax异步读取，回调函数
2015-04-24 18:05 0<%@ page language="ja ... -
js获取带checkbox行的记录的Id值的两个方法
2015-04-09 14:44 655第一种： function updateProcess() ... -
基于注解的SpringMVC简单介绍
2015-04-03 11:58 568SpringMVC是一个基于DispatcherServle ... -
Linux环境下Nginx的安装
2014-03-28 13:26 458Linux下命令： /usr/local/nginx/s ... -
自用打印（二）
2014-03-19 17:30 541<html> <title> ... -
自用打印方法（一）
2014-03-19 17:25 435经过验证的，自用的。 ... -
各种微博分享代码
2014-03-19 15:56 882<HTML> <HEAD> ... -
JSP页面打印（二）
2014-03-18 11:04 736<HTML><HEAD><T ... -
JSP页面打印方法(一)
2014-03-18 10:57 3080JSP页面打印方法 方法一：function Print ... -
jquery和js初始化加载的多种方法
2014-01-20 14:48 877jquery和js初始化加载页面的区别： jquery：等 ... -
js自动刷新当前页面
2013-12-19 14:09 763reload 方法，该方法强 ...
相关推荐
java,android,天气预报,返回7天内指定城市的天气预报
2. 服务器端提供http（get）获取方式获取json数据的api例子，是用php写的。 4. android的apk给你说完成http数据请求和json获取解析。 5. 把index.php部署到你的php服务器. 6. 修改MainActivity.java中的 final ...
java获取电子称重量数据 称重收银一体机 收银电子秤 java获取电子称重量数据 读取com口电子秤 读取串口电子秤
处理嵌套json格式的数据。。。 public static void main(String[] args) { // 官方API http://www.json.org/java/ /* 购物车中信息 goods_cart={cart_1325036696007:{goods_id:"100015",goods_name:"澳大利亚...
echarts 全国地图json数据
2022最新json数据，2022年8月9日更新，因为最近需要用到中国省市列表的JSON数据 本来想把县也包含进去的，但是数量太多了~ 中国总共有23个省、5个自治区、4个直辖市、2个特别行政区。 然后决定到市级别(包含县级市) ...
将中国天气网近六天（因为有些城市提供的接口只有五天数据，所以我暂时取5天，程序里依然是六天的标准）的天气情况抓取，用JSON解析，存入数据库，数据库是两张表，城市表和天气表。程序可以运行，两个表都能读入...
该项目是在Qt5环境下开发的，利用天气预报接口获取天气数据并转成Json格式，输入城市代码就能获取该城市的当天的天气预报和未来一个星期的天气，界面美观大方。
用Java编写的天气预报软件，可以显示所在地未来5天的温度和天气状况以及当天的生活小提示。 首次运行会产生一个 city.txt 的UTF编码文件纪录城市信息。软件无需安装。可以在任何支持jre 1.6 以上的环境中运行。 ...
【C++ QT项目5】-基于HTTP与JSON数据流的天气预报界面设计
Java网络天气预报程序，一个天气软件,提供未来5天的天气预报同时提供当天的生活小提示。首次运行会产生一个 city.txt 的UTF编码文件纪录城市信息。软件无需安装。可以在任何支持jre 1.6 以上的环境中运行。这个软件...
这是一个天气软件,提供未来5天的天气预报同时提供当天的生活小提示。本软件是免费软件，允许在非商业环境下完全自由的使用。 带源码。
echarts地图json数据(含全国和省以及全国、省、地市)最全，绝对有县级数据，非常完整的省市区县json数据，本人也是找了很久才找到
JAVA,工具类,xml,json,MD5,加密,解密 有JAR包 注释清晰，绝对给力！ 标准代码
cJSON使用的例子，介绍cJSON...5、遇到JSON数据值是布尔值，调用cJSON_IsFalse、cJSON_IsTrue可以判断是false还是true； 6、遇到JSON数据值是null，调用cJSON_IsNull可以判断是否为null。 7、最后，使用完记得释放内存
json转yolo v5数据集训练格式，需要改训练分类和文件地址
天气预报WEB服务JAVA实例 axis1.4 天气预报WEB服务JAVA实例使用接口：2400多个城市5日天气预报 WEB服务Endpoint: ...
一个Java库，它将JSON5或JSON格式解析为Java对象的普通树（使用Map<>作为字典，而使用List<>作为数组）。 它在JitPack上直播。 在线演示; 它显示了用于将JSON5对象转换为纯JSON的库。 解析器是从参考...
第25章 天气预报模块 641 25.1 天气预报模块概述 642 25.1.1 天气预报概述 642 ...25.4 查询各城市未来5天的天气预报、指数预报 653 25.5 获取中国气象局未来72小时城市天气预报 655 25.6 程序调试 660