Java爬虫——常用的maven依赖
2024-10-14 18:08:39
java实现爬虫常用的第三方包:
- httpclient,for http
- jsoup,for dom
- rhino,for js
- jackson,for json
pom.xml摘录
<dependencies> <!-- simulate web browser -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.7</version>
</dependency> <!-- parse DOM -->
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.11.3</version>
</dependency> <!-- jackson -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.9.8</version>
</dependency> <!-- parse javascript -->
<dependency>
<groupId>org.mozilla</groupId>
<artifactId>rhino</artifactId>
<version>1.7.10</version>
</dependency> <!-- simulate client action -->
<dependency>
<groupId>net.sourceforge.htmlunit</groupId>
<artifactId>htmlunit</artifactId>
<version>2.33</version>
</dependency> <!-- upgrade junit to junit4 -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12<!-- default is v3.8.1 --></version>
<scope>test</scope>
</dependency> <!-- log -->
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-api -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-log4j12 -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.25</version>
<!-- <scope>test</scope> -->
</dependency> </dependencies>
启用log4j基本配置,在main方法中加入语句:
public static void main(String[] args) { //启用log4j基本配置
//不想去写配置文件,可以用Java基本配置
BasicConfigurator.configure();
//...
}
最新文章
- 鼠标右击.exe的程序出现闪退(桌面重启)怎么办
- XPath 实例
- poj3311 Hie with the Pie (状态压缩dp,旅行商)
- _jobdu_1001
- Redis Sets
- UCOS-消息队列(学习笔记)
- [九度OJ]1011.最大连续子序列
- 【hdu-2588】GCD(容斥定理+欧拉函数+GCD()原理)
- HDU 5348 MZL&#39;s endless loop
- 描述下@Component,@Repository,@Service,@Scope,@Autowired,@Inject,@Value标记的作用
- 使用DD 创建SWAP
- ubuntu18.04搭建nfs
- Exception异常转String【转】
- spring cloud配置中心属性加密处理
- [Guitar self-practising] 【吉他练习王-节奏练习】曲目1 基本扫弦节奏练习
- hdu1540 Tunnel Warfare 线段树/树状数组
- centos 磁盘清理 /dev/vda1系统盘满了
- vue学习视频网址(各种前端视频 网址)
- intelliJ 打包jar的多种方式
- Java使用String类格式化当前日期