Eclipse 导入 Hadoop 源码
2024-10-18 23:34:20
1.准备工作
- jdk:
- eclipse:
- Maven:
- libprotoc :https://developers.google.com/protocol-buffers/
- hadoop:http://www.apache.org/dyn/closer.cgi/hadoop/common/
添加开源中国maven库:http://maven.oschina.net/home.html
maven\conf\settings.xml
<localRepository>path</localRepository>
<localRepository>path</localRepository>
<mirrors>
<mirror>
<id>nexus-osc</id>
<mirrorOf>*</mirrorOf>
<name>Nexus osc</name>
<url>http://maven.oschina.net/content/groups/public/</url>
</mirror>
</mirrors>
2.导入
- 将hadoop源码解压到一个目录,注意目录层次不要太深,否则可能无法解压。
- 进入hadoop-maven-plugins文件夹,执行 mvn install
- 返回源码根目录,执行 mvn eclipse:eclipse –DskipTests
- eclipse在任意目录创建新的WorkSpace
- eclipse设置Maven:window->preference->maven->{Installations...;user Settings:maven\conf\settings.xml}
- eclipse:File->inport->Existing Projects into WorkSpace->Hadoop源码根目录
3.错误处理
- maven下载pom失败->重新操作
- hadoop-streaming中build path错误->Java Build Path->Source:
- 删除...hadoop-yarn-server-resourcemanager/conf
- Link Source:源码根目录/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf,再随便起个名字;inclusion patterns:capacity-scheduler.xml;exclusion patters:**/*.java
- org.apache.hadoop.io.serializer.avro.TestAvroSerialization
- 下载avro-tools-1.7.4.jar:http://archive.apache.org/dist/avro/avro-1.7.4/java/
- 进入目录:源码根目录\hadoop-common-project\hadoop-common\src\test\avro
- java -jar path/to/avro-tools-1.7..jar compile schema avroRecord.avsc ..\java
- eclipse 刷新
- org.apache.hadoop.ipc.protobuf.TestProtos
- 进入目录:源码根目录\hadoop-common-project\hadoop-common\src\test\proto
- protoc --java_out=../java *.proto
- eclipse 刷新
- project->clean..->clean all projects & Build the entire workspace
最新文章
- 简单的学习心得:网易云课堂Android开发第四章服务、广播与酷特性
- 解决.NET Core中MailKit无法使用阿里云邮件推送服务的问题
- 16s及宏基因组测序公司资源--20161104
- cookie 的“Value”=“xxxxx,xxxxx”部分无效
- Linux ncurses编写 FlapyBird 第一步
- redmine 2.5.2 安装后邮件无法发送
- BaiduMap开发,获取公交站点信息。
- UVa 127 - ";Accordian"; Patience
- hadoop1中hdfs原理详解
- Android - 获取字符串长度的宽度
- MD5加密 Java源代码
- ansible Strategies
- C#调用TSC条码打印机打印二维码
- 1023. Have Fun with Numbers (20)
- python学习笔记-Day2 Numpy数组
- 在IDEA中实战Git-branch
- linux 压缩命令
- 一种通过async/await实现函数同步执行的方式
- signal_windows
- 部署DNS从服务器