Hadoop本地编写的jar包放到集群执行时报错处理

错误描述：

020-03-24 22:45:23,204 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code from container container_1585061045134_0001_01_000001 is : 1

2020-03-24 22:45:23,205 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exception from container-launch with container ID: container_1585061045134_0001_01_000001 and exit code: 1

ExitCodeException exitCode=1:

	at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008)

	at org.apache.hadoop.util.Shell.run(Shell.java:901)

	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213)

	at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:294)

	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:491)

	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:303)

	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:101)

	at java.util.concurrent.FutureTask.run(FutureTask.java:266)

	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

	at java.lang.Thread.run(Thread.java:745)

错误原因：查看日志发现有这样一条信息：conf.Configuration: resource-types.xml not found    还有这个： bad substitution  查了半天发现是提交任务出错了！
解决办法：
hadoop3的配置和hadoop2的配置基本相同，但是，在hadoop3中还需要在mapred-site.xml上新增一些东西：没有添加以下内容会找不到各种各样的类！添加之后问题解决。

<property>

  <name>yarn.app.mapreduce.am.env</name>

  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>

</property>

<property>

  <name>mapreduce.map.env</name>

  <value>HADOOP_MAPRED_HOME=${HADOP_HOME}</value>

</property>

<property>

  <name>mapreduce.reduce.env</name>

  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>

</property>

 <property>

        <name>mapreduce.application.classpath</name>

        <value>

            ${HADOOP_HOME}/etc/hadoop,

            ${HADOOP_HOME}/share/hadoop/common/*,

            ${HADOOP_HOME}/share/hadoop/common/lib/*,

            ${HADOOP_HOME}/share/hadoop/hdfs/*,

            ${HADOOP_HOME}/share/hadoop/hdfs/lib/*,

            ${HADOOP_HOME}/share/hadoop/mapreduce/*,

            ${HADOOP_HOME}/share/hadoop/mapreduce/lib/*,

            ${HADOOP_HOME}/share/hadoop/yarn/*,

            ${HADOOP_HOME}/share/hadoop/yarn/lib/*

        </value>
 </property>

巴特西

Hadoop本地编写的jar包放到集群执行时报错处理

最新文章

热门文章