文章目录
Error: Could not find or load main class org.apache.hadoop.mapred.YarnChild
解决方案:
- 1, 在命令行执行,复制信息。冒号用逗号代替
hadoop classpath
- 2 , 编辑yarn-site.xml,添加信息:
设置完毕后用xsync命令分发到其他服务器上并使用</configuration> <property> <name>namemapreduce.reduce.env/name</name> <value>valueHADOOP_MAPRED_HOME=/opt/module/hadoop- 3.1.3/value</value> </property> <property> <name>yarn.application.classpath</name> <value>复制的Hadoop classpath信息,如 /opt/hadoop-3.1.0/etc/hadoop, /opt/hadoop-3.1.0/share/hadoop/common/*, /opt/hadoop-3.1.0/share/hadoop/common/lib/*, /opt/hadoop-3.1.0/share/hadoop/hdfs/*, /opt/hadoop-3.1.0/share/hadoop/hdfs/lib/*, /opt/hadoop-3.1.0/share/hadoop/mapreduce/*, /opt/hadoop-3.1.0/share/hadoop/mapreduce/lib/*, /opt/hadoop-3.1.0/share/hadoop/yarn/*, /opt/hadoop-3.1.0/share/hadoop/yarn/lib/* </value> </property> </configuration>
start-all.sh
命令重启Hadoop集群,
成功运行。
ERROR:Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
解决方案:
- 1,编辑mapred-site.xml,添加以下信息
<property> <name>yarn.app.mapreduce.am.env</name> <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value> </property> <property> <name>mapreduce.map.env</name> <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value> </property> <property> <name>mapreduce.reduce.env</name> <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value> </property>
- 2, 设置完毕后用xsync命令分发到其他服务器上并使用
start-all.sh
命令重启Hadoop集群,
成功运行。
其他
如果仍旧报错,请参考可成功运行的配置:
mapred-site.xml配置如下
<configuration>
<!-- 指定 MapReduce 程序运行在 Yarn 上 -->
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>namemapreduce.reduce.env/name</name>
<value>valueHADOOP_MAPRED_HOME=/opt/module/hadoop-3.1.3/value</value>
</property>
<property>
<name>yarn.app.mapreduce.am.env</name>
<value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
<property>
<name>mapreduce.map.env</name>
<value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
<property>
<name>mapreduce.reduce.env</name>
<value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
</configuration>
yarn-site.xml 配置如下
<configuration>
<!-- 指定 MR 走 shuffle -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<!-- 指定 ResourceManager 的地址-->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop103</value>
</property>
<!-- 环境变量的继承 -->
<property>
<name>yarn.nodemanager.env-whitelist</name>
<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CO
NF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAP
RED_HOME</value>
</property>
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.application.classpath</name>
<value>
/opt/module/hadoop-3.1.3/etc/hadoop,/opt/module/hadoop-3.1.3/share/hadoop/common/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/common/*,/opt/module/hadoop-3.1.3/share/hadoop/hdfs,/opt/module/hadoop-3.1.3/share/hadoop/hdfs/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/hdfs/*,/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/*,/opt/module/hadoop-3.1.3/share/hadoop/yarn,/opt/module/hadoop-3.1.3/share/hadoop/yarn/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/yarn/*
</value>
</property>
</configuration>
运行成功截图: