You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by Jacob <17...@163.com> on 2020/12/10 06:30:27 UTC
Flink 1.11.2 on yarn 提交job失败(附完整日志)
大家好，有一问题困扰许久。*Flink1.7.2升级到1.11.2
job无法提交*【现象】可以提交demo的job（WordCount）,无法提交自定义的job，自己的job有操作Hadoop，读ORC，写hive等操作提交自己的job，各种日志如下
<http://apache-flink.147419.n8.nabble.com/file/t1162/jobManager-logs_LI.jpg>
<http://apache-flink.147419.n8.nabble.com/file/t1162/jobManager-Stdout.png>
<http://apache-flink.147419.n8.nabble.com/file/t1162/LogList.png>
<http://apache-flink.147419.n8.nabble.com/file/t1162/LogList-jobmanager.jpg>
Flink 配置文件：env.java.home:
/usr/java/jdk1.8.0_162yarn.taskmanager.env.JAVA_HOME:
/usr/java/jdk1.8.0_162containerized.master.env.JAVA_HOME:
/usr/java/jdk1.8.0_162containerized.taskmanager.env.JAVA_HOME:
/usr/java/jdk1.8.0_162jobmanager.rpc.address: localhostjobmanager.rpc.port:
6123jobmanager.memory.process.size:
1600mjobmanager.execution.failover-strategy:
regiontaskmanager.memory.process.size: 1728mtaskmanager.numberOfTaskSlots:
4parallelism.default: 10yarn.application-attempts: 10state.backend:
filesystemstate.checkpoints.dir:
hdfs:///user/flink/flink-checkpointsstate.savepoints.dir:
hdfs:///user/flink/flink-savepointsstate.backend.incremental:
falsestate.backend.fs.memory-threshold: 1024state.checkpoints.num-retained:
3restart-strategy: fixed-delayrestart-strategy.fixed-delay.attempts:
1000restart-strategy.fixed-delay.delay: 30 s项目POM文件<?xml version="1.0"
encoding="UTF-8"?> 4.0.0 0.0.1-SNAPSHOT UTF-8
1.8 1.8 1.11.2 2.11 0.10 1.1.0-cdh5.8.3
org.apache.orc orc-core 1.5.4 org.apache.hadoop
hadoop-common org.apache.hadoop hadoop-hdfs
org.slf4j slf4j-api org.redisson
redisson 3.12.4 org.slf4j
slf4j-api
com.microsoft.sqlserver mssql-jdbc 6.2.2.jre8
org.apache.hive hive-exec ${hive.version}
* *
ch.qos.logback logback-classic 1.2.3
org.slf4j slf4j-api
com.sun.mail javax.mail 1.6.2
org.apache.flink
flink-connector-kafka-${kafka.version}_${scala.binary.version}
${flink.version} org.apache.flink
flink-streaming-java_${scala.binary.version} ${flink.version}
provided org.apache.flink
flink-connector-filesystem_${scala.binary.version} ${flink.version}
junit junit 4.12 test
org.apache.hive hive-jdbc 1.1.0
org.slf4j slf4j-api
org.apache.hadoop hadoop-common 2.6.0-cdh5.8.3
org.slf4j slf4j-api 。。。。。。。。。。。。。。。。。。。
搞了很久了不知道原因了，开始压根提交不上去，看不到这个portal，后来发现Hadoop一些jar包冲突，已排除，见pom求教，谢谢！
--
Sent from: http://apache-flink.147419.n8.nabble.com/