[jira] [Created] (FLINK-25253) An ClassNotFoundException of missing Hadoop class occurred when submitting to yarn

["ghost synth (Jira)" <ji...@apache.org>]

ghost synth created FLINK-25253:
-----------------------------------

             Summary: An ClassNotFoundException of missing Hadoop class occurred when submitting to yarn
                 Key: FLINK-25253
                 URL: https://issues.apache.org/jira/browse/FLINK-25253
             Project: Flink
          Issue Type: Bug
          Components: Deployment / YARN
    Affects Versions: 1.14.0
         Environment: Environment version:
Hadoop 3.1.1
Hive 3.1.1
Flink 1.4.0
kafka 2.6.1

 
            Reporter: ghost synth
         Attachments: FlinkPlaySubmit.scala, flink lib.png, original-TropicalaLink-1.0-SNAPSHOT.jar, pom.xml, submit_log.log

I use the Hive Table connector to write hive from Kafka and submit to yarn successfully, but it will always report during execution
*Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConf*
Before submitting, I executed "export HADOOP_CLASSPATH=`hadoop classpath`" to import hadoop dependencies.

I found in the JM log that the classpath already contains Hadoop dependencies, but an exception still occurs

The original jar that I submitted only contains code and does not contain dependencies,The program loads dependencies from the hadoop path and lib under the flink directory

The attachment contains flink lib, code, jar and JM log

Thanks



--
This message was sent by Atlassian Jira
(v8.20.1#820001)