You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by yujianbo <15...@163.com> on 2021/01/26 11:06:53 UTC
flink启动任务时异常，加载不到flink/lib目录下的包
环境： flink 1.12.0
报这个错：报加载不到这个log4j-slf4j-impl-2.12.1.jar包，但是我的lib目录下是有这个包的：
$ cd lib/
$ ll
-rw-r--r-- 1 yujianbo yujianbo 91554 1月 26 18:44 flink-csv-1.12.0.jar
-rw-r--r-- 1 yujianbo yujianbo 114119885 1月 26 18:45
flink-dist_2.11-1.12.0.jar
-rw-r--r-- 1 yujianbo yujianbo 136664 1月 26 18:44 flink-json-1.12.0.jar
-rw-r--r-- 1 yujianbo yujianbo 7709742 1月 26 18:44
flink-shaded-zookeeper-3.4.14.jar
-rw-r--r-- 1 yujianbo yujianbo 36147824 1月 26 18:44
flink-table_2.11-1.12.0.jar
-rw-r--r-- 1 yujianbo yujianbo 40286363 1月 26 18:45
flink-table-blink_2.11-1.12.0.jar
-rw-r--r-- 1 yujianbo yujianbo 67114 1月 26 18:44
log4j-1.2-api-2.12.1.jar
-rw-r--r-- 1 yujianbo yujianbo 276771 1月 26 18:44 log4j-api-2.12.1.jar
-rw-r--r-- 1 yujianbo yujianbo 1674433 1月 26 18:44 log4j-core-2.12.1.jar
-rw-r--r-- 1 yujianbo yujianbo 23518 1月 26 18:44
log4j-slf4j-impl-2.12.1.jar
The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The main method
caused an error: Could not deploy Yarn job cluster.
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:330)
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198)
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
at
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:743)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:242)
at
org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:971)
at
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1047)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
at
org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1047)
Caused by: org.apache.flink.client.deployment.ClusterDeploymentException:
Could not deploy Yarn job cluster.
at
org.apache.flink.yarn.YarnClusterDescriptor.deployJobCluster(YarnClusterDescriptor.java:460)
at
org.apache.flink.client.deployment.executors.AbstractJobClusterExecutor.execute(AbstractJobClusterExecutor.java:70)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1940)
at
org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:128)
at
org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:76)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1822)
at com.xm4399.yhzx.task.VersionTest.main(VersionTest.java:118)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:316)
... 11 more
Caused by:
org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The
YARN application unexpectedly switched to state FAILED during deployment.
Diagnostics from YARN: Application application_1611303948765_0050 failed 1
times (global limit =2; local limit is =1) due to AM Container for
appattempt_1611303948765_0050_000001 exited with exitCode: -1000
*Failing this attempt.Diagnostics: [2021-01-26 19:00:46.608]File does not
exist:
hdfs://4399cluster/user/hadoop/.flink/application_1611303948765_0050/lib/log4j-slf4j-impl-2.12.1.jar
java.io.FileNotFoundException: File does not exist:
hdfs://4399cluster/user/hadoop/.flink/application_1611303948765_0050/lib/log4j-slf4j-impl-2.12.1.jar*
at
org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1729)
at
org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1722)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1737)
at
org.apache.hadoop.yarn.util.FSDownload.verifyAndCopy(FSDownload.java:271)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:68)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:415)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:412)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:412)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:247)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:240)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:228)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
--
Sent from: http://apache-flink.147419.n8.nabble.com/
Re: flink启动任务时异常，加载不到flink/lib目录下的包
Posted by yujianbo <15...@163.com>.
我已经解决了，是因为我跑的任务jar包里的resources目录下的一个文件hdfs-site.xml是上一份集群的，我把这个去掉就可以了。
--
Sent from: http://apache-flink.147419.n8.nabble.com/