You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by whh_960101 <wh...@163.com> on 2020/10/26 08:38:50 UTC

提交pyflink作业到YARN集群报错

Hi，各位大佬，     想请教一下，我使用flink run -m yarn-cluster -p 4 -py myjob.py,报错java.lang.RuntimeException: Found 0 flink-python jar.
 at org.apache.flink.client.program.PackagedProgram.getJobJarAndDependencies(PackagedProgram.java:263)
 at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:140)
 at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:68)
 at org.apache.flink.client.program.PackagedProgram$Builder.build(PackagedProgram.java:590)
 at org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:758)
 at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:250)
 at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:968)
 at org.apache.flink.client.cli.CliFrontend.lambda$main$12(CliFrontend.java:1042)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
 at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
 at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1042)

没有发现flink-python jar，这个jar在哪里可以找见，site-packages/pyflink里面，还是自己构建，目前本机已经安装pyflink 1.11.1

Re: 提交pyflink作业到YARN集群报错

Posted by Dian Fu <di...@gmail.com>.
看一下目录site-packages/pyflink/opt，里面是否有名为flink-python的jar

> 在 2020年10月26日，下午4:38，whh_960101 <wh...@163.com> 写道：
> 
> Hi，各位大佬，     想请教一下，我使用flink run -m yarn-cluster -p 4 -py myjob.py,报错java.lang.RuntimeException: Found 0 flink-python jar.
> at org.apache.flink.client.program.PackagedProgram.getJobJarAndDependencies(PackagedProgram.java:263)
> at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:140)
> at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:68)
> at org.apache.flink.client.program.PackagedProgram$Builder.build(PackagedProgram.java:590)
> at org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:758)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:250)
> at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:968)
> at org.apache.flink.client.cli.CliFrontend.lambda$main$12(CliFrontend.java:1042)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1042)
> 
> 没有发现flink-python jar，这个jar在哪里可以找见，site-packages/pyflink里面，还是自己构建，目前本机已经安装pyflink 1.11.1