You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Joshua Fan <jo...@gmail.com> on 2019/05/17 03:47:03 UTC
run python job with flink 1.7
Hi all
When I run the python example in flink 1.7, it always got a excepthon.
The command is: ./bin/pyflink.sh ./examples/python/streaming/word_count.py
The return message is:
2019-05-17 11:43:22,900 INFO org.apache.hadoop.yarn.client.RMProxy
- Connecting to ResourceManager at
data01.hj.shbt.qihoo.net/10.203.82.17:8832
Starting execution of program
Traceback (most recent call last):
File "/tmp/flink_plan_0a1aed4b-4155-4cfa-b2ba-8083a4a61f6e/plan.py", line
21, in <module>
from org.apache.flink.api.common.functions import FlatMapFunction
ImportError: No module named org.apache.flink.api.common.functions
Failed to run plan: Plan file caused an error. Check log-files for details.
The program didn't contain a Flink job. Perhaps you forgot to call
execute() on the execution environment.
Can not find any help in google.
Appreciate your help very much.
Sincerely
Joshua
Re: run python job with flink 1.7
Posted by Joshua Fan <jo...@gmail.com>.
when I look into the log file, it turned out that flink cannot get the plan
when create the plan file, full log message is below.
107 2019-05-17 12:24:56.950 [main] ERROR
org.apache.flink.python.api.PythonPlanBinder - Failed to run plan.
108 java.lang.RuntimeException: Plan file caused an error. Check log-files
for details.
109 at
org.apache.flink.python.api.streaming.plan.PythonPlanStreamer.preparePlanMode(PythonPlanStreamer.java:107)
110 at
org.apache.flink.python.api.PythonPlanBinder.runPlan(PythonPlanBinder.java:178)
111 at
org.apache.flink.python.api.PythonPlanBinder.main(PythonPlanBinder.java:98)
112 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
113 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
114 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
115 at java.lang.reflect.Method.invoke(Method.java:498)
116 at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
117 at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
118 at
org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:427)
119 at
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:813)
120 at
org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:287)
121 at
org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
122 at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
123 at
org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
124 at java.security.AccessController.doPrivileged(Native Method)
125 at javax.security.auth.Subject.doAs(Subject.java:422)
126 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
127 at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
128 at
org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
129 2019-05-17 12:24:56.951 [main] INFO
org.apache.flink.runtime.rest.RestClient - Shutting down rest endpoint.
130 2019-05-17 12:24:56.959 [main] INFO
org.apache.flink.runtime.rest.RestClient - Rest endpoint shutdown complete.
I am not familiar with python. Thanks for your help.
On Fri, May 17, 2019 at 11:47 AM Joshua Fan <jo...@gmail.com> wrote:
> Hi all
>
> When I run the python example in flink 1.7, it always got a excepthon.
>
> The command is: ./bin/pyflink.sh ./examples/python/streaming/word_count.py
>
> The return message is:
> 2019-05-17 11:43:22,900 INFO org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at
> data01.hj.shbt.qihoo.net/10.203.82.17:8832
> Starting execution of program
> Traceback (most recent call last):
> File "/tmp/flink_plan_0a1aed4b-4155-4cfa-b2ba-8083a4a61f6e/plan.py",
> line 21, in <module>
> from org.apache.flink.api.common.functions import FlatMapFunction
> ImportError: No module named org.apache.flink.api.common.functions
> Failed to run plan: Plan file caused an error. Check log-files for details.
>
> The program didn't contain a Flink job. Perhaps you forgot to call
> execute() on the execution environment.
>
> Can not find any help in google.
> Appreciate your help very much.
>
> Sincerely
> Joshua
>