You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Ashish Raghav <as...@corecompete.com> on 2020/05/28 11:35:44 UTC
Issue while submitting python beam pipeline on flink cluster - local
Hi Guys,
I have another issue when I submit the python beam pipeline ( wordcount example provided by apache beam team) directly on flink cluster running local.
PipelineOptions are :
"--runner=FlinkRunner",
"--flink_version=1.9",
"--environment_type=LOOPBACK",
"--flink_master=192.168.99.100:8081",
versions:
Apache Beam SDK=2.21.0
python=3.7.6
Flink=1.9.3
Please suggest , if I am missing something.
Ashish Raghav
RE: Issue while submitting python beam pipeline on flink cluster -
local
Posted by Ashish Raghav <as...@corecompete.com>.
I can set that up. Can I test on linux?
Ashish Raghav | DE
Core Compete | Ashish.Raghav@corecompete.com
Accelerating Cloud Analytics
-----Original Message-----
From: Maximilian Michels <mx...@apache.org>
Sent: 28 May 2020 17:34
To: user@beam.apache.org
Subject: Re: Issue while submitting python beam pipeline on flink cluster - local
EXTERNAL EMAIL
Do not click links or open attachments unless you recognise the sender and know the content is safe. Report suspicious email to infosec@corecompete.com.
Potentially a Windows issue. Do you have a Unix environment for testing?
On 28.05.20 13:35, Ashish Raghav wrote:
> Hi Guys,
>
>
>
> I have another issue when I submit the python beam pipeline (
> wordcount example provided by apache beam team) directly on flink
> cluster running local.
>
>
>
> PipelineOptions are :
>
> "--runner=FlinkRunner",
>
> "--flink_version=1.9",
>
> "--environment_type=LOOPBACK",
>
> "--flink_master=192.168.99.100:8081",
>
>
>
> versions:
>
> Apache Beam SDK=2.21.0
> python=3.7.6
>
> Flink=1.9.3
>
> Please suggest , if I am missing something.
>
>
>
> *Ashish Raghav *
>
>
>
Re: Issue while submitting python beam pipeline on flink cluster -
local
Posted by Ashish Raghav <as...@corecompete.com>.
Hi Michels,
Can you help me with a setup required on gcloud? Since i am not able to run FlinkRunner locally , I want to try over cloud and see if it works there.
Do you have a doc for that??
Fyi, In another email chain, I am discussing about issues while running PortableRunner. So, the python pipeline works on beam but not if I want to submit on flink via PortableRunner or FlinkRunner.
Thanks.
________________________________
From: Maximilian Michels <mx...@apache.org>
Sent: Thursday, May 28, 2020, 5:34 PM
To: user@beam.apache.org
Subject: Re: Issue while submitting python beam pipeline on flink cluster - local
EXTERNAL EMAIL
Do not click links or open attachments unless you recognise the sender and know the content is safe. Report suspicious email to infosec@corecompete.com.
Potentially a Windows issue. Do you have a Unix environment for testing?
On 28.05.20 13:35, Ashish Raghav wrote:
> Hi Guys,
>
>
>
> I have another issue when I submit the python beam pipeline ( wordcount
> example provided by apache beam team) directly on flink cluster running
> local.
>
>
>
> PipelineOptions are :
>
> "--runner=FlinkRunner",
>
> "--flink_version=1.9",
>
> "--environment_type=LOOPBACK",
>
> "--flink_master=192.168.99.100:8081",
>
>
>
> versions:
>
> Apache Beam SDK=2.21.0
> python=3.7.6
>
> Flink=1.9.3
>
> Please suggest , if I am missing something.
>
>
>
> *Ashish Raghav *
>
>
>
Re: Issue while submitting python beam pipeline on flink cluster -
local
Posted by Maximilian Michels <mx...@apache.org>.
Potentially a Windows issue. Do you have a Unix environment for testing?
On 28.05.20 13:35, Ashish Raghav wrote:
> Hi Guys,
>
>
>
> I have another issue when I submit the python beam pipeline ( wordcount
> example provided by apache beam team) directly on flink cluster running
> local.
>
>
>
> PipelineOptions are :
>
> "--runner=FlinkRunner",
>
> "--flink_version=1.9",
>
> "--environment_type=LOOPBACK",
>
> "--flink_master=192.168.99.100:8081",
>
>
>
> versions:
>
> Apache Beam SDK=2.21.0
> python=3.7.6
>
> Flink=1.9.3
>
> Please suggest , if I am missing something.
>
>
>
> *Ashish Raghav *
>
>
>