You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by nagi data monkey <na...@gmail.com> on 2022/02/06 16:08:39 UTC

pyflink datastream job

Hi all,

Anyone got a pyflink datastream job working? I think I'm having difficulty
seeing a small flunk cluster I've set up in docker. I can't see any way
that pyflink can pick up a Remote Execution Enviroment. This is the only
'compiling' code snippet I can find:

from pyflink.datastream import StreamExecutionEnvironment
env = StreamExecutionEnvironment.get_execution_environment()

which at least allows me to run pyflink code, but not see any flink
clusters etc. Any ideas how I'm meant to actually get a pyflink job running?

TIA

Re: pyflink datastream job

Posted by nagi data monkey <na...@gmail.com>.

ah, thanks, those doc pages are what I missed!

On Mon, Feb 7, 2022 at 2:48 AM Dian Fu <di...@gmail.com> wrote:

> The following code snippet should just work:
>
> ```
> from pyflink.datastream import StreamExecutionEnvironment
> env = StreamExecutionEnvironment.get_execution_environment()
> ```
>
> It works both in local deployment and in flink clusters.
>
> You could refer to [1] on how to submit PyFlink jobs to a remote cluster.
>
> [1]
> https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/#submitting-pyflink-jobs
>
> Regards,
> Dian
>
>
> On Mon, Feb 7, 2022 at 12:09 AM nagi data monkey <na...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> Anyone got a pyflink datastream job working? I think I'm having
>> difficulty seeing a small flunk cluster I've set up in docker. I can't see
>> any way that pyflink can pick up a Remote Execution Enviroment. This is the
>> only 'compiling' code snippet I can find:
>>
>> from pyflink.datastream import StreamExecutionEnvironment
>> env = StreamExecutionEnvironment.get_execution_environment()
>>
>> which at least allows me to run pyflink code, but not see any flink
>> clusters etc. Any ideas how I'm meant to actually get a pyflink job running?
>>
>> TIA
>>
>

Re: pyflink datastream job

Posted by Dian Fu <di...@gmail.com>.

The following code snippet should just work:

```
from pyflink.datastream import StreamExecutionEnvironment
env = StreamExecutionEnvironment.get_execution_environment()
```

It works both in local deployment and in flink clusters.

You could refer to [1] on how to submit PyFlink jobs to a remote cluster.

[1]
https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/cli/#submitting-pyflink-jobs

Regards,
Dian


On Mon, Feb 7, 2022 at 12:09 AM nagi data monkey <na...@gmail.com>
wrote:

> Hi all,
>
> Anyone got a pyflink datastream job working? I think I'm having difficulty
> seeing a small flunk cluster I've set up in docker. I can't see any way
> that pyflink can pick up a Remote Execution Enviroment. This is the only
> 'compiling' code snippet I can find:
>
> from pyflink.datastream import StreamExecutionEnvironment
> env = StreamExecutionEnvironment.get_execution_environment()
>
> which at least allows me to run pyflink code, but not see any flink
> clusters etc. Any ideas how I'm meant to actually get a pyflink job running?
>
> TIA
>