You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Yik San Chan <ev...@gmail.com> on 2021/05/05 04:03:55 UTC
How to tell between a local mode run vs. remote mode run?
Hi,
According to
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/python/faq/
> When executing jobs in mini cluster(e.g. when executing jobs in IDE) ...
please remember to explicitly wait for the job execution to finish as these
APIs are asynchronous.
I hope my program will be able to run in both local mode as well as in
remote mode. Therefore I hope to do something like:
```python
result = ...
if local_mode:
result.wait()
else:
result
```
Is there a way to tell if the program is run under local mode vs. remote
mode?
Best,
Yik San
Re: How to tell between a local mode run vs. remote mode run?
Posted by Yik San Chan <ev...@gmail.com>.
Hi Xingbo,
Thank you!
On Thu, May 6, 2021 at 10:01 AM Xingbo Huang <hx...@gmail.com> wrote:
> Hi Yik San,
> You can check whether the execution environment used is
> `LocalStreamEnvironment` and you can get the class object corresponding to
> the corresponding java object through py4j in PyFlink. You can take a look
> at the example I wrote below, I hope it will help you
> ```
> from pyflink.table import EnvironmentSettings, StreamTableEnvironment
> from pyflink.datastream import StreamExecutionEnvironment
> from pyflink.java_gateway import get_gateway
> from py4j.java_gateway import get_java_class
>
>
> def test():
> env = StreamExecutionEnvironment.get_execution_environment()
> table_env = StreamTableEnvironment.create(
> env, environment_settings=EnvironmentSettings.new_instance()
> .in_streaming_mode().use_blink_planner().build())
> gateway = get_gateway()
>
> # get the execution environment class
> env_class = table_env._j_tenv.getPlanner().getExecEnv().getClass()
>
> # get the LocalStreamEnvironment class
> local_stream_environment_class = get_java_class(
>
> gateway.jvm.org.apache.flink.streaming.api.environment.LocalStreamEnvironment)
> print(env_class == local_stream_environment_class)
>
>
> if __name__ == '__main__':
> test()
>
> ```
>
> Yik San Chan <ev...@gmail.com> 于2021年5月5日周三 下午12:04写道:
>
>> Hi,
>>
>> According to
>> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/python/faq/
>>
>> > When executing jobs in mini cluster(e.g. when executing jobs in IDE)
>> ... please remember to explicitly wait for the job execution to finish as
>> these APIs are asynchronous.
>>
>> I hope my program will be able to run in both local mode as well as in
>> remote mode. Therefore I hope to do something like:
>>
>> ```python
>> result = ...
>> if local_mode:
>> result.wait()
>> else:
>> result
>> ```
>>
>> Is there a way to tell if the program is run under local mode vs. remote
>> mode?
>>
>> Best,
>> Yik San
>>
>
Re: How to tell between a local mode run vs. remote mode run?
Posted by Xingbo Huang <hx...@gmail.com>.
Hi Yik San,
You can check whether the execution environment used is
`LocalStreamEnvironment` and you can get the class object corresponding to
the corresponding java object through py4j in PyFlink. You can take a look
at the example I wrote below, I hope it will help you
```
from pyflink.table import EnvironmentSettings, StreamTableEnvironment
from pyflink.datastream import StreamExecutionEnvironment
from pyflink.java_gateway import get_gateway
from py4j.java_gateway import get_java_class
def test():
env = StreamExecutionEnvironment.get_execution_environment()
table_env = StreamTableEnvironment.create(
env, environment_settings=EnvironmentSettings.new_instance()
.in_streaming_mode().use_blink_planner().build())
gateway = get_gateway()
# get the execution environment class
env_class = table_env._j_tenv.getPlanner().getExecEnv().getClass()
# get the LocalStreamEnvironment class
local_stream_environment_class = get_java_class(
gateway.jvm.org.apache.flink.streaming.api.environment.LocalStreamEnvironment)
print(env_class == local_stream_environment_class)
if __name__ == '__main__':
test()
```
Yik San Chan <ev...@gmail.com> 于2021年5月5日周三 下午12:04写道:
> Hi,
>
> According to
> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/python/faq/
>
> > When executing jobs in mini cluster(e.g. when executing jobs in IDE)
> ... please remember to explicitly wait for the job execution to finish as
> these APIs are asynchronous.
>
> I hope my program will be able to run in both local mode as well as in
> remote mode. Therefore I hope to do something like:
>
> ```python
> result = ...
> if local_mode:
> result.wait()
> else:
> result
> ```
>
> Is there a way to tell if the program is run under local mode vs. remote
> mode?
>
> Best,
> Yik San
>