You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Dian Fu (Jira)" <ji...@apache.org> on 2020/04/02 10:34:00 UTC
[jira] [Commented] (FLINK-16666) Support new Python dependency
configuration options in flink-java, flink-streaming-java and flink-table
[ https://issues.apache.org/jira/browse/FLINK-16666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073593#comment-17073593 ]
Dian Fu commented on FLINK-16666:
---------------------------------
Hi [~aljoscha], as explained by [~zhongwei], the requirement is that we need to process the Python dependency management related config options and register the dependencies to the distributed cache to make sure that the dependencies could be accessed during execution. In the `configure` method of ExecutionEnvironment / StreamExecutionEnvironment, it will perform some pre-processing according to the configuration object, e.g. setup the execution config and checkpoint config according to the give configuration object. So conceptually, this seems a good place for this requirement, e.g. register the Python dependencies to the distributed cache according to the given configuration object. Besides, it will also avoid code duplication as this requirements not only apply for PyFlink Table API, but also for any other kinds of API, such as PyFlink DataStream API, etc which maybe introduced in the near future.
What's your thought?
> Support new Python dependency configuration options in flink-java, flink-streaming-java and flink-table
> -------------------------------------------------------------------------------------------------------
>
> Key: FLINK-16666
> URL: https://issues.apache.org/jira/browse/FLINK-16666
> Project: Flink
> Issue Type: Sub-task
> Components: API / Python
> Reporter: Wei Zhong
> Assignee: Wei Zhong
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)