You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Jeff Zhang (Jira)" <ji...@apache.org> on 2020/04/13 15:11:00 UTC

[jira] [Commented] (FLINK-17124) The PyFlink Job runs into infinite loop if the UDF file imports job code.

    [ https://issues.apache.org/jira/browse/FLINK-17124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082394#comment-17082394 ] 

Jeff Zhang commented on FLINK-17124:
------------------------------------

Should it be a blocker issue for 1.10.1 ?

> The PyFlink Job runs into infinite loop if the UDF file imports job code.
> -------------------------------------------------------------------------
>
>                 Key: FLINK-17124
>                 URL: https://issues.apache.org/jira/browse/FLINK-17124
>             Project: Flink
>          Issue Type: Bug
>          Components: API / Python
>    Affects Versions: 1.10.0, 1.11.0
>            Reporter: Huang Xingbo
>            Priority: Major
>
> If the UDF file imports job code directly or indirectly, the PyFlink Job will run into a infinite loop as follows:
>  - submit job
>  - execute job
>  - launch UDF worker
>  - import UDF
>  - (If the job file is depended by UDF or imported as the top level module) import job code
>  - (If the job code is executed outside the "*if __name__ == '__main__':*") launch gateway server and submit job to local executor
>  - execute job in local mode
>  - launch UDF worker
>  - import UDF
>  - import job code
>  ...
> This infinite loop will create new Java processes and Python processes endlessly until the resources on the machine are exhausted. We should fix it ASAP.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)