[jira] [Commented] (FLINK-10352) Bind Task's URLClassLoader to TaskExecutor's slot

["Stephan Ewen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-10352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619601#comment-16619601 ] 

Stephan Ewen commented on FLINK-10352:
--------------------------------------

This is sensitive core change and behavior breaking. This needs to have a design first.

Some issues that come immediately to mind to clarify first:

  - This changes behavior of static variables across tasks and across re-executions, can hence break programs.
  - Will the old model still be supported for applications that relied on that? What will the config switch be?
  - How how do we deal with (unexpected but possible) different sets of libraries attached to tasks?


> Bind Task's URLClassLoader to TaskExecutor's slot
> -------------------------------------------------
>
>                 Key: FLINK-10352
>                 URL: https://issues.apache.org/jira/browse/FLINK-10352
>             Project: Flink
>          Issue Type: Improvement
>          Components: Distributed Coordination
>    Affects Versions: 1.5.3, 1.6.0, 1.7.0
>            Reporter: Till Rohrmann
>            Assignee: Jiayi Liao
>            Priority: Major
>             Fix For: 1.7.0
>
>
> The {{Task's}} {{URLClassLoader}} is used to dynamically load user code classes. The lifetime of this class loader is bound to the {{Task}} and, thus, it can happen that we generate multiple class loader for the same job if multiple {{Tasks}} are deployed to the same slot successively.
> We could avoid this situation by binding the lifetime of the {{URLClassLoader}} to the slot instead of the {{Task}}. The slot will only be released if the {{JobMaster}} no longer needs it. That way we won't potentially recreate the {{URLClassLoader}} for every submitted {{Task}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)