You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/06/18 21:05:25 UTC

[GitHub] [flink] zentol opened a new pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

zentol opened a new pull request #16191:
URL: https://github.com/apache/flink/pull/16191


   With this PR it will be possibly to load a plugin temporarily, for example within a single test that uses a MiniCluster.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] zentol merged pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
zentol merged pull request #16191:
URL: https://github.com/apache/flink/pull/16191


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] AHeise commented on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
AHeise commented on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-865309172


   Okay that all makes sense. Last question, do we want to add `(Auto)closeable`. I'd like to make it absolutely clear that this is a life-cycle method that should invoked at the very end?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] zentol merged pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
zentol merged pull request #16191:
URL: https://github.com/apache/flink/pull/16191


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] zentol edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
zentol edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-865300941






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] AHeise edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
AHeise edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-865309172


   Okay that all makes sense. Last question, do we want to add `(Auto)closeable`. I'd like to make it absolutely clear that this is a life-cycle method that should only be invoked at the very end?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-863808302






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-863822678






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] zentol commented on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
zentol commented on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-865300941


   > What happens if we use close in production code? Would everything fail miserable with cryptic messages?
   
   Production code has to ensure that the loaded plugin is no longer used before closing the class loader. In practice I intend to call close when the ClusterEntrypoint/TaskManagerRunner/MiniCluster is being shutdown, after the RPCService has been shut down.
   
    > Could we add @VisibleForTesting for now to avoid the issues?
   
   `@VisibleForTesting` is not applicable in this case, because the MiniCluster (the primary user) is _not_ test code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] zentol commented on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
zentol commented on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-865300941






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] AHeise edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
AHeise edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-865309172


   Okay that all makes sense. Last question, do we want to add `(Auto)closeable`. I'd like to make it absolutely clear that this is a life-cycle method that should only be invoked at the very end?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] AHeise commented on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
AHeise commented on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-865309172


   Okay that all makes sense. Last question, do we want to add `(Auto)closeable`. I'd like to make it absolutely clear that this is a life-cycle method that should invoked at the very end?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] zentol commented on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
zentol commented on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-865327585


   I'm not sure how auto-closable make it clearer that this is a lifecycle method, but I can change it nonetheless to support try-with-resources which could be useful for testing.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-863822678


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ce0811d23c53c492e466a144fdd3de6fc1b1006e",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19096",
       "triggerID" : "ce0811d23c53c492e466a144fdd3de6fc1b1006e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8349827a782e23ae82592e79362a8397c4e2dd61",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "8349827a782e23ae82592e79362a8397c4e2dd61",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ce0811d23c53c492e466a144fdd3de6fc1b1006e Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19096) 
   * 8349827a782e23ae82592e79362a8397c4e2dd61 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-863808302


   Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress of the review.
   
   
   ## Automated Checks
   Last check on commit 8349827a782e23ae82592e79362a8397c4e2dd61 (Sat Aug 28 12:20:40 UTC 2021)
   
   **Warnings:**
    * No documentation files were touched! Remember to keep the Flink docs up to date!
   
   
   <sub>Mention the bot in a comment to re-run the automated checks.</sub>
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
    The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
    - `@flinkbot approve all` to approve all aspects
    - `@flinkbot approve-until architecture` to approve everything until `architecture`
    - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
    - `@flinkbot disapprove architecture` to remove an approval you gave earlier
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-863822678


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ce0811d23c53c492e466a144fdd3de6fc1b1006e",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19096",
       "triggerID" : "ce0811d23c53c492e466a144fdd3de6fc1b1006e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8349827a782e23ae82592e79362a8397c4e2dd61",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19272",
       "triggerID" : "8349827a782e23ae82592e79362a8397c4e2dd61",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ce0811d23c53c492e466a144fdd3de6fc1b1006e Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19096) 
   * 8349827a782e23ae82592e79362a8397c4e2dd61 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19272) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] zentol edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
zentol edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-865300941


   > What happens if we use close in production code?
   
   Production code has to ensure that the loaded plugin is no longer used before closing the class loader. In practice I intend to call close when the ClusterEntrypoint/TaskManagerRunner/MiniCluster is being shutdown, after the RPCService has been shut down.
   At that point exceptions are usually not a major concern.
   
   >  Would everything fail miserable with cryptic messages?
   
   You'd get the usual ClassNotFoundExceptions etc. when trying to load a new class; which I would be fine with for the time being because it is only for an internal use-case that isn't visible to users.
   
   > Should we add our own guard for all respective accesses?
   
   We could do that yes, but see above.
   
    > Could we add @VisibleForTesting for now to avoid the issues?
   
   `@VisibleForTesting` is not applicable in this case, because the MiniCluster (the primary user) is _not_ test code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-863822678


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ce0811d23c53c492e466a144fdd3de6fc1b1006e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19096",
       "triggerID" : "ce0811d23c53c492e466a144fdd3de6fc1b1006e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "8349827a782e23ae82592e79362a8397c4e2dd61",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19272",
       "triggerID" : "8349827a782e23ae82592e79362a8397c4e2dd61",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 8349827a782e23ae82592e79362a8397c4e2dd61 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19272) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] zentol edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
zentol edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-865300941


   > What happens if we use close in production code?
   
   Production code has to ensure that the loaded plugin is no longer used before closing the class loader. In practice I intend to call close when the ClusterEntrypoint/TaskManagerRunner/MiniCluster is being shutdown, after the RPCService has been shut down.
   
   >  Would everything fail miserable with cryptic messages?
   
   You'd get the usual ClassNotFoundExceptions etc. when trying to load a new class; which I would be fine with for the time being because it is only for an internal use-case that isn't visible to users.
   
   > Should we add our own guard for all respective accesses?
   
   We could do that yes, but see above.
   
    > Could we add @VisibleForTesting for now to avoid the issues?
   
   `@VisibleForTesting` is not applicable in this case, because the MiniCluster (the primary user) is _not_ test code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #16191: [FLINK-23029] Extend PluginLoader to allow closing ClassLoader

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16191:
URL: https://github.com/apache/flink/pull/16191#issuecomment-863822678






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org