You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/05/20 16:10:35 UTC

[GitHub] [flink] azagrebin opened a new pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

azagrebin opened a new pull request #12270:
URL: https://github.com/apache/flink/pull/12270


   ## What is the purpose of the change
   
   #11109 did not export `jdk.internal.misc` package used by reflection for GC cleaners of managed memory. Our PR CI does not run Java 11 tests atm. The package has to be exported by a JVM runtime arg: `--add-opens java.base/jdk.internal.misc=ALL-UNNAMED;`
   
   If this arg is set for Java 8, it fails the JVM process. Therefore, the fix is complicated as we have to do it also for e.g. Yarn CLI where client and cluster may run different Java versions.
   
   This PR suggests a quicker fix. We call directly the private method (has to be made accessible via reflection):
   - java.lang.ref.Reference.tryHandlePending(false) // for Java 8
   - java.lang.ref.Reference.waitForReferenceProcessing() // for Java 11
   
   Unfortunately, this leads to the annoying warning for Java 11 about Illegal reflective access.
   We can do the quick fix and think how to tackle the warning in a follow-up.
   
   ## Verifying this change
   
   Existing unit tests but also running them and e2e tests with Java 11.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] azagrebin commented on pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
azagrebin commented on pull request #12270:
URL: https://github.com/apache/flink/pull/12270#issuecomment-633425077


   Thanks for the review @tillrohrmann 
   merging this


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] azagrebin commented on pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
azagrebin commented on pull request #12270:
URL: https://github.com/apache/flink/pull/12270#issuecomment-632510359


   @flinkbot run azure


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12270:
URL: https://github.com/apache/flink/pull/12270#issuecomment-631585690


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=1965",
       "triggerID" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5472f93e8f79940941b5ba6637be6810d6b8b46a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=1965) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12270:
URL: https://github.com/apache/flink/pull/12270#issuecomment-631585690


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=1965",
       "triggerID" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "94c1fcd295e0f8fece52041c8bac2395fa18eba8",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "94c1fcd295e0f8fece52041c8bac2395fa18eba8",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5472f93e8f79940941b5ba6637be6810d6b8b46a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=1965) 
   * 94c1fcd295e0f8fece52041c8bac2395fa18eba8 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12270:
URL: https://github.com/apache/flink/pull/12270#issuecomment-631585690


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=1965",
       "triggerID" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5472f93e8f79940941b5ba6637be6810d6b8b46a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=1965) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12270:
URL: https://github.com/apache/flink/pull/12270#issuecomment-631585690


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=1965",
       "triggerID" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "94c1fcd295e0f8fece52041c8bac2395fa18eba8",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2076",
       "triggerID" : "94c1fcd295e0f8fece52041c8bac2395fa18eba8",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5472f93e8f79940941b5ba6637be6810d6b8b46a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=1965) 
   * 94c1fcd295e0f8fece52041c8bac2395fa18eba8 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2076) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #12270:
URL: https://github.com/apache/flink/pull/12270#issuecomment-631585690


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5472f93e8f79940941b5ba6637be6810d6b8b46a UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #12270:
URL: https://github.com/apache/flink/pull/12270#issuecomment-631574892


   Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress of the review.
   
   
   ## Automated Checks
   Last check on commit 5472f93e8f79940941b5ba6637be6810d6b8b46a (Wed May 20 16:12:20 UTC 2020)
   
   **Warnings:**
    * No documentation files were touched! Remember to keep the Flink docs up to date!
   
   
   <sub>Mention the bot in a comment to re-run the automated checks.</sub>
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
    The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
    - `@flinkbot approve all` to approve all aspects
    - `@flinkbot approve-until architecture` to approve everything until `architecture`
    - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
    - `@flinkbot disapprove architecture` to remove an approval you gave earlier
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12270:
URL: https://github.com/apache/flink/pull/12270#issuecomment-631585690


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=1965",
       "triggerID" : "5472f93e8f79940941b5ba6637be6810d6b8b46a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "94c1fcd295e0f8fece52041c8bac2395fa18eba8",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2076",
       "triggerID" : "94c1fcd295e0f8fece52041c8bac2395fa18eba8",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 94c1fcd295e0f8fece52041c8bac2395fa18eba8 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=2076) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot edited a comment on pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12270:
URL: https://github.com/apache/flink/pull/12270#issuecomment-631574892


   Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress of the review.
   
   
   ## Automated Checks
   Last check on commit 94c1fcd295e0f8fece52041c8bac2395fa18eba8 (Fri Oct 16 10:54:24 UTC 2020)
   
   **Warnings:**
    * No documentation files were touched! Remember to keep the Flink docs up to date!
   
   
   <sub>Mention the bot in a comment to re-run the automated checks.</sub>
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
    The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
    - `@flinkbot approve all` to approve all aspects
    - `@flinkbot approve-until architecture` to approve everything until `architecture`
    - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
    - `@flinkbot disapprove architecture` to remove an approval you gave earlier
   </details>


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] azagrebin closed pull request #12270: [FLINK-17822][MemMan] Use private Reference.tryHandlePending|waitForReferenceProcessing to trigger GC cleaner

Posted by GitBox <gi...@apache.org>.
azagrebin closed pull request #12270:
URL: https://github.com/apache/flink/pull/12270


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org