You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/12/29 06:33:23 UTC

[GitHub] [hudi] cxzl25 opened a new pull request, #7579: [HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap

cxzl25 opened a new pull request, #7579:
URL: https://github.com/apache/hudi/pull/7579

   ### Change Logs
   
   Reduce duplicate Logs in ExternalSpillableMap
   
   We see hundreds of thousands of duplicate logs in the executor log.
   
   ```
   22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
   22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
   22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
   22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
   22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
   22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
   22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
   ```
   
   ### Impact
   
   
   
   ### Risk level (write none, low medium or high below)
   
   
   
   ### Documentation Update
   
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Change Logs and Impact were stated clearly
   - [ ] Adequate tests were added if applicable
   - [ ] CI passed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 merged pull request #7579: [HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #7579:
URL: https://github.com/apache/hudi/pull/7579


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] cxzl25 commented on pull request #7579: [HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap

Posted by GitBox <gi...@apache.org>.
cxzl25 commented on PR #7579:
URL: https://github.com/apache/hudi/pull/7579#issuecomment-1367234808

   @hudi-bot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #7579: [HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7579:
URL: https://github.com/apache/hudi/pull/7579#issuecomment-1367390604

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14032",
       "triggerID" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14037",
       "triggerID" : "1367234808",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * ba9aa020afa608a3b51d7085c48217d97bbc1881 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14032) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14037) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #7579: [HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7579:
URL: https://github.com/apache/hudi/pull/7579#issuecomment-1367228017

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14032",
       "triggerID" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ba9aa020afa608a3b51d7085c48217d97bbc1881 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14032) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #7579: [HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7579:
URL: https://github.com/apache/hudi/pull/7579#issuecomment-1367272629

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14032",
       "triggerID" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14037",
       "triggerID" : "1367234808",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * ba9aa020afa608a3b51d7085c48217d97bbc1881 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14032) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14037) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #7579: [HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7579:
URL: https://github.com/apache/hudi/pull/7579#issuecomment-1367133768

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14032",
       "triggerID" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ba9aa020afa608a3b51d7085c48217d97bbc1881 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14032) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #7579: [HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7579:
URL: https://github.com/apache/hudi/pull/7579#issuecomment-1367131174

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ba9aa020afa608a3b51d7085c48217d97bbc1881",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ba9aa020afa608a3b51d7085c48217d97bbc1881 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] cxzl25 commented on pull request #7579: [HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap

Posted by "cxzl25 (via GitHub)" <gi...@apache.org>.
cxzl25 commented on PR #7579:
URL: https://github.com/apache/hudi/pull/7579#issuecomment-1408190626

   Help review. Thanks.
   @XuQianJin-Stars @alexeykudinkin 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org