You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2023/03/13 18:35:07 UTC

[GitHub] [hudi] yihua opened a new pull request, #8172: [HUDI-5927] Improve parallelism of deleting invalid files

yihua opened a new pull request, #8172:
URL: https://github.com/apache/hudi/pull/8172

   ### Change Logs
   
   This PR improves the parallelism of deleting invalid files when finalizing the write, so that the file deletion is parallelized at the file level instead of the partition level.
   
   ### Impact
   
   Improves the parallelism of deleting invalid files.
   
   ### Risk level
   
   none
   
   ### Documentation Update
   
   N/A
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Change Logs and Impact were stated clearly
   - [ ] Adequate tests were added if applicable
   - [ ] CI passed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #8172: [HUDI-5927] Improve parallelism of deleting invalid files

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8172:
URL: https://github.com/apache/hudi/pull/8172#issuecomment-1466816853

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15697",
       "triggerID" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d1eee370f542d5160632f4fdb5a7017e733f2cd7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15697) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #8172: [HUDI-5927] Improve parallelism of deleting invalid files

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8172:
URL: https://github.com/apache/hudi/pull/8172#issuecomment-1467336671

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15697",
       "triggerID" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "37322403cb6352bc74c008e1a80c71a953ff0602",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15703",
       "triggerID" : "37322403cb6352bc74c008e1a80c71a953ff0602",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 37322403cb6352bc74c008e1a80c71a953ff0602 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15703) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yihua merged pull request #8172: [HUDI-5927] Improve parallelism of deleting invalid files

Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua merged PR #8172:
URL: https://github.com/apache/hudi/pull/8172


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #8172: [HUDI-5927] Improve parallelism of deleting invalid files

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8172:
URL: https://github.com/apache/hudi/pull/8172#issuecomment-1467207114

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15697",
       "triggerID" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "37322403cb6352bc74c008e1a80c71a953ff0602",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "37322403cb6352bc74c008e1a80c71a953ff0602",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d1eee370f542d5160632f4fdb5a7017e733f2cd7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15697) 
   * 37322403cb6352bc74c008e1a80c71a953ff0602 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #8172: [HUDI-5927] Improve parallelism of deleting invalid files

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8172:
URL: https://github.com/apache/hudi/pull/8172#issuecomment-1467124146

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15697",
       "triggerID" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d1eee370f542d5160632f4fdb5a7017e733f2cd7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15697) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #8172: [HUDI-5927] Improve parallelism of deleting invalid files

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8172:
URL: https://github.com/apache/hudi/pull/8172#issuecomment-1466738671

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d1eee370f542d5160632f4fdb5a7017e733f2cd7 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #8172: [HUDI-5927] Improve parallelism of deleting invalid files

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8172:
URL: https://github.com/apache/hudi/pull/8172#issuecomment-1467213865

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15697",
       "triggerID" : "d1eee370f542d5160632f4fdb5a7017e733f2cd7",
       "triggerType" : "PUSH"
     }, {
       "hash" : "37322403cb6352bc74c008e1a80c71a953ff0602",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15703",
       "triggerID" : "37322403cb6352bc74c008e1a80c71a953ff0602",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d1eee370f542d5160632f4fdb5a7017e733f2cd7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15697) 
   * 37322403cb6352bc74c008e1a80c71a953ff0602 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15703) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org