You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "nsivabalan (via GitHub)" <gi...@apache.org> on 2025/12/04 01:10:32 UTC

[PR] minor: Enabling streaming writes to metadata code paths only when eligible metadata partitions [hudi]

nsivabalan opened a new pull request, #17478:
URL: https://github.com/apache/hudi/pull/17478

   ### Describe the issue this Pull Request addresses
   
   We added streaming writes to metadata table in 1.1.0. Not all partitions in mdt supports streaming writes as of now. So, fixing writes to leverage streaming writes to metadata table only when eligible metadata partitions are enabled. 
   
   ### Summary and Changelog
   
   - Leverage streaming writes to metadata table in spark only when either of RLI (global or non global) or secondary index is enabled. 
   
   ### Impact
   
   - Avoids the new dag if not really required. Down the line, when we move col stats and other partitions to streaming flow, probably we will get rid of this optimization. 
   
   ### Risk Level
   
   low
   
   ### Documentation Update
   
   <!-- Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".
   
   - The config description must be updated if new configs are added or the default value of the configs are changed.
   - Any new feature or user-facing change requires updating the Hudi website. Please follow the 
     [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website. -->
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Enough context is provided in the sections above
   - [ ] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] minor: Enabling streaming writes to metadata code paths only when eligible metadata partitions [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17478:
URL: https://github.com/apache/hudi/pull/17478#issuecomment-3609496273

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "6a36a05947b9585cd72585ef8d54449d6c9e8815",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=9878",
       "triggerID" : "6a36a05947b9585cd72585ef8d54449d6c9e8815",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 6a36a05947b9585cd72585ef8d54449d6c9e8815 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=9878) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] minor: Enabling streaming writes to metadata code paths only when eligible metadata partitions [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17478:
URL: https://github.com/apache/hudi/pull/17478#issuecomment-3609492625

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "6a36a05947b9585cd72585ef8d54449d6c9e8815",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6a36a05947b9585cd72585ef8d54449d6c9e8815",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 6a36a05947b9585cd72585ef8d54449d6c9e8815 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] minor: Enabling streaming writes to metadata code paths only when eligible metadata partitions [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #17478:
URL: https://github.com/apache/hudi/pull/17478#issuecomment-3609645971

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "6a36a05947b9585cd72585ef8d54449d6c9e8815",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=9878",
       "triggerID" : "6a36a05947b9585cd72585ef8d54449d6c9e8815",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 6a36a05947b9585cd72585ef8d54449d6c9e8815 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=9878) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org