You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/10/27 11:28:58 UTC

[GitHub] [hudi] codope opened a new pull request #3875: [HUDI-2442] Change default values for certin clustering configs

codope opened a new pull request #3875:
URL: https://github.com/apache/hudi/pull/3875


   ## What is the purpose of the pull request
   
   In order to support better out-of-box clustering configs, we need to have better default values for those configs. Most of them already have a good default value, this PR changes defaut for two configs: 
   - Clustering plan strategy class: The previously set default value made certain assumptions on the partition path like it should be day-based partition. The new default is purely file size based. 
   - Enable commit metadata preservation: Previously, it was disabled by default. Enanling it by default will be useful for incremental query support with replace commits in the timeline.
   
   On ease of use: 
   
   Now user only need to enable clustering. That's it! Though it is advisable to set sort columns to cluster by to make use of the default clustering execution strategy. 
   
   ## Committer checklist
   
    - [ ] Has a corresponding JIRA in PR title & commit
    
    - [ ] Commit message is descriptive of the change
    
    - [ ] CI is green
   
    - [ ] Necessary doc changes done or have another open PR
          
    - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] vinothchandar commented on pull request #3875: [HUDI-2442] Change default values for certin clustering configs

Posted by GitBox <gi...@apache.org>.
vinothchandar commented on pull request #3875:
URL: https://github.com/apache/hudi/pull/3875#issuecomment-952851955


   cc @umehrot2 you had brought up the case for simplification of the clustering configs before. Could you please review this? 
   cc @prashantwason @nbalajee @vingov from Uber as well. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] vinothchandar commented on pull request #3875: [HUDI-2442] Change default values for certain clustering configs

Posted by GitBox <gi...@apache.org>.
vinothchandar commented on pull request #3875:
URL: https://github.com/apache/hudi/pull/3875#issuecomment-964836911


   Makes sense to me as well. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #3875: [HUDI-2442] Change default values for certin clustering configs

Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #3875:
URL: https://github.com/apache/hudi/pull/3875#issuecomment-952828494


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895",
       "triggerID" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * aa28cf2aea8712f65e194706e8cd7a32b931126e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #3875: [HUDI-2442] Change default values for certin clustering configs

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3875:
URL: https://github.com/apache/hudi/pull/3875#issuecomment-952828494


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * aa28cf2aea8712f65e194706e8cd7a32b931126e UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #3875: [HUDI-2442] Change default values for certin clustering configs

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3875:
URL: https://github.com/apache/hudi/pull/3875#issuecomment-961588691


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895",
       "triggerID" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * aa28cf2aea8712f65e194706e8cd7a32b931126e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #3875: [HUDI-2442] Change default values for certin clustering configs

Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #3875:
URL: https://github.com/apache/hudi/pull/3875#issuecomment-952828494


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895",
       "triggerID" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * aa28cf2aea8712f65e194706e8cd7a32b931126e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot edited a comment on pull request #3875: [HUDI-2442] Change default values for certin clustering configs

Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3875:
URL: https://github.com/apache/hudi/pull/3875#issuecomment-952828494


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895",
       "triggerID" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * aa28cf2aea8712f65e194706e8cd7a32b931126e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot removed a comment on pull request #3875: [HUDI-2442] Change default values for certin clustering configs

Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #3875:
URL: https://github.com/apache/hudi/pull/3875#issuecomment-952828494


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895",
       "triggerID" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * aa28cf2aea8712f65e194706e8cd7a32b931126e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] codope merged pull request #3875: [HUDI-2442] Change default values for certain clustering configs

Posted by GitBox <gi...@apache.org>.
codope merged pull request #3875:
URL: https://github.com/apache/hudi/pull/3875


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot edited a comment on pull request #3875: [HUDI-2442] Change default values for certin clustering configs

Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3875:
URL: https://github.com/apache/hudi/pull/3875#issuecomment-952828494


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895",
       "triggerID" : "aa28cf2aea8712f65e194706e8cd7a32b931126e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * aa28cf2aea8712f65e194706e8cd7a32b931126e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=2895) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run travis` re-run the last Travis build
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #3875: [HUDI-2442] Change default values for certin clustering configs

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3875:
URL: https://github.com/apache/hudi/pull/3875#issuecomment-961588691






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org