You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 10:53:47 UTC

[I] Limit the number of columns for which to aggregate stats by default [hudi]

hudi-bot opened a new issue, #16670:
URL: https://github.com/apache/hudi/issues/16670

   We need to limit the number of columns for which to aggregate stats by default otherwise building and updating partition stats or column stats index can be slow if we do so for all columns. 
   
   ## JIRA info
   
   - Link: https://issues.apache.org/jira/browse/HUDI-8327
   - Type: Task
   - Epic: https://issues.apache.org/jira/browse/HUDI-512
   - Fix version(s):
     - 1.1.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Limit the number of columns for which to aggregate stats by default [hudi]

Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #16670: Limit the number of columns for which to aggregate stats by default
URL: https://github.com/apache/hudi/issues/16670


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Limit the number of columns for which to aggregate stats by default [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on issue #16670:
URL: https://github.com/apache/hudi/issues/16670#issuecomment-3593874443

   This issue belongs to a closed JIRA Epic: https://issues.apache.org/jira/browse/HUDI-512


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Limit the number of columns for which to aggregate stats by default [hudi]

Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #16670:
URL: https://github.com/apache/hudi/issues/16670#issuecomment-4795630335

   This appears to already be implemented on master. The config `hoodie.metadata.index.column.stats.max.columns.to.index` was added in [HUDI-8556 / #12310](https://github.com/apache/hudi/pull/12310) (commit 898eb1c) with a default value of **32**. It is applied in `HoodieTableMetadataUtil.getColumnsToIndex(...)`, which is the shared path for both column stats and partition stats indexing — so when `hoodie.metadata.index.column.stats.columns` is not explicitly set, only the first 32 columns of the table schema are indexed by default for both indices.
   
   If the intent was a *different* default or a *separate* knob specifically for partition stats, please clarify; otherwise this can probably be closed as completed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org