You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "voonhous (via GitHub)" <gi...@apache.org> on 2025/04/23 13:39:54 UTC
[PR] [MINOR] Fix default value of metadata partition stats enable [hudi]
voonhous opened a new pull request, #13214:
URL: https://github.com/apache/hudi/pull/13214
### Change Logs
`ENABLE_METADATA_INDEX_PARTITION_STATS` is enabled by default, a change that was introduce in the PR below that sets its default value to the value of `ENABLE_METADATA_INDEX_COLUMN_STATS ` here:
https://github.com/apache/hudi/pull/12707/files#diff-11e9ef6bd53ef1001b669a1dc68dde2aba9b33c9eb72cc1e4198750336d79772R893
However, the config value is false. This PR changes the config value to remove this discrepancy. Automated config docs will also be updated as a result to reduce confusion.
### Impact
None
### Risk level (write none, low medium or high below)
None
### Documentation Update
Default value of `ENABLE_METADATA_INDEX_PARTITION_STATS` / `hoodie.metadata.index.partition.enable` will be changed in the config docs.
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Change Logs and Impact were stated clearly
- [x] Adequate tests were added if applicable
- [x] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Fix default value of metadata partition stats enable [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #13214:
URL: https://github.com/apache/hudi/pull/13214#discussion_r2056252489
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java:
##########
@@ -381,9 +381,10 @@ public final class HoodieMetadataConfig extends HoodieConfig {
public static final ConfigProperty<Boolean> ENABLE_METADATA_INDEX_PARTITION_STATS = ConfigProperty
.key(METADATA_PREFIX + ".index.partition.stats.enable")
- .defaultValue(false)
+ .defaultValue(true)
Review Comment:
we do this at per engine level. thats why we did not flip this value.
if you check HoodieMetadataConfig,
we have getDefaultColStatsEnable() which will choose the right value for col stats for different engines.
and the default value for partitions stats enable goes along w/ the derived value for col stats.
if col stats is enabled, PSI will also be enabled. if not, it will not.
We just need to update our release docs to call this out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Fix default value of metadata partition stats enable [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13214:
URL: https://github.com/apache/hudi/pull/13214#issuecomment-2824350622
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "05353ced6a8a79f38368892ff970dead4b337595",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "05353ced6a8a79f38368892ff970dead4b337595",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 05353ced6a8a79f38368892ff970dead4b337595 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Fix default value of metadata partition stats enable [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #13214:
URL: https://github.com/apache/hudi/pull/13214#issuecomment-2824997886
> not sure apart from doc discrepancy, what else we are trying to solve here.
No other issues aside from doc discrepancy. Let me change the doc description for this to call the behaviour out then and keep the default value as **false**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Fix default value of metadata partition stats enable [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13214:
URL: https://github.com/apache/hudi/pull/13214#issuecomment-2824621122
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "05353ced6a8a79f38368892ff970dead4b337595",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "05353ced6a8a79f38368892ff970dead4b337595",
"triggerType" : "PUSH"
}, {
"hash" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5104",
"triggerID" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 05353ced6a8a79f38368892ff970dead4b337595 UNKNOWN
* 6083ec0cdbd718361d5026a92fa7b68ad6fee395 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5104)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Update doc description for metadata index partition enable [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13214:
URL: https://github.com/apache/hudi/pull/13214#issuecomment-2825034828
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "05353ced6a8a79f38368892ff970dead4b337595",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "05353ced6a8a79f38368892ff970dead4b337595",
"triggerType" : "PUSH"
}, {
"hash" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5104",
"triggerID" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"triggerType" : "PUSH"
}, {
"hash" : "faba473d8af79ea4c4bcc3f3cb54678bed4cc97b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "faba473d8af79ea4c4bcc3f3cb54678bed4cc97b",
"triggerType" : "PUSH"
}, {
"hash" : "53eb9215378e5c2420af5e8c241c9cc8532fe098",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "53eb9215378e5c2420af5e8c241c9cc8532fe098",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 05353ced6a8a79f38368892ff970dead4b337595 UNKNOWN
* 6083ec0cdbd718361d5026a92fa7b68ad6fee395 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5104)
* faba473d8af79ea4c4bcc3f3cb54678bed4cc97b UNKNOWN
* 53eb9215378e5c2420af5e8c241c9cc8532fe098 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Fix default value of metadata partition stats enable [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13214:
URL: https://github.com/apache/hudi/pull/13214#issuecomment-2824358943
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "05353ced6a8a79f38368892ff970dead4b337595",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "05353ced6a8a79f38368892ff970dead4b337595",
"triggerType" : "PUSH"
}, {
"hash" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5104",
"triggerID" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 05353ced6a8a79f38368892ff970dead4b337595 UNKNOWN
* 6083ec0cdbd718361d5026a92fa7b68ad6fee395 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5104)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Update doc description for metadata index partition enable [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13214:
URL: https://github.com/apache/hudi/pull/13214#issuecomment-2825025508
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "05353ced6a8a79f38368892ff970dead4b337595",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "05353ced6a8a79f38368892ff970dead4b337595",
"triggerType" : "PUSH"
}, {
"hash" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5104",
"triggerID" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"triggerType" : "PUSH"
}, {
"hash" : "faba473d8af79ea4c4bcc3f3cb54678bed4cc97b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "faba473d8af79ea4c4bcc3f3cb54678bed4cc97b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 05353ced6a8a79f38368892ff970dead4b337595 UNKNOWN
* 6083ec0cdbd718361d5026a92fa7b68ad6fee395 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5104)
* faba473d8af79ea4c4bcc3f3cb54678bed4cc97b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Update doc description for metadata index partition enable [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on code in PR #13214:
URL: https://github.com/apache/hudi/pull/13214#discussion_r2056563299
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java:
##########
@@ -381,9 +381,10 @@ public final class HoodieMetadataConfig extends HoodieConfig {
public static final ConfigProperty<Boolean> ENABLE_METADATA_INDEX_PARTITION_STATS = ConfigProperty
.key(METADATA_PREFIX + ".index.partition.stats.enable")
- .defaultValue(false)
+ .defaultValue(true)
Review Comment:
I have updated the description ,PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Update doc description for metadata index partition enable [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13214:
URL: https://github.com/apache/hudi/pull/13214#issuecomment-2825068865
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "05353ced6a8a79f38368892ff970dead4b337595",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "05353ced6a8a79f38368892ff970dead4b337595",
"triggerType" : "PUSH"
}, {
"hash" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5104",
"triggerID" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"triggerType" : "PUSH"
}, {
"hash" : "faba473d8af79ea4c4bcc3f3cb54678bed4cc97b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "faba473d8af79ea4c4bcc3f3cb54678bed4cc97b",
"triggerType" : "PUSH"
}, {
"hash" : "53eb9215378e5c2420af5e8c241c9cc8532fe098",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "53eb9215378e5c2420af5e8c241c9cc8532fe098",
"triggerType" : "PUSH"
}, {
"hash" : "eac36752f34c080caef29a4cde001db5cc6849ac",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "eac36752f34c080caef29a4cde001db5cc6849ac",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 05353ced6a8a79f38368892ff970dead4b337595 UNKNOWN
* 6083ec0cdbd718361d5026a92fa7b68ad6fee395 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5104)
* faba473d8af79ea4c4bcc3f3cb54678bed4cc97b UNKNOWN
* 53eb9215378e5c2420af5e8c241c9cc8532fe098 UNKNOWN
* eac36752f34c080caef29a4cde001db5cc6849ac UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Update doc description for metadata index partition enable [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan merged PR #13214:
URL: https://github.com/apache/hudi/pull/13214
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8680] Fix default value of metadata partition stats enable [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13214:
URL: https://github.com/apache/hudi/pull/13214#issuecomment-2824354696
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "05353ced6a8a79f38368892ff970dead4b337595",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "05353ced6a8a79f38368892ff970dead4b337595",
"triggerType" : "PUSH"
}, {
"hash" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6083ec0cdbd718361d5026a92fa7b68ad6fee395",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 05353ced6a8a79f38368892ff970dead4b337595 UNKNOWN
* 6083ec0cdbd718361d5026a92fa7b68ad6fee395 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Fix default value of metadata partition stats enable [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #13214:
URL: https://github.com/apache/hudi/pull/13214#issuecomment-2824347870
Actually, let me reuse the JIRA of pr [https://github.com/apache/hudi/pull/12707](https://github.com/apache/hudi/pull/12707/files#diff-11e9ef6bd53ef1001b669a1dc68dde2aba9b33c9eb72cc1e4198750336d79772R893)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org