You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/03/19 03:31:40 UTC
[GitHub] [hudi] alexeykudinkin opened a new pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
alexeykudinkin opened a new pull request #5070:
URL: https://github.com/apache/hudi/pull/5070
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
Currently, Column Stats is not able to properly index first commit in the table b/c it uses `TableSchemaResolver`, which is not able to fetch schema for empty tables.
Instead, we should leverage writer's schema provided w/in `HoodieCommitMetadata`
Fixing Column Stats index to properly handle first Data Table commit
## Brief change log
See above
## Verify this pull request
This pull request is already covered by existing tests, such as *(please describe tests)*.
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072930974
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* af8744478e331744286491ed8733f8bc760e3546 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073076727
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
}, {
"hash" : "0557906c0549d5550b2593517117cd08b630ae59",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0557906c0549d5550b2593517117cd08b630ae59",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 3d88e80e20641007f105ac2ed61faa2703036453 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096)
* 0557906c0549d5550b2593517117cd08b630ae59 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073054956
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 422aa2d6ddff9ad9b1022e78877befae3fd09c6d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084)
* 3d88e80e20641007f105ac2ed61faa2703036453 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072932468
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* af8744478e331744286491ed8733f8bc760e3546 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083)
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072930597
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* af8744478e331744286491ed8733f8bc760e3546 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072932762
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* af8744478e331744286491ed8733f8bc760e3546 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083)
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 422aa2d6ddff9ad9b1022e78877befae3fd09c6d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073073622
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 3d88e80e20641007f105ac2ed61faa2703036453 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073073622
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 3d88e80e20641007f105ac2ed61faa2703036453 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a change in pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
codope commented on a change in pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#discussion_r830555950
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
##########
@@ -461,11 +461,9 @@ public String getPreCombineField() {
}
public Option<String[]> getRecordKeyFields() {
- if (contains(RECORDKEY_FIELDS)) {
- return Option.of(Arrays.stream(getString(RECORDKEY_FIELDS).split(","))
- .filter(p -> p.length() > 0).collect(Collectors.toList()).toArray(new String[] {}));
- }
- return Option.empty();
Review comment:
Should we localize this change? With your change, this method will no longer return Option.empty()? Is there any caller dependent on this behavior?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072932762
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* af8744478e331744286491ed8733f8bc760e3546 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083)
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 422aa2d6ddff9ad9b1022e78877befae3fd09c6d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a change in pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
codope commented on a change in pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#discussion_r830436453
##########
File path: hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java
##########
@@ -838,55 +857,52 @@ public static HoodieTableFileSystemView getFileSystemView(HoodieTableMetaClient
public static HoodieData<HoodieRecord> convertMetadataToColumnStatsRecords(HoodieCommitMetadata commitMetadata,
HoodieEngineContext engineContext,
MetadataRecordsGenerationParams recordsGenerationParams) {
- try {
- List<HoodieWriteStat> allWriteStats = commitMetadata.getPartitionToWriteStats().values().stream()
- .flatMap(entry -> entry.stream()).collect(Collectors.toList());
- return HoodieTableMetadataUtil.createColumnStatsFromWriteStats(engineContext, allWriteStats, recordsGenerationParams);
- } catch (Exception e) {
- throw new HoodieException("Failed to generate column stats records for metadata table ", e);
- }
- }
+ List<HoodieWriteStat> allWriteStats = commitMetadata.getPartitionToWriteStats().values().stream()
+ .flatMap(Collection::stream).collect(Collectors.toList());
- /**
- * Create column stats from write status.
- *
- * @param engineContext - Engine context
- * @param allWriteStats - Write status to convert
- * @param recordsGenerationParams - Parameters for columns stats record generation
- */
- public static HoodieData<HoodieRecord> createColumnStatsFromWriteStats(HoodieEngineContext engineContext,
- List<HoodieWriteStat> allWriteStats,
- MetadataRecordsGenerationParams recordsGenerationParams) {
if (allWriteStats.isEmpty()) {
return engineContext.emptyHoodieData();
}
- final List<String> columnsToIndex = getColumnsToIndex(recordsGenerationParams.getDataMetaClient(), recordsGenerationParams.isAllColumnStatsIndexEnabled());
- final int parallelism = Math.max(Math.min(allWriteStats.size(), recordsGenerationParams.getColumnStatsIndexParallelism()), 1);
- HoodieData<HoodieWriteStat> allWriteStatsRDD = engineContext.parallelize(allWriteStats, parallelism);
- return allWriteStatsRDD.flatMap(writeStat -> translateWriteStatToColumnStats(writeStat, recordsGenerationParams.getDataMetaClient(), columnsToIndex).iterator());
+
+ try {
+ Option<Schema> writerSchema =
+ Option.ofNullable(commitMetadata.getMetadata(HoodieCommitMetadata.SCHEMA_KEY))
+ .map(writerSchemaStr -> new Schema.Parser().parse(writerSchemaStr));
+
+ HoodieTableMetaClient dataTableMetaClient = recordsGenerationParams.getDataMetaClient();
+
+ List<String> columnsToIndex = getColumnsToIndex(recordsGenerationParams,
+ dataTableMetaClient.getTableConfig(), writerSchema);
+
+ if (columnsToIndex.isEmpty()) {
+ // In case there are no columns to index, bail
+ return engineContext.emptyHoodieData();
+ }
+
+ int parallelism = Math.max(Math.min(allWriteStats.size(), recordsGenerationParams.getColumnStatsIndexParallelism()), 1);
+ return engineContext.parallelize(allWriteStats, parallelism)
+ .flatMap(writeStat ->
+ translateWriteStatToColumnStats(writeStat, dataTableMetaClient, columnsToIndex).iterator());
+ } catch (Exception e) {
+ throw new HoodieException("Failed to generate column stats records for metadata table", e);
+ }
}
/**
* Get the latest columns for the table for column stats indexing.
- *
- * @param datasetMetaClient - Data table meta client
- * @param isMetaIndexColumnStatsForAllColumns - Is column stats indexing enabled for all columns
*/
- private static List<String> getColumnsToIndex(HoodieTableMetaClient datasetMetaClient, boolean isMetaIndexColumnStatsForAllColumns) {
- if (!isMetaIndexColumnStatsForAllColumns
- || datasetMetaClient.getCommitsTimeline().filterCompletedInstants().countInstants() < 1) {
- return Arrays.asList(datasetMetaClient.getTableConfig().getRecordKeyFieldProp().split(","));
+ private static List<String> getColumnsToIndex(MetadataRecordsGenerationParams recordsGenParams,
+ HoodieTableConfig tableConfig,
+ Option<Schema> writerSchema) {
+ if (!recordsGenParams.isAllColumnStatsIndexEnabled()) {
+ // TODO why are we only indexing primary key? revisit fallback
Review comment:
This will change as we'll be adding support to index multiple columns: https://github.com/apache/hudi/pull/4693/files#diff-11e9ef6bd53ef1001b669a1dc68dde2aba9b33c9eb72cc1e4198750336d79772
No change needed from your side. I'll take care of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073076727
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
}, {
"hash" : "0557906c0549d5550b2593517117cd08b630ae59",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0557906c0549d5550b2593517117cd08b630ae59",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 3d88e80e20641007f105ac2ed61faa2703036453 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096)
* 0557906c0549d5550b2593517117cd08b630ae59 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072932468
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* af8744478e331744286491ed8733f8bc760e3546 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083)
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073054956
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 422aa2d6ddff9ad9b1022e78877befae3fd09c6d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084)
* 3d88e80e20641007f105ac2ed61faa2703036453 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072930974
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* af8744478e331744286491ed8733f8bc760e3546 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope merged pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
codope merged pull request #5070:
URL: https://github.com/apache/hudi/pull/5070
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073112174
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
}, {
"hash" : "0557906c0549d5550b2593517117cd08b630ae59",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7097",
"triggerID" : "0557906c0549d5550b2593517117cd08b630ae59",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 3d88e80e20641007f105ac2ed61faa2703036453 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096)
* 0557906c0549d5550b2593517117cd08b630ae59 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7097)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073120047
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
}, {
"hash" : "0557906c0549d5550b2593517117cd08b630ae59",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7097",
"triggerID" : "0557906c0549d5550b2593517117cd08b630ae59",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 0557906c0549d5550b2593517117cd08b630ae59 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7097)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072930597
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* af8744478e331744286491ed8733f8bc760e3546 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072939079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* af8744478e331744286491ed8733f8bc760e3546 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083)
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 422aa2d6ddff9ad9b1022e78877befae3fd09c6d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072948481
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 422aa2d6ddff9ad9b1022e78877befae3fd09c6d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073112174
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
}, {
"hash" : "0557906c0549d5550b2593517117cd08b630ae59",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7097",
"triggerID" : "0557906c0549d5550b2593517117cd08b630ae59",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 3d88e80e20641007f105ac2ed61faa2703036453 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096)
* 0557906c0549d5550b2593517117cd08b630ae59 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7097)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072948481
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 422aa2d6ddff9ad9b1022e78877befae3fd09c6d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073064881
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 422aa2d6ddff9ad9b1022e78877befae3fd09c6d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084)
* 3d88e80e20641007f105ac2ed61faa2703036453 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1072939079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* af8744478e331744286491ed8733f8bc760e3546 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083)
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 422aa2d6ddff9ad9b1022e78877befae3fd09c6d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #5070: [HUDI-3663] Fixing Column Stats index to properly handle first Data Table commit
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #5070:
URL: https://github.com/apache/hudi/pull/5070#issuecomment-1073064881
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "af8744478e331744286491ed8733f8bc760e3546",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7083",
"triggerID" : "af8744478e331744286491ed8733f8bc760e3546",
"triggerType" : "PUSH"
}, {
"hash" : "28aba9f8744c52180a577886b85012ea655a72e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "28aba9f8744c52180a577886b85012ea655a72e1",
"triggerType" : "PUSH"
}, {
"hash" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084",
"triggerID" : "422aa2d6ddff9ad9b1022e78877befae3fd09c6d",
"triggerType" : "PUSH"
}, {
"hash" : "3d88e80e20641007f105ac2ed61faa2703036453",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096",
"triggerID" : "3d88e80e20641007f105ac2ed61faa2703036453",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 28aba9f8744c52180a577886b85012ea655a72e1 UNKNOWN
* 422aa2d6ddff9ad9b1022e78877befae3fd09c6d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7084)
* 3d88e80e20641007f105ac2ed61faa2703036453 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7096)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org