You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "codope (via GitHub)" <gi...@apache.org> on 2024/07/15 17:50:08 UTC
[PR] [HUDI-7989][HUDI-7990] Fix secondary index updates [hudi]
codope opened a new pull request, #11634:
URL: https://github.com/apache/hudi/pull/11634
### Change Logs
Secondary index update fails because the current logic created a wasteful `secondary_index_` partition in addition to the actual index partition at the time of initialization, and was unable to detect the index partition correctly. This PR fixes the problem:
- Removes unnecessary `secondary_index_` partition
- Fixes the MDT writer methods to work with exact MDT partition name instead of `MetadataPartitionType`.
- Added a test to udpate secondary keys
### Impact
fix index update
### Risk level (write none, low medium or high below)
low
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none"._
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1743752355
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1874,7 +1874,7 @@ public static HoodieData<HoodieRecord> readSecondaryKeysFromBaseFiles(HoodieEngi
List<String> logFilePaths = new ArrayList<>();
baseAndLogFiles.getValue().forEach(logFile -> logFilePaths.add(basePath + StoragePath.SEPARATOR + partition + StoragePath.SEPARATOR + logFile));
String filePath = baseAndLogFiles.getKey();
- Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(filePath(basePath, partition, filePath));
+ Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(FSUtils.constructAbsolutePath(basePath, filePath));
Review Comment:
Additional separator will be ignored. Added non-partition test. I have made all tests parameterized over tableType and (non)partition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1743759371
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/metadata/JavaHoodieBackedTableMetadataWriter.java:
##########
@@ -87,7 +87,7 @@ protected void initRegistry() {
}
@Override
- protected void commit(String instantTime, Map<MetadataPartitionType, HoodieData<HoodieRecord>> partitionRecordsMap) {
+ protected void commit(String instantTime, Map<String, HoodieData<HoodieRecord>> partitionRecordsMap) {
Review Comment:
The map initially meant to be per index type under the assumption that there is going to be one MDT partition per index type. However, functional or secondary index types break this assumption. As we discussed, I will add a separate abstraction to manage different indexes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989][HUDI-7990] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2229098477
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4b399d0313d3b0f70043fe503dae9aea0240193e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2267385168
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4249bc7c2b583171358ba640c24795e85e3e7487 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037)
* 969884fcb6e3de032bd656c8e85823ced1a999fc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2306374337
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3dc614cb9b24df29109c5b40df829ed9ad90581c Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2327070857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
}, {
"hash" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223",
"triggerID" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"triggerType" : "PUSH"
}, {
"hash" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
* ddfd1250ff8ca56d28119d07dc040c4b146fec63 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223)
* cef42490ae22481fad830a07f69733408e8f5c5f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989][HUDI-7990] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2229084641
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4b399d0313d3b0f70043fe503dae9aea0240193e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2242720418
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b6a2b51dcca2750c1dd969b2395486045342558e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865)
* e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2306303344
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c8a773f22e00a0827a8eebf76c0a22138b480105 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107)
* 3dc614cb9b24df29109c5b40df829ed9ad90581c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2326792696
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
}, {
"hash" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222)
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
* ddfd1250ff8ca56d28119d07dc040c4b146fec63 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2326877875
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
}, {
"hash" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223",
"triggerID" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222)
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
* ddfd1250ff8ca56d28119d07dc040c4b146fec63 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2258826079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944)
* 4249bc7c2b583171358ba640c24795e85e3e7487 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2258810491
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944)
* 4249bc7c2b583171358ba640c24795e85e3e7487 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2327110479
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
}, {
"hash" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223",
"triggerID" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"triggerType" : "PUSH"
}, {
"hash" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=225",
"triggerID" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
* ddfd1250ff8ca56d28119d07dc040c4b146fec63 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223)
* cef42490ae22481fad830a07f69733408e8f5c5f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=225)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2329373929
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
}, {
"hash" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223",
"triggerID" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"triggerType" : "PUSH"
}, {
"hash" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=225",
"triggerID" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"triggerType" : "PUSH"
}, {
"hash" : "901f2f6f7a6b797ef86749802cc8831db79f68a7",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=241",
"triggerID" : "901f2f6f7a6b797ef86749802cc8831db79f68a7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
* 901f2f6f7a6b797ef86749802cc8831db79f68a7 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=241)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1743787940
##########
hudi-common/src/test/java/org/apache/hudi/metadata/TestMetadataPartitionType.java:
##########
@@ -183,4 +187,40 @@ public void testGetMetadataPartitionRecordType() {
assertEquals(6, MetadataPartitionType.PARTITION_STATS.getRecordType());
assertEquals(7, MetadataPartitionType.SECONDARY_INDEX.getRecordType());
}
+
+ @Test
+ public void testGetFunctionalIndexPath() {
+ MetadataPartitionType partitionType = MetadataPartitionType.FUNCTIONAL_INDEX;
+ HoodieTableMetaClient metaClient = mock(HoodieTableMetaClient.class);
+ String indexName = "testIndex";
+
+ Map<String, HoodieIndexDefinition> indexDefinitions = new HashMap<>();
+ indexDefinitions.put(
+ indexName,
+ new HoodieIndexDefinition("func_index_testIndex", "column_stats", "lower", Collections.singletonList("name"), null));
+ HoodieIndexMetadata indexMetadata = new HoodieIndexMetadata(indexDefinitions);
+ when(metaClient.getIndexMetadata()).thenReturn(Option.of(indexMetadata));
+
+ String result = partitionType.getPartitionPath(metaClient, indexName);
+ assertEquals("func_index_testIndex", result);
+ }
+
+ @Test
+ public void testGetNonFunctionalIndexPath() {
+ MetadataPartitionType partitionType = MetadataPartitionType.COLUMN_STATS;
+ HoodieTableMetaClient metaClient = mock(HoodieTableMetaClient.class);
+
+ String result = partitionType.getPartitionPath(metaClient, null);
+ assertEquals(partitionType.getPartitionPath(), result);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989][HUDI-7990] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2229174990
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4b399d0313d3b0f70043fe503dae9aea0240193e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2316770120
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3dc614cb9b24df29109c5b40df829ed9ad90581c Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149)
* 3d898cec079263d0df507fc2b2760ad9ab3e21bf UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2326770713
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222)
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope merged PR #11634:
URL: https://github.com/apache/hudi/pull/11634
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1743769890
##########
hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java:
##########
@@ -319,6 +331,16 @@ public boolean isMetadataPartitionAvailable(HoodieTableMetaClient metaClient) {
this.recordType = recordType;
}
+ /**
+ * Get the partition name from the metadata partition type.
+ * NOTE: For certain types of metadata partition, such as functional index and secondary index,
+ * partition path defined enum is just the prefix to denote the type of metadata partition.
+ * The actual partition name is contained in the index definition.
+ */
+ public String getPartitionPath(HoodieTableMetaClient metaClient, String indexName) {
+ return getPartitionPath();
+ }
+
public String getPartitionPath() {
return partitionPath;
}
Review Comment:
It's still being used at some places where we don't have metaclient or indexName. Technically, this can be removed by making the other method accept `Option<HoodieTableMetaClient> metaClient, Option<String> indexName` as arguments, but it will change a lot of places. If you're ok, I can do it in a separate PR after landing this one (just to keep this PR a bit clean).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1744814299
##########
hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java:
##########
@@ -319,6 +331,16 @@ public boolean isMetadataPartitionAvailable(HoodieTableMetaClient metaClient) {
this.recordType = recordType;
}
+ /**
+ * Get the partition name from the metadata partition type.
+ * NOTE: For certain types of metadata partition, such as functional index and secondary index,
+ * partition path defined enum is just the prefix to denote the type of metadata partition.
+ * The actual partition name is contained in the index definition.
+ */
+ public String getPartitionPath(HoodieTableMetaClient metaClient, String indexName) {
+ return getPartitionPath();
+ }
+
public String getPartitionPath() {
return partitionPath;
}
Review Comment:
Makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1743757156
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/metadata/FlinkHoodieBackedTableMetadataWriter.java:
##########
@@ -109,12 +109,13 @@ protected List<HoodieRecord> convertHoodieDataToEngineSpecificData(HoodieData<Ho
}
@Override
- protected void bulkCommit(String instantTime, MetadataPartitionType partitionType, HoodieData<HoodieRecord> records, int fileGroupCount) {
- commitInternal(instantTime, Collections.singletonMap(partitionType, records), true, Option.empty());
+ protected void bulkCommit(String instantTime, String partitionName, HoodieData<HoodieRecord> records, int fileGroupCount) {
+ // TODO: functional and secondary index are not supported with Flink yet, but we should fix the partition name when we support them.
Review Comment:
Agree, but enum is limiting the extensibility. As discussed, we will introduce a separate abstract class to manage all the different indexes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2293139244
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4a5fb9700bcd37c709a8ac468158d48b360ac093 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72)
* b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2297848473
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c8a773f22e00a0827a8eebf76c0a22138b480105 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2316771663
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3dc614cb9b24df29109c5b40df829ed9ad90581c Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149)
* 3d898cec079263d0df507fc2b2760ad9ab3e21bf Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2326740053
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3d898cec079263d0df507fc2b2760ad9ab3e21bf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195)
* 5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1742488182
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -448,9 +448,9 @@ private boolean initializeFromFilesystem(String initializationTime, List<Metadat
// Perform the commit using bulkCommit
HoodieData<HoodieRecord> records = fileGroupCountAndRecordsPair.getValue();
- bulkCommit(commitTimeForPartition, partitionType, records, fileGroupCount);
- metadataMetaClient.reloadActiveTimeline();
String partitionPath = (partitionType == FUNCTIONAL_INDEX || partitionType == SECONDARY_INDEX) ? dataWriteConfig.getIndexingConfig().getIndexName() : partitionType.getPartitionPath();
+ bulkCommit(commitTimeForPartition, partitionPath, records, fileGroupCount);
Review Comment:
+1
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestSecondaryIndexPruning.scala:
##########
@@ -86,6 +196,82 @@ class TestSecondaryIndexPruning extends SecondaryIndexTestBase {
)
verifyQueryPredicate(hudiOpts, "not_record_key_col")
+ // update the secondary key column
+ spark.sql(s"update $tableName set not_record_key_col = 'xyz' where record_key_col = 'row1'")
+ // validate the secondary index records themselves
+ checkAnswer(s"select key, SecondaryIndexMetadata.recordKey, SecondaryIndexMetadata.isDeleted from hudi_metadata('$basePath') where type=7")(
+ Seq("abc", "row1", true),
+ Seq("cde", "row2", false),
+ Seq("def", "row3", false),
+ Seq("xyz", "row1", false)
+ )
+ // validate data and data skipping
+ checkAnswer(s"select ts, record_key_col, not_record_key_col, partition_key_col from $tableName where record_key_col = 'row1'")(
+ Seq(1, "row1", "xyz", "p1")
+ )
+ verifyQueryPredicate(hudiOpts, "not_record_key_col")
+ }
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = Array("COPY_ON_WRITE", "MERGE_ON_READ"))
+ def testSecondaryIndexWithPartitionStatsIndex(tableType: String): Unit = {
+ if (HoodieSparkUtils.gteqSpark3_3) {
+ var hudiOpts = commonOpts
+ hudiOpts = hudiOpts + (
+ DataSourceWriteOptions.TABLE_TYPE.key -> tableType,
+ DataSourceReadOptions.ENABLE_DATA_SKIPPING.key -> "true")
+ val sqlTableType = if (tableType.equals(HoodieTableType.COPY_ON_WRITE.name())) "cow" else "mor"
+ tableName += "test_secondary_index_with_partition_stats_index" + sqlTableType
+
+ spark.sql(
+ s"""
+ |create table $tableName (
+ | ts bigint,
+ | name string,
+ | record_key_col string,
+ | secondary_key_col string,
+ | partition_key_col string
+ |) using hudi
+ | options (
+ | primaryKey ='record_key_col',
+ | type = '$sqlTableType',
+ | hoodie.metadata.enable = 'true',
+ | hoodie.metadata.record.index.enable = 'true',
+ | hoodie.datasource.write.recordkey.field = 'record_key_col',
+ | 'hoodie.metadata.index.partition.stats.enable' = 'true',
+ | 'hoodie.metadata.index.column.stats.column.list' = 'name',
+ | hoodie.enable.data.skipping = 'true'
+ | )
+ | partitioned by(partition_key_col)
+ | location '$basePath'
+ """.stripMargin)
+ spark.sql(s"insert into $tableName values(1, 'gandhi', 'row1', 'abc', 'p1')")
+ spark.sql(s"insert into $tableName values(2, 'nehru', 'row2', 'cde', 'p2')")
+ spark.sql(s"insert into $tableName values(3, 'patel', 'row3', 'def', 'p2')")
+ // create secondary index
+ spark.sql(s"create index idx_secondary_key_col on $tableName using secondary_index(secondary_key_col)")
+ // validate index created successfully
+ metaClient = HoodieTableMetaClient.builder()
+ .setBasePath(basePath)
+ .setConf(HoodieTestUtils.getDefaultStorageConf)
+ .build()
+ assert(metaClient.getTableConfig.getMetadataPartitions.contains("secondary_index_idx_secondary_key_col"))
+ // validate the secondary index records themselves
+ checkAnswer(s"select key, SecondaryIndexMetadata.recordKey from hudi_metadata('$basePath') where type=7")(
+ Seq("abc", "row1"),
+ Seq("cde", "row2"),
+ Seq("def", "row3")
+ )
+ // validate data skipping with filters on secondary key column
+ spark.sql("set hoodie.metadata.enable=true")
+ spark.sql("set hoodie.enable.data.skipping=true")
+ spark.sql("set hoodie.fileIndex.dataSkippingFailureMode=strict")
+ checkAnswer(s"select ts, record_key_col, secondary_key_col, partition_key_col from $tableName where secondary_key_col = 'abc'")(
+ Seq(1, "row1", "abc", "p1")
+ )
+ verifyQueryPredicate(hudiOpts, "secondary_key_col")
Review Comment:
minor.
can you close HoodieFileIndex within verifyQueryPredicate
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1890,7 +1890,7 @@ public static HoodieData<HoodieRecord> readSecondaryKeysFromBaseFiles(HoodieEngi
List<String> logFilePaths = new ArrayList<>();
baseAndLogFiles.getValue().forEach(logFile -> logFilePaths.add(basePath + StoragePath.SEPARATOR + partition + StoragePath.SEPARATOR + logFile));
String filePath = baseAndLogFiles.getKey();
- Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(filePath(basePath, partition, filePath));
+ Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(FSUtils.constructAbsolutePath(basePath, filePath));
Review Comment:
btw, I did triage the basefilePath. Its coming from HoodieDeltaWriteStat. It refers to just base file name according to source code. We might have to revisit tests to see if we generate right values if we are mocking the HoodieDeltaWriteStat.
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestSecondaryIndexPruning.scala:
##########
@@ -105,7 +138,210 @@ class TestSecondaryIndexPruning extends SecondaryIndexTestBase {
}
}
- private def checkAnswer(sql: String)(expects: Seq[Any]*): Unit = {
- assertResult(expects.map(row => Row(row: _*)).toArray.sortBy(_.toString()))(spark.sql(sql).collect().sortBy(_.toString()))
+ @Test
+ def testSecondaryIndexPruningWithUpdates(): Unit = {
+ if (HoodieSparkUtils.gteqSpark3_3) {
+ var hudiOpts = commonOpts
+ hudiOpts = hudiOpts + (
+ DataSourceWriteOptions.TABLE_TYPE.key -> HoodieTableType.COPY_ON_WRITE.name(),
Review Comment:
lets also add a test for non partitioned table
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1890,7 +1890,7 @@ public static HoodieData<HoodieRecord> readSecondaryKeysFromBaseFiles(HoodieEngi
List<String> logFilePaths = new ArrayList<>();
baseAndLogFiles.getValue().forEach(logFile -> logFilePaths.add(basePath + StoragePath.SEPARATOR + partition + StoragePath.SEPARATOR + logFile));
String filePath = baseAndLogFiles.getKey();
- Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(filePath(basePath, partition, filePath));
+ Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(FSUtils.constructAbsolutePath(basePath, filePath));
Review Comment:
can we rename filePath in L 1877 to baseFilePath
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1890,7 +1890,7 @@ public static HoodieData<HoodieRecord> readSecondaryKeysFromBaseFiles(HoodieEngi
List<String> logFilePaths = new ArrayList<>();
baseAndLogFiles.getValue().forEach(logFile -> logFilePaths.add(basePath + StoragePath.SEPARATOR + partition + StoragePath.SEPARATOR + logFile));
String filePath = baseAndLogFiles.getKey();
- Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(filePath(basePath, partition, filePath));
+ Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(FSUtils.constructAbsolutePath(basePath, filePath));
Review Comment:
yes, same q as ethan. was there a bug before that we are fixing ?
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1874,7 +1874,7 @@ public static HoodieData<HoodieRecord> readSecondaryKeysFromBaseFiles(HoodieEngi
List<String> logFilePaths = new ArrayList<>();
baseAndLogFiles.getValue().forEach(logFile -> logFilePaths.add(basePath + StoragePath.SEPARATOR + partition + StoragePath.SEPARATOR + logFile));
String filePath = baseAndLogFiles.getKey();
- Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(filePath(basePath, partition, filePath));
+ Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(FSUtils.constructAbsolutePath(basePath, filePath));
Review Comment:
comment on L1875. Do we have tests for non-partitioned table for functional and sec index.
```
baseAndLogFiles.getValue().forEach(logFile -> logFilePaths.add(basePath + StoragePath.SEPARATOR + partition + StoragePath.SEPARATOR + logFile));
```
This could create a path w/ 2 separators (/) for non-partitioned table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1729190191
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1097,7 +1096,7 @@ private HoodieData<HoodieRecord> getFunctionalIndexUpdates(HoodieCommitMetadata
return getFunctionalIndexRecords(partitionFileSlicePairs, indexDefinition, dataMetaClient, parallelism, readerSchema, storageConf);
}
- private void updateSecondaryIndexIfPresent(HoodieCommitMetadata commitMetadata, Map<MetadataPartitionType, HoodieData<HoodieRecord>> partitionToRecordMap, HoodieData<WriteStatus> writeStatus) {
+ private void updateSecondaryIndexIfPresent(HoodieCommitMetadata commitMetadata, Map<String, HoodieData<HoodieRecord>> partitionToRecordMap, HoodieData<WriteStatus> writeStatus) {
Review Comment:
I am planning to introduce an Index manager and I will take it up in a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2329044528
> index on a column that is part of record key where in record key is complex. for eg, say record key is on colA and colB. can we add sec or functional index on colA.
Yes, sec index can be created. The only condition for sec index to create is that the column should exist and record index should exist. There is no check as such whether other index types are enabled for same column or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2292872680
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b635082f240f6238c6908362a63570f92c71b07f Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71)
* 4a5fb9700bcd37c709a8ac468158d48b360ac093 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2293137514
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4a5fb9700bcd37c709a8ac468158d48b360ac093 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72)
* b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2316884561
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3d898cec079263d0df507fc2b2760ad9ab3e21bf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1737510637
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -448,9 +448,9 @@ private boolean initializeFromFilesystem(String initializationTime, List<Metadat
// Perform the commit using bulkCommit
HoodieData<HoodieRecord> records = fileGroupCountAndRecordsPair.getValue();
- bulkCommit(commitTimeForPartition, partitionType, records, fileGroupCount);
- metadataMetaClient.reloadActiveTimeline();
String partitionPath = (partitionType == FUNCTIONAL_INDEX || partitionType == SECONDARY_INDEX) ? dataWriteConfig.getIndexingConfig().getIndexName() : partitionType.getPartitionPath();
+ bulkCommit(commitTimeForPartition, partitionPath, records, fileGroupCount);
Review Comment:
IMO, we should remove the following and use `MetadataPartitionType#getPartitionPath` as suggested above to avoid leaking index specific logic in this metadata writer. And we can keep passing the `MetadataPartitionType` instance as before, and use `#getPartitionPath` when needed.
```
String partitionPath = (partitionType == FUNCTIONAL_INDEX || partitionType == SECONDARY_INDEX) ? dataWriteConfig.getIndexingConfig().getIndexName() : partitionType.getPartitionPath();
```
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndexUtils.java:
##########
@@ -395,7 +395,7 @@ public static <R> HoodieRecord<R> createNewTaggedHoodieRecord(HoodieRecord<R> ol
* The actual partition name is contained in the index definition.
*/
public static String getPartitionNameFromPartitionType(MetadataPartitionType partitionType, HoodieTableMetaClient metaClient, String indexName) {
- if (MetadataPartitionType.FUNCTIONAL_INDEX.equals(partitionType)) {
+ if (MetadataPartitionType.FUNCTIONAL_INDEX.equals(partitionType) || MetadataPartitionType.SECONDARY_INDEX.equals(partitionType)) {
Review Comment:
Can this conversion logic be put into `MetadataPartitionType` for each type, by providing a new API
```
public enum MetadataPartitionType {
...
FUNCTIONAL_INDEX(HoodieTableMetadataUtil.PARTITION_NAME_FUNCTIONAL_INDEX_PREFIX, "func-index-", -1) {
...
@Override
public String getPartitionPath(...) {
return metaClient.getIndexMetadata().get().getIndexDefinitions().get(indexName).getIndexName();
}
...
},
...
public String getPartitionPath(...) {
...
}
}
```
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndexUtils.java:
##########
@@ -395,7 +395,7 @@ public static <R> HoodieRecord<R> createNewTaggedHoodieRecord(HoodieRecord<R> ol
* The actual partition name is contained in the index definition.
*/
public static String getPartitionNameFromPartitionType(MetadataPartitionType partitionType, HoodieTableMetaClient metaClient, String indexName) {
- if (MetadataPartitionType.FUNCTIONAL_INDEX.equals(partitionType)) {
+ if (MetadataPartitionType.FUNCTIONAL_INDEX.equals(partitionType) || MetadataPartitionType.SECONDARY_INDEX.equals(partitionType)) {
Review Comment:
In this way, we can avoid the confusion of what partition name or path to use, by consolidating the logic into this new API.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1396,27 +1395,25 @@ protected void preWrite(String instantTime) {
* records and hence is more suited to bulkInsert for write performance.
*
* @param instantTime - Action instant time for this commit
- * @param partitionType - The MDT partition to which records are to be committed
+ * @param partitionName - The MDT partition to which records are to be committed
* @param records - records to be bulk inserted
* @param fileGroupCount - The maximum number of file groups to which the records will be written.
*/
protected abstract void bulkCommit(
- String instantTime, MetadataPartitionType partitionType, HoodieData<HoodieRecord> records,
+ String instantTime, String partitionName, HoodieData<HoodieRecord> records,
Review Comment:
Could you add more details to the docs of `partitionName`? Is `partitionName` going to be different for secondary/functional index, e.g., each secondary/functional index definition has its own partition path?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1672,13 +1669,14 @@ private HoodieData<HoodieRecord> getRecordIndexUpserts(HoodieData<WriteStatus> w
}
private HoodieData<HoodieRecord> getRecordIndexReplacedRecords(HoodieReplaceCommitMetadata replaceCommitMetadata) {
- final HoodieMetadataFileSystemView fsView = new HoodieMetadataFileSystemView(dataMetaClient,
- dataMetaClient.getActiveTimeline(), metadata);
- List<Pair<String, HoodieBaseFile>> partitionBaseFilePairs = replaceCommitMetadata
- .getPartitionToReplaceFileIds()
- .keySet().stream().flatMap(partition
- -> fsView.getLatestBaseFiles(partition).map(f -> Pair.of(partition, f)))
- .collect(Collectors.toList());
+ List<Pair<String, HoodieBaseFile>> partitionBaseFilePairs;
+ try (HoodieMetadataFileSystemView fsView = new HoodieMetadataFileSystemView(dataMetaClient, dataMetaClient.getActiveTimeline(), metadata)) {
+ partitionBaseFilePairs = replaceCommitMetadata
+ .getPartitionToReplaceFileIds()
+ .keySet().stream()
+ .flatMap(partition -> fsView.getLatestBaseFiles(partition).map(f -> Pair.of(partition, f)))
+ .collect(Collectors.toList());
Review Comment:
Move `return` statement inside to avoid separate `partitionBaseFilePairs` declaration.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -448,9 +448,9 @@ private boolean initializeFromFilesystem(String initializationTime, List<Metadat
// Perform the commit using bulkCommit
HoodieData<HoodieRecord> records = fileGroupCountAndRecordsPair.getValue();
- bulkCommit(commitTimeForPartition, partitionType, records, fileGroupCount);
- metadataMetaClient.reloadActiveTimeline();
String partitionPath = (partitionType == FUNCTIONAL_INDEX || partitionType == SECONDARY_INDEX) ? dataWriteConfig.getIndexingConfig().getIndexName() : partitionType.getPartitionPath();
+ bulkCommit(commitTimeForPartition, partitionPath, records, fileGroupCount);
Review Comment:
We can avoid most type changes in that way in the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2327019752
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
}, {
"hash" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223",
"triggerID" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
* ddfd1250ff8ca56d28119d07dc040c4b146fec63 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2242734815
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b6a2b51dcca2750c1dd969b2395486045342558e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865)
* e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2242956324
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989][HUDI-7990] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2230161269
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b6a2b51dcca2750c1dd969b2395486045342558e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2267390845
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4249bc7c2b583171358ba640c24795e85e3e7487 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037)
* 969884fcb6e3de032bd656c8e85823ced1a999fc Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1697249355
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1097,7 +1096,7 @@ private HoodieData<HoodieRecord> getFunctionalIndexUpdates(HoodieCommitMetadata
return getFunctionalIndexRecords(partitionFileSlicePairs, indexDefinition, dataMetaClient, parallelism, readerSchema, storageConf);
}
- private void updateSecondaryIndexIfPresent(HoodieCommitMetadata commitMetadata, Map<MetadataPartitionType, HoodieData<HoodieRecord>> partitionToRecordMap, HoodieData<WriteStatus> writeStatus) {
+ private void updateSecondaryIndexIfPresent(HoodieCommitMetadata commitMetadata, Map<String, HoodieData<HoodieRecord>> partitionToRecordMap, HoodieData<WriteStatus> writeStatus) {
Review Comment:
Good suggestion, i'm working on it. Can we land #11597 first?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2297780870
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73)
* c8a773f22e00a0827a8eebf76c0a22138b480105 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2292968768
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4a5fb9700bcd37c709a8ac468158d48b360ac093 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989][HUDI-7990] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2230103146
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4b399d0313d3b0f70043fe503dae9aea0240193e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860)
* b6a2b51dcca2750c1dd969b2395486045342558e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1743755468
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -448,9 +448,9 @@ private boolean initializeFromFilesystem(String initializationTime, List<Metadat
// Perform the commit using bulkCommit
HoodieData<HoodieRecord> records = fileGroupCountAndRecordsPair.getValue();
- bulkCommit(commitTimeForPartition, partitionType, records, fileGroupCount);
- metadataMetaClient.reloadActiveTimeline();
String partitionPath = (partitionType == FUNCTIONAL_INDEX || partitionType == SECONDARY_INDEX) ? dataWriteConfig.getIndexingConfig().getIndexName() : partitionType.getPartitionPath();
+ bulkCommit(commitTimeForPartition, partitionPath, records, fileGroupCount);
Review Comment:
Have done some of the refactoring. Discussed offline with Ethan - we need a separate abstraction, enum is not sufficient to pack everything. I will do that in a subsequent PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1743753919
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1890,7 +1890,7 @@ public static HoodieData<HoodieRecord> readSecondaryKeysFromBaseFiles(HoodieEngi
List<String> logFilePaths = new ArrayList<>();
baseAndLogFiles.getValue().forEach(logFile -> logFilePaths.add(basePath + StoragePath.SEPARATOR + partition + StoragePath.SEPARATOR + logFile));
String filePath = baseAndLogFiles.getKey();
- Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(filePath(basePath, partition, filePath));
+ Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(FSUtils.constructAbsolutePath(basePath, filePath));
Review Comment:
Yes partition is already included in file path - `partition_key_col=p1/11e1fc7b-c623-4fbc-b4e0-163288801a36-0_0-86-242_20240904181553915.parquet`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2292811119
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 969884fcb6e3de032bd656c8e85823ced1a999fc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064)
* b635082f240f6238c6908362a63570f92c71b07f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1737516761
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1890,7 +1890,7 @@ public static HoodieData<HoodieRecord> readSecondaryKeysFromBaseFiles(HoodieEngi
List<String> logFilePaths = new ArrayList<>();
baseAndLogFiles.getValue().forEach(logFile -> logFilePaths.add(basePath + StoragePath.SEPARATOR + partition + StoragePath.SEPARATOR + logFile));
String filePath = baseAndLogFiles.getKey();
- Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(filePath(basePath, partition, filePath));
+ Option<StoragePath> dataFilePath = filePath.isEmpty() ? Option.empty() : Option.of(FSUtils.constructAbsolutePath(basePath, filePath));
Review Comment:
Is `partition` already included in `filePath`?
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestSecondaryIndexPruning.scala:
##########
@@ -105,7 +138,210 @@ class TestSecondaryIndexPruning extends SecondaryIndexTestBase {
}
}
- private def checkAnswer(sql: String)(expects: Seq[Any]*): Unit = {
- assertResult(expects.map(row => Row(row: _*)).toArray.sortBy(_.toString()))(spark.sql(sql).collect().sortBy(_.toString()))
+ @Test
+ def testSecondaryIndexPruningWithUpdates(): Unit = {
+ if (HoodieSparkUtils.gteqSpark3_3) {
+ var hudiOpts = commonOpts
+ hudiOpts = hudiOpts + (
+ DataSourceWriteOptions.TABLE_TYPE.key -> HoodieTableType.COPY_ON_WRITE.name(),
Review Comment:
Also add a test with a permutation of MOR table?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1692359991
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1097,7 +1096,7 @@ private HoodieData<HoodieRecord> getFunctionalIndexUpdates(HoodieCommitMetadata
return getFunctionalIndexRecords(partitionFileSlicePairs, indexDefinition, dataMetaClient, parallelism, readerSchema, storageConf);
}
- private void updateSecondaryIndexIfPresent(HoodieCommitMetadata commitMetadata, Map<MetadataPartitionType, HoodieData<HoodieRecord>> partitionToRecordMap, HoodieData<WriteStatus> writeStatus) {
+ private void updateSecondaryIndexIfPresent(HoodieCommitMetadata commitMetadata, Map<String, HoodieData<HoodieRecord>> partitionToRecordMap, HoodieData<WriteStatus> writeStatus) {
Review Comment:
Should we abstract these logic into a separate index class containing the logic of updating index? It will be easier to maintain such logic and avoid bugs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2292812030
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 969884fcb6e3de032bd656c8e85823ced1a999fc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064)
* b635082f240f6238c6908362a63570f92c71b07f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2292829674
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 969884fcb6e3de032bd656c8e85823ced1a999fc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064)
* b635082f240f6238c6908362a63570f92c71b07f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71)
* 4a5fb9700bcd37c709a8ac468158d48b360ac093 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2292830613
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b635082f240f6238c6908362a63570f92c71b07f Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71)
* 4a5fb9700bcd37c709a8ac468158d48b360ac093 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1744813551
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1396,27 +1395,25 @@ protected void preWrite(String instantTime) {
* records and hence is more suited to bulkInsert for write performance.
*
* @param instantTime - Action instant time for this commit
- * @param partitionType - The MDT partition to which records are to be committed
+ * @param partitionName - The MDT partition to which records are to be committed
* @param records - records to be bulk inserted
* @param fileGroupCount - The maximum number of file groups to which the records will be written.
*/
protected abstract void bulkCommit(
- String instantTime, MetadataPartitionType partitionType, HoodieData<HoodieRecord> records,
+ String instantTime, String partitionName, HoodieData<HoodieRecord> records,
Review Comment:
Synced offline that the `partitionName` is going to be different per index definition within secondary and functional index. Enum may not be sufficient to handle the difference. So we'll use `String` now and refactor it as a difference class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1744814469
##########
hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java:
##########
@@ -319,6 +331,16 @@ public boolean isMetadataPartitionAvailable(HoodieTableMetaClient metaClient) {
this.recordType = recordType;
}
+ /**
+ * Get the partition name from the metadata partition type.
+ * NOTE: For certain types of metadata partition, such as functional index and secondary index,
+ * partition path defined enum is just the prefix to denote the type of metadata partition.
+ * The actual partition name is contained in the index definition.
+ */
+ public String getPartitionPath(HoodieTableMetaClient metaClient, String indexName) {
+ return getPartitionPath();
+ }
+
public String getPartitionPath() {
return partitionPath;
}
Review Comment:
Let's have a JIRA to track it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2329079565
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
}, {
"hash" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223",
"triggerID" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"triggerType" : "PUSH"
}, {
"hash" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=225",
"triggerID" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"triggerType" : "PUSH"
}, {
"hash" : "901f2f6f7a6b797ef86749802cc8831db79f68a7",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "901f2f6f7a6b797ef86749802cc8831db79f68a7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
* cef42490ae22481fad830a07f69733408e8f5c5f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=225)
* 901f2f6f7a6b797ef86749802cc8831db79f68a7 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1742771361
##########
hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java:
##########
@@ -319,6 +331,16 @@ public boolean isMetadataPartitionAvailable(HoodieTableMetaClient metaClient) {
this.recordType = recordType;
}
+ /**
+ * Get the partition name from the metadata partition type.
+ * NOTE: For certain types of metadata partition, such as functional index and secondary index,
+ * partition path defined enum is just the prefix to denote the type of metadata partition.
+ * The actual partition name is contained in the index definition.
+ */
+ public String getPartitionPath(HoodieTableMetaClient metaClient, String indexName) {
+ return getPartitionPath();
+ }
+
public String getPartitionPath() {
return partitionPath;
}
Review Comment:
We should remove this API now.
##########
hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java:
##########
@@ -319,6 +331,16 @@ public boolean isMetadataPartitionAvailable(HoodieTableMetaClient metaClient) {
this.recordType = recordType;
}
+ /**
+ * Get the partition name from the metadata partition type.
+ * NOTE: For certain types of metadata partition, such as functional index and secondary index,
+ * partition path defined enum is just the prefix to denote the type of metadata partition.
+ * The actual partition name is contained in the index definition.
+ */
+ public String getPartitionPath(HoodieTableMetaClient metaClient, String indexName) {
+ return getPartitionPath();
Review Comment:
Return `partitionPath` instead.
##########
hudi-common/src/test/java/org/apache/hudi/metadata/TestMetadataPartitionType.java:
##########
@@ -183,4 +187,40 @@ public void testGetMetadataPartitionRecordType() {
assertEquals(6, MetadataPartitionType.PARTITION_STATS.getRecordType());
assertEquals(7, MetadataPartitionType.SECONDARY_INDEX.getRecordType());
}
+
+ @Test
+ public void testGetFunctionalIndexPath() {
+ MetadataPartitionType partitionType = MetadataPartitionType.FUNCTIONAL_INDEX;
+ HoodieTableMetaClient metaClient = mock(HoodieTableMetaClient.class);
+ String indexName = "testIndex";
+
+ Map<String, HoodieIndexDefinition> indexDefinitions = new HashMap<>();
+ indexDefinitions.put(
+ indexName,
+ new HoodieIndexDefinition("func_index_testIndex", "column_stats", "lower", Collections.singletonList("name"), null));
+ HoodieIndexMetadata indexMetadata = new HoodieIndexMetadata(indexDefinitions);
+ when(metaClient.getIndexMetadata()).thenReturn(Option.of(indexMetadata));
+
+ String result = partitionType.getPartitionPath(metaClient, indexName);
+ assertEquals("func_index_testIndex", result);
+ }
+
+ @Test
+ public void testGetNonFunctionalIndexPath() {
+ MetadataPartitionType partitionType = MetadataPartitionType.COLUMN_STATS;
+ HoodieTableMetaClient metaClient = mock(HoodieTableMetaClient.class);
+
+ String result = partitionType.getPartitionPath(metaClient, null);
+ assertEquals(partitionType.getPartitionPath(), result);
Review Comment:
Cover all partition types?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1742778508
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1396,27 +1395,25 @@ protected void preWrite(String instantTime) {
* records and hence is more suited to bulkInsert for write performance.
*
* @param instantTime - Action instant time for this commit
- * @param partitionType - The MDT partition to which records are to be committed
+ * @param partitionName - The MDT partition to which records are to be committed
* @param records - records to be bulk inserted
* @param fileGroupCount - The maximum number of file groups to which the records will be written.
*/
protected abstract void bulkCommit(
- String instantTime, MetadataPartitionType partitionType, HoodieData<HoodieRecord> records,
+ String instantTime, String partitionName, HoodieData<HoodieRecord> records,
Review Comment:
Is this an issue we need to fix? Do we have a JIRA to fix it?
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/metadata/JavaHoodieBackedTableMetadataWriter.java:
##########
@@ -87,7 +87,7 @@ protected void initRegistry() {
}
@Override
- protected void commit(String instantTime, Map<MetadataPartitionType, HoodieData<HoodieRecord>> partitionRecordsMap) {
+ protected void commit(String instantTime, Map<String, HoodieData<HoodieRecord>> partitionRecordsMap) {
Review Comment:
I still have this question: could we still keep `MetadataPartitionType` in the map and all other places?
Whenever needed, call `metadataPartitionType.getPartitionPath` and use the value. Does this map keep records per partition or per index type, because functional or secondary indexes may contain multiple index definitions? Even so, this PR does not seem to fix that mapping.
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/metadata/FlinkHoodieBackedTableMetadataWriter.java:
##########
@@ -109,12 +109,13 @@ protected List<HoodieRecord> convertHoodieDataToEngineSpecificData(HoodieData<Ho
}
@Override
- protected void bulkCommit(String instantTime, MetadataPartitionType partitionType, HoodieData<HoodieRecord> records, int fileGroupCount) {
- commitInternal(instantTime, Collections.singletonMap(partitionType, records), true, Option.empty());
+ protected void bulkCommit(String instantTime, String partitionName, HoodieData<HoodieRecord> records, int fileGroupCount) {
+ // TODO: functional and secondary index are not supported with Flink yet, but we should fix the partition name when we support them.
Review Comment:
Using a strong type `MetadataPartitionType` is better than `String` so developer knows what's expected here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2326766845
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3d898cec079263d0df507fc2b2760ad9ab3e21bf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195)
* 5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222)
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2267451698
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 969884fcb6e3de032bd656c8e85823ced1a999fc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2293254146
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1744813551
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1396,27 +1395,25 @@ protected void preWrite(String instantTime) {
* records and hence is more suited to bulkInsert for write performance.
*
* @param instantTime - Action instant time for this commit
- * @param partitionType - The MDT partition to which records are to be committed
+ * @param partitionName - The MDT partition to which records are to be committed
* @param records - records to be bulk inserted
* @param fileGroupCount - The maximum number of file groups to which the records will be written.
*/
protected abstract void bulkCommit(
- String instantTime, MetadataPartitionType partitionType, HoodieData<HoodieRecord> records,
+ String instantTime, String partitionName, HoodieData<HoodieRecord> records,
Review Comment:
Synced offline that the `partitionName` is going to be different per index definition within secondary and functional index.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11634:
URL: https://github.com/apache/hudi/pull/11634#discussion_r1744814608
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/metadata/JavaHoodieBackedTableMetadataWriter.java:
##########
@@ -87,7 +87,7 @@ protected void initRegistry() {
}
@Override
- protected void commit(String instantTime, Map<MetadataPartitionType, HoodieData<HoodieRecord>> partitionRecordsMap) {
+ protected void commit(String instantTime, Map<String, HoodieData<HoodieRecord>> partitionRecordsMap) {
Review Comment:
Sg
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2329175337
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
}, {
"hash" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223",
"triggerID" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"triggerType" : "PUSH"
}, {
"hash" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=225",
"triggerID" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"triggerType" : "PUSH"
}, {
"hash" : "901f2f6f7a6b797ef86749802cc8831db79f68a7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=241",
"triggerID" : "901f2f6f7a6b797ef86749802cc8831db79f68a7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
* cef42490ae22481fad830a07f69733408e8f5c5f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=225)
* 901f2f6f7a6b797ef86749802cc8831db79f68a7 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=241)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2326736147
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3d898cec079263d0df507fc2b2760ad9ab3e21bf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195)
* 5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2327239879
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
}, {
"hash" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=195",
"triggerID" : "3d898cec079263d0df507fc2b2760ad9ab3e21bf",
"triggerType" : "PUSH"
}, {
"hash" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=222",
"triggerID" : "5908cec63e5b7eaba3c2a47fe3cbc2c2bb8a692b",
"triggerType" : "PUSH"
}, {
"hash" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4d9a90b172b207e2b81251ac146d3d4b4ef7662f",
"triggerType" : "PUSH"
}, {
"hash" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=223",
"triggerID" : "ddfd1250ff8ca56d28119d07dc040c4b146fec63",
"triggerType" : "PUSH"
}, {
"hash" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=225",
"triggerID" : "cef42490ae22481fad830a07f69733408e8f5c5f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4d9a90b172b207e2b81251ac146d3d4b4ef7662f UNKNOWN
* cef42490ae22481fad830a07f69733408e8f5c5f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=225)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2258927030
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4249bc7c2b583171358ba640c24795e85e3e7487 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2297779809
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73)
* c8a773f22e00a0827a8eebf76c0a22138b480105 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7989] Fix secondary index updates [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11634:
URL: https://github.com/apache/hudi/pull/11634#issuecomment-2306304513
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24860",
"triggerID" : "4b399d0313d3b0f70043fe503dae9aea0240193e",
"triggerType" : "PUSH"
}, {
"hash" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24865",
"triggerID" : "b6a2b51dcca2750c1dd969b2395486045342558e",
"triggerType" : "PUSH"
}, {
"hash" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24944",
"triggerID" : "e04769bd74ac822b5fc7dcfe45e6e3d38633f2c9",
"triggerType" : "PUSH"
}, {
"hash" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25037",
"triggerID" : "4249bc7c2b583171358ba640c24795e85e3e7487",
"triggerType" : "PUSH"
}, {
"hash" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=25064",
"triggerID" : "969884fcb6e3de032bd656c8e85823ced1a999fc",
"triggerType" : "PUSH"
}, {
"hash" : "b635082f240f6238c6908362a63570f92c71b07f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=71",
"triggerID" : "b635082f240f6238c6908362a63570f92c71b07f",
"triggerType" : "PUSH"
}, {
"hash" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=72",
"triggerID" : "4a5fb9700bcd37c709a8ac468158d48b360ac093",
"triggerType" : "PUSH"
}, {
"hash" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=73",
"triggerID" : "b4f32b18b14a922f97a0f16d5fa25f6b15cf2dcc",
"triggerType" : "PUSH"
}, {
"hash" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107",
"triggerID" : "c8a773f22e00a0827a8eebf76c0a22138b480105",
"triggerType" : "PUSH"
}, {
"hash" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149",
"triggerID" : "3dc614cb9b24df29109c5b40df829ed9ad90581c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c8a773f22e00a0827a8eebf76c0a22138b480105 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=107)
* 3dc614cb9b24df29109c5b40df829ed9ad90581c Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=149)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org