You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org> on 2023/11/07 14:23:29 UTC
[PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
zhuanshenbsj1 opened a new pull request, #10002:
URL: https://github.com/apache/hudi/pull/10002
### Change Logs
Mainly to solve the OOM when generating clean plan if there are too many partitions.
1. Clean allows grouping execution by setting cleanerParallelism.
2. After generating cleaning information for a partition, clean the metadata of that partition in TableFileSystemView.
3. Optimize the use of read and write locks in AbstractTableFileSystemView.
### Impact
_Describe any public API or user-facing feature change or any performance impact._
### Risk level (write none, low medium or high below)
_If medium or high, explain what verification was done to mitigate the risks._
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817339409
Related issue for another cleaner OOM improvement: https://github.com/apache/hudi/pull/10116
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1822749867
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fc27baa8c2df9135bc6e4b0d14e50a127ecb434f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1825437733
Rebase & add ut, cc @danny0405.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823820723
Thanks for the contribution, I have reviewed and created a patch:
[7041.patch.zip](https://github.com/apache/hudi/files/13446123/7041.patch.zip)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1399248053
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -305,8 +305,13 @@ void removeFileGroupsInPendingClustering(Stream<Pair<HoodieFileGroupId, HoodieIn
*/
@Override
Stream<HoodieFileGroup> fetchAllStoredFileGroups(String partition) {
- final List<HoodieFileGroup> fileGroups = new ArrayList<>(partitionToFileGroupsMap.get(partition));
- return fileGroups.stream();
+ readLock.lock();
+ try {
+ final List<HoodieFileGroup> fileGroups = new ArrayList<>(partitionToFileGroupsMap.get(partition));
+ return fileGroups.stream();
+ } finally {
+ readLock.unlock();
+ }
Review Comment:
Ensure thread safety for partitionToFileGroupsMap.However, some read locks can indeed be removed because they are maps themselves
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -305,8 +305,13 @@ void removeFileGroupsInPendingClustering(Stream<Pair<HoodieFileGroupId, HoodieIn
*/
@Override
Stream<HoodieFileGroup> fetchAllStoredFileGroups(String partition) {
- final List<HoodieFileGroup> fileGroups = new ArrayList<>(partitionToFileGroupsMap.get(partition));
- return fileGroups.stream();
+ readLock.lock();
+ try {
+ final List<HoodieFileGroup> fileGroups = new ArrayList<>(partitionToFileGroupsMap.get(partition));
+ return fileGroups.stream();
+ } finally {
+ readLock.unlock();
+ }
Review Comment:
Ensure thread safety for partitionToFileGroupsMap.However, some read locks can indeed be removed because they are maps themselves
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817901646
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 85d14854ac69b515c4557d18f8b09d1debb5fb9a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005)
* 356bb49429ab96b078dbf5bd968418f01729619d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1404479408
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -202,6 +207,13 @@ private Map<String, String> getParamsWithPartitionPath(String partitionPath) {
return paramsMap;
}
+ private Map<String, String> getParamsWithPartitionPaths(List<String> partitionPaths) {
Review Comment:
Removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the memory usage of timeline server for table service [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #10002:
URL: https://github.com/apache/hudi/pull/10002
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1399962836
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java:
##########
@@ -491,10 +491,13 @@ public Pair<Boolean, List<CleanFileInfo>> getDeletePaths(String partitionPath, O
Pair<Boolean, List<CleanFileInfo>> deletePaths;
if (policy == HoodieCleaningPolicy.KEEP_LATEST_COMMITS) {
deletePaths = getFilesToCleanKeepingLatestCommits(partitionPath, earliestCommitToRetain);
+ fileSystemView.cleanFileGroupsInPartitions(Collections.singletonList(partitionPath));
} else if (policy == HoodieCleaningPolicy.KEEP_LATEST_FILE_VERSIONS) {
deletePaths = getFilesToCleanKeepingLatestVersions(partitionPath);
+ fileSystemView.cleanFileGroupsInPartitions(Collections.singletonList(partitionPath));
} else if (policy == HoodieCleaningPolicy.KEEP_LATEST_BY_HOURS) {
deletePaths = getFilesToCleanKeepingLatestHours(partitionPath, earliestCommitToRetain);
+ fileSystemView.cleanFileGroupsInPartitions(Collections.singletonList(partitionPath));
Review Comment:

It seems that some scenarios do not require calling this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1398364599
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -378,19 +383,45 @@ protected Option<Pair<String, CompactionOperation>> getPendingLogCompactionOpera
@Override
protected boolean isPartitionAvailableInStore(String partitionPath) {
- return partitionToFileGroupsMap.containsKey(partitionPath);
+ fsLock.readLock();
+ try {
+ return partitionToFileGroupsMap.containsKey(partitionPath);
+ } finally {
+ fsLock.readUnlock();
+ }
}
@Override
protected void storePartitionView(String partitionPath, List<HoodieFileGroup> fileGroups) {
- LOG.debug("Adding file-groups for partition :" + partitionPath + ", #FileGroups=" + fileGroups.size());
- List<HoodieFileGroup> newList = new ArrayList<>(fileGroups);
- partitionToFileGroupsMap.put(partitionPath, newList);
+ fsLock.writeLock();
+ try {
+ LOG.debug("Adding file-groups for partition :" + partitionPath + ", #FileGroups=" + fileGroups.size());
+ List<HoodieFileGroup> newList = new ArrayList<>(fileGroups);
+ partitionToFileGroupsMap.put(partitionPath, newList);
+ } finally {
+ fsLock.writeUnlock();
+ }
+ }
+
+ @Override
+ protected void removePartitionView(String partitionPath) {
+ fsLock.writeLock();
Review Comment:
Adjust to - > assert hasWriteLock
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -378,19 +383,45 @@ protected Option<Pair<String, CompactionOperation>> getPendingLogCompactionOpera
@Override
protected boolean isPartitionAvailableInStore(String partitionPath) {
- return partitionToFileGroupsMap.containsKey(partitionPath);
+ fsLock.readLock();
+ try {
+ return partitionToFileGroupsMap.containsKey(partitionPath);
+ } finally {
+ fsLock.readUnlock();
+ }
}
@Override
protected void storePartitionView(String partitionPath, List<HoodieFileGroup> fileGroups) {
- LOG.debug("Adding file-groups for partition :" + partitionPath + ", #FileGroups=" + fileGroups.size());
- List<HoodieFileGroup> newList = new ArrayList<>(fileGroups);
- partitionToFileGroupsMap.put(partitionPath, newList);
+ fsLock.writeLock();
Review Comment:
Adjust to - > assert hasWriteLock
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1822665716
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 476071649c9cf7048435217bd30d246b18d9be65 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1800913385
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d10a137bff419d0d4befb5dac8380ac0bf0f12f8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705)
* 4dcb8f7bea46847202c2444e1a99901484239f4f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823913218
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 35fed0de0587b411f9470e1c69db43501df5a725 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121)
* 6024de8ab05dd38e9cdb58afc10b70991542c392 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the memory usage of timeline server for table service [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1826311068
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
}, {
"hash" : "8723081c68a37fa68b9665fad562d84543ed8533",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143",
"triggerID" : "8723081c68a37fa68b9665fad562d84543ed8533",
"triggerType" : "PUSH"
}, {
"hash" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144",
"triggerID" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"triggerType" : "PUSH"
}, {
"hash" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144",
"triggerID" : "1826280731",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 3d733303d36f8f9531337a15d6dc265b8a0b5cff Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1825460606
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6024de8ab05dd38e9cdb58afc10b70991542c392 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122)
* bc5aa1807fab83dd00391b1dfb86d4e213692c7c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1825883719
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bc5aa1807fab83dd00391b1dfb86d4e213692c7c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139)
* a400e5e76089c0af8236fc938ea6b432d8c39da7 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817850698
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 45443c1149800a6cbe40b0fd8a4080ac7d4f91c4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004)
* 85d14854ac69b515c4557d18f8b09d1debb5fb9a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817833634
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 45443c1149800a6cbe40b0fd8a4080ac7d4f91c4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1815793435
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb60d3f2fe5737fc43a700bcc6c37806fe48868a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722)
* 47200120dd37ebaee77c583628bfddac1f564b3b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817819508
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b4aebce1c5d648e0c06d2330a930eaa3e24466e7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003)
* 45443c1149800a6cbe40b0fd8a4080ac7d4f91c4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1399250779
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/PriorityBasedFileSystemView.java:
##########
@@ -222,6 +222,16 @@ public Stream<HoodieFileGroup> getAllFileGroups(String partitionPath) {
return execute(partitionPath, preferredView::getAllFileGroups, secondaryView::getAllFileGroups);
}
+ @Override
+ public boolean isPartitionAvailableInStoreForTest(String partitionPath) {
+ return execute(partitionPath, preferredView::isPartitionAvailableInStoreForTest, secondaryView::isPartitionAvailableInStoreForTest);
+ }
Review Comment:
But without this method, it seems difficult to confirm whether the partition metadata has been cleaned up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1399228278
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -164,27 +164,32 @@ public List<HoodieFileGroup> addFilesToView(FileStatus[] statuses) {
List<HoodieFileGroup> fileGroups = buildFileGroups(statuses, visibleCommitsAndCompactionTimeline, true);
long fgBuildTimeTakenMs = timer.endTimer();
timer.startTimer();
- // Group by partition for efficient updates for both InMemory and DiskBased structures.
- fileGroups.stream().collect(Collectors.groupingBy(HoodieFileGroup::getPartitionPath)).forEach((partition, value) -> {
- if (!isPartitionAvailableInStore(partition)) {
- if (bootstrapIndex.useIndex()) {
- try (BootstrapIndex.IndexReader reader = bootstrapIndex.createReader()) {
- LOG.info("Bootstrap Index available for partition " + partition);
- List<BootstrapFileMapping> sourceFileMappings =
- reader.getSourceFileMappingForPartition(partition);
- addBootstrapBaseFileMapping(sourceFileMappings.stream()
- .map(s -> new BootstrapBaseFileMapping(new HoodieFileGroupId(s.getPartitionPath(),
- s.getFileId()), s.getBootstrapFileStatus())));
+ writeLock.lock();
+ try {
Review Comment:
foreach(partition) : isPartitionAvailableInStore(partition) -> storePartitionView(partition, value);
This must be atomic to ensure thread safety for partitionToFileGroupsMap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1399217769
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java:
##########
@@ -118,17 +120,25 @@ HoodieCleanerPlan requestClean(HoodieEngineContext context) {
context.setJobStatus(this.getClass().getSimpleName(), "Generating list of file slices to be cleaned: " + config.getTableName());
- Map<String, Pair<Boolean, List<CleanFileInfo>>> cleanOpsWithPartitionMeta = context
- .map(partitionsToClean, partitionPathToClean -> Pair.of(partitionPathToClean, planner.getDeletePaths(partitionPathToClean, earliestInstant)), cleanerParallelism)
- .stream()
- .collect(Collectors.toMap(Pair::getKey, Pair::getValue));
+ Map<String, List<HoodieCleanFileInfo>> cleanOps = new HashMap<>();
+ List<String> partitionsToDelete = new ArrayList<>();
+ for (int i = 0; i < partitionsToClean.size(); i += cleanerParallelism) {
+ List<String> subPartitionsToClean = partitionsToClean.subList(i, Math.min(i + cleanerParallelism, partitionsToClean.size()));
Review Comment:
Yes, and this is also reasonable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1820151050
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0d2b1cf6af86fed4821e9ec5b41e8477ad915b64 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008)
* 03e698c23b11e2b1891b755ddae1e6431796680d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1801060265
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4dcb8f7bea46847202c2444e1a99901484239f4f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718)
* bb60d3f2fe5737fc43a700bcc6c37806fe48868a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823920316
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 35fed0de0587b411f9470e1c69db43501df5a725 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121)
* 6024de8ab05dd38e9cdb58afc10b70991542c392 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1404222907
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -408,6 +408,11 @@ protected void removeReplacedFileIdsAtInstants(Set<String> instants) {
fgIdToReplaceInstants.entrySet().removeIf(entry -> instants.contains(entry.getValue().getTimestamp()));
}
+ @Override
+ protected boolean hasReplacedFilesInPartition(String partitionPath) {
+ return fgIdToReplaceInstants.keySet().stream().anyMatch(fg -> fg.getPartitionPath().equals(partitionPath));
Review Comment:
we should also format the `partitionPath` to trim the `/` if possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1404706614
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -408,6 +408,11 @@ protected void removeReplacedFileIdsAtInstants(Set<String> instants) {
fgIdToReplaceInstants.entrySet().removeIf(entry -> instants.contains(entry.getValue().getTimestamp()));
}
+ @Override
+ protected boolean hasReplacedFilesInPartition(String partitionPath) {
+ return fgIdToReplaceInstants.keySet().stream().anyMatch(fg -> fg.getPartitionPath().equals(partitionPath));
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the memory usage of timeline server for table service [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1826286031
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
}, {
"hash" : "8723081c68a37fa68b9665fad562d84543ed8533",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143",
"triggerID" : "8723081c68a37fa68b9665fad562d84543ed8533",
"triggerType" : "PUSH"
}, {
"hash" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144",
"triggerID" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"triggerType" : "PUSH"
}, {
"hash" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1826280731",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 3d733303d36f8f9531337a15d6dc265b8a0b5cff Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823959739
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6024de8ab05dd38e9cdb58afc10b70991542c392 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823878156
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fc27baa8c2df9135bc6e4b0d14e50a127ecb434f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100)
* 35fed0de0587b411f9470e1c69db43501df5a725 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1402972184
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RemoteHoodieTableFileSystemView.java:
##########
@@ -202,6 +207,13 @@ private Map<String, String> getParamsWithPartitionPath(String partitionPath) {
return paramsMap;
}
+ private Map<String, String> getParamsWithPartitionPaths(List<String> partitionPaths) {
Review Comment:
This method is useless now right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the memory usage of timeline server for table service [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1833053275
> Hi, Could you please merge the 0.14.1 (0.x) branch to support it? When will version 0.14.1 be released?
>
> @zhuanshenbsj1 @danny0405
Yes, it will merge to 0.14.1 and will be released soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817793659
> @zhuanshenbsj1 I would recommened to separate the fs view locking from this PR. Let this PR only have the optimization for cleaner. Cleaner changes look good. You can still keep the `cleanFileGroupsInPartitions` method in fs view. But, raise another PR for fs view locking with details about the problem.
Nice advice~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817852647
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 45443c1149800a6cbe40b0fd8a4080ac7d4f91c4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004)
* 85d14854ac69b515c4557d18f8b09d1debb5fb9a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817946715
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 11c765e0318e6d410143da323c3962aba366fbc5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1800908197
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d10a137bff419d0d4befb5dac8380ac0bf0f12f8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705)
* 4dcb8f7bea46847202c2444e1a99901484239f4f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1820145804
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0d2b1cf6af86fed4821e9ec5b41e8477ad915b64 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008)
* 03e698c23b11e2b1891b755ddae1e6431796680d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1822582047
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 03e698c23b11e2b1891b755ddae1e6431796680d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038)
* 476071649c9cf7048435217bd30d246b18d9be65 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1815889971
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 47200120dd37ebaee77c583628bfddac1f564b3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817808175
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 47200120dd37ebaee77c583628bfddac1f564b3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967)
* b4aebce1c5d648e0c06d2330a930eaa3e24466e7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003)
* 45443c1149800a6cbe40b0fd8a4080ac7d4f91c4 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1818154301
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0d2b1cf6af86fed4821e9ec5b41e8477ad915b64 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1815788149
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb60d3f2fe5737fc43a700bcc6c37806fe48868a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722)
* 47200120dd37ebaee77c583628bfddac1f564b3b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817794756
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 47200120dd37ebaee77c583628bfddac1f564b3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967)
* b4aebce1c5d648e0c06d2330a930eaa3e24466e7 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1820419758
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 03e698c23b11e2b1891b755ddae1e6431796680d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817796713
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 47200120dd37ebaee77c583628bfddac1f564b3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967)
* b4aebce1c5d648e0c06d2330a930eaa3e24466e7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817918031
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 356bb49429ab96b078dbf5bd968418f01729619d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817863716
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 85d14854ac69b515c4557d18f8b09d1debb5fb9a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1818108850
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 11c765e0318e6d410143da323c3962aba366fbc5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007)
* 0d2b1cf6af86fed4821e9ec5b41e8477ad915b64 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the memory usage of timeline server for table service [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1826239398
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
}, {
"hash" : "8723081c68a37fa68b9665fad562d84543ed8533",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143",
"triggerID" : "8723081c68a37fa68b9665fad562d84543ed8533",
"triggerType" : "PUSH"
}, {
"hash" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144",
"triggerID" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3d733303d36f8f9531337a15d6dc265b8a0b5cff Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817936511
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 356bb49429ab96b078dbf5bd968418f01729619d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006)
* 11c765e0318e6d410143da323c3962aba366fbc5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817934622
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 356bb49429ab96b078dbf5bd968418f01729619d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006)
* 11c765e0318e6d410143da323c3962aba366fbc5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1819075803
> Thanks for the contribution @zhuanshenbsj1 , I'm not sure why the read/write lock changes are introduced, can we just implement another interface `getAllFileGroupsStatelessly(String partition)` on fs view similar with `getAllFileGroups(String partiton)`, in this method it does 2 steps to load a file group.
>
> * if the partition is already in the file group, just return it;
> * if it is not, load the file group on the fly, but we do not cache it in the file group mapping of the fs view.
>
> The clean can just use this method for cleaning.
Before getAllFileGroups is called, the partition metadata has been loaded (through getReplacedFileGroupsBefore | | getAllReplacedFileGroups) in fsView, which is reused here. If modified according to this approach, it will result in duplicate loading of metadata and very poor performance.


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1818103809
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 11c765e0318e6d410143da323c3962aba366fbc5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007)
* 0d2b1cf6af86fed4821e9ec5b41e8477ad915b64 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1397071042
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -378,19 +383,45 @@ protected Option<Pair<String, CompactionOperation>> getPendingLogCompactionOpera
@Override
protected boolean isPartitionAvailableInStore(String partitionPath) {
- return partitionToFileGroupsMap.containsKey(partitionPath);
+ fsLock.readLock();
+ try {
+ return partitionToFileGroupsMap.containsKey(partitionPath);
+ } finally {
+ fsLock.readUnlock();
+ }
}
@Override
protected void storePartitionView(String partitionPath, List<HoodieFileGroup> fileGroups) {
- LOG.debug("Adding file-groups for partition :" + partitionPath + ", #FileGroups=" + fileGroups.size());
- List<HoodieFileGroup> newList = new ArrayList<>(fileGroups);
- partitionToFileGroupsMap.put(partitionPath, newList);
+ fsLock.writeLock();
+ try {
+ LOG.debug("Adding file-groups for partition :" + partitionPath + ", #FileGroups=" + fileGroups.size());
+ List<HoodieFileGroup> newList = new ArrayList<>(fileGroups);
+ partitionToFileGroupsMap.put(partitionPath, newList);
+ } finally {
+ fsLock.writeUnlock();
+ }
+ }
+
+ @Override
+ protected void removePartitionView(String partitionPath) {
+ fsLock.writeLock();
Review Comment:
Same here - this is also being called alrady from the critical section in `cleanFileGroupsInPartitions`. Why take lock again?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -378,19 +383,45 @@ protected Option<Pair<String, CompactionOperation>> getPendingLogCompactionOpera
@Override
protected boolean isPartitionAvailableInStore(String partitionPath) {
- return partitionToFileGroupsMap.containsKey(partitionPath);
+ fsLock.readLock();
+ try {
+ return partitionToFileGroupsMap.containsKey(partitionPath);
+ } finally {
+ fsLock.readUnlock();
+ }
}
@Override
protected void storePartitionView(String partitionPath, List<HoodieFileGroup> fileGroups) {
- LOG.debug("Adding file-groups for partition :" + partitionPath + ", #FileGroups=" + fileGroups.size());
- List<HoodieFileGroup> newList = new ArrayList<>(fileGroups);
- partitionToFileGroupsMap.put(partitionPath, newList);
+ fsLock.writeLock();
Review Comment:
Isn't this already being called under write lock?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1826210797
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
}, {
"hash" : "8723081c68a37fa68b9665fad562d84543ed8533",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143",
"triggerID" : "8723081c68a37fa68b9665fad562d84543ed8533",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8723081c68a37fa68b9665fad562d84543ed8533 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1402970319
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -598,35 +603,49 @@ private FileSlice filterUncommittedLogs(FileSlice fileSlice) {
}
protected HoodieFileGroup addBootstrapBaseFileIfPresent(HoodieFileGroup fileGroup) {
+ return addBootstrapBaseFileIfPresent(fileGroup, this::getBootstrapBaseFile);
+ }
+
+ protected HoodieFileGroup addBootstrapBaseFileIfPresent(HoodieFileGroup fileGroup, Function<HoodieFileGroupId, Option<BootstrapBaseFileMapping>> bootstrapBaseFileMappingFunc) {
boolean hasBootstrapBaseFile = fileGroup.getAllFileSlices()
.anyMatch(fs -> fs.getBaseInstantTime().equals(METADATA_BOOTSTRAP_INSTANT_TS));
if (hasBootstrapBaseFile) {
HoodieFileGroup newFileGroup = new HoodieFileGroup(fileGroup);
newFileGroup.getAllFileSlices().filter(fs -> fs.getBaseInstantTime().equals(METADATA_BOOTSTRAP_INSTANT_TS))
.forEach(fs -> fs.setBaseFile(
- addBootstrapBaseFileIfPresent(fs.getFileGroupId(), fs.getBaseFile().get())));
+ addBootstrapBaseFileIfPresent(fs.getFileGroupId(), fs.getBaseFile().get(), bootstrapBaseFileMappingFunc)));
return newFileGroup;
}
return fileGroup;
}
protected FileSlice addBootstrapBaseFileIfPresent(FileSlice fileSlice) {
+ return addBootstrapBaseFileIfPresent(fileSlice, this::getBootstrapBaseFile);
+ }
+
+ protected FileSlice addBootstrapBaseFileIfPresent(FileSlice fileSlice, Function<HoodieFileGroupId, Option<BootstrapBaseFileMapping>> bootstrapBaseFileMappingFunc) {
if (fileSlice.getBaseInstantTime().equals(METADATA_BOOTSTRAP_INSTANT_TS)) {
FileSlice copy = new FileSlice(fileSlice);
copy.getBaseFile().ifPresent(dataFile -> {
Option<BootstrapBaseFileMapping> edf = getBootstrapBaseFile(copy.getFileGroupId());
Review Comment:
Oops, this line is useless now, can you fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1402973135
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RocksDbBasedFileSystemView.java:
##########
@@ -553,6 +553,10 @@ protected void removeReplacedFileIdsAtInstants(Set<String> instants) {
);
}
+ protected boolean hasReplacedFilesInPartition(String partitionPath) {
+ throw new UnsupportedOperationException("isReplacedFileExistWithinSpecifiedPartition() is not supported for RocksDbBasedFileSystemView!");
Review Comment:
We must support it correctly. Actually there is no need to override it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1826200990
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
}, {
"hash" : "8723081c68a37fa68b9665fad562d84543ed8533",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8723081c68a37fa68b9665fad562d84543ed8533",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a400e5e76089c0af8236fc938ea6b432d8c39da7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142)
* 8723081c68a37fa68b9665fad562d84543ed8533 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1798657341
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d10a137bff419d0d4befb5dac8380ac0bf0f12f8 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1801295720
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb60d3f2fe5737fc43a700bcc6c37806fe48868a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the memory usage of timeline server for table service [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1826229153
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
}, {
"hash" : "8723081c68a37fa68b9665fad562d84543ed8533",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143",
"triggerID" : "8723081c68a37fa68b9665fad562d84543ed8533",
"triggerType" : "PUSH"
}, {
"hash" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144",
"triggerID" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8723081c68a37fa68b9665fad562d84543ed8533 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143)
* 3d733303d36f8f9531337a15d6dc265b8a0b5cff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1826222857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
}, {
"hash" : "8723081c68a37fa68b9665fad562d84543ed8533",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143",
"triggerID" : "8723081c68a37fa68b9665fad562d84543ed8533",
"triggerType" : "PUSH"
}, {
"hash" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8723081c68a37fa68b9665fad562d84543ed8533 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143)
* 3d733303d36f8f9531337a15d6dc265b8a0b5cff UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1826202148
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
}, {
"hash" : "8723081c68a37fa68b9665fad562d84543ed8533",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143",
"triggerID" : "8723081c68a37fa68b9665fad562d84543ed8533",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a400e5e76089c0af8236fc938ea6b432d8c39da7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142)
* 8723081c68a37fa68b9665fad562d84543ed8533 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the memory usage of timeline server for table service [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1826280731
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1825891285
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bc5aa1807fab83dd00391b1dfb86d4e213692c7c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139)
* a400e5e76089c0af8236fc938ea6b432d8c39da7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1825526996
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bc5aa1807fab83dd00391b1dfb86d4e213692c7c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the memory usage of timeline server for table service [hudi]
Posted by "zyclove (via GitHub)" <gi...@apache.org>.
zyclove commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1833035662
Hi, Could you please merge the 0.14.1 (0.x) branch to support it? When will version 0.14.1 be released?
@zhuanshenbsj1 @danny0405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1398637356
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java:
##########
@@ -118,17 +120,25 @@ HoodieCleanerPlan requestClean(HoodieEngineContext context) {
context.setJobStatus(this.getClass().getSimpleName(), "Generating list of file slices to be cleaned: " + config.getTableName());
- Map<String, Pair<Boolean, List<CleanFileInfo>>> cleanOpsWithPartitionMeta = context
- .map(partitionsToClean, partitionPathToClean -> Pair.of(partitionPathToClean, planner.getDeletePaths(partitionPathToClean, earliestInstant)), cleanerParallelism)
- .stream()
- .collect(Collectors.toMap(Pair::getKey, Pair::getValue));
+ Map<String, List<HoodieCleanFileInfo>> cleanOps = new HashMap<>();
+ List<String> partitionsToDelete = new ArrayList<>();
+ for (int i = 0; i < partitionsToClean.size(); i += cleanerParallelism) {
+ List<String> subPartitionsToClean = partitionsToClean.subList(i, Math.min(i + cleanerParallelism, partitionsToClean.size()));
Review Comment:
This only works when `cleanerParallelism` is smaller than the size of `partitionsToClean`.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -164,27 +164,32 @@ public List<HoodieFileGroup> addFilesToView(FileStatus[] statuses) {
List<HoodieFileGroup> fileGroups = buildFileGroups(statuses, visibleCommitsAndCompactionTimeline, true);
long fgBuildTimeTakenMs = timer.endTimer();
timer.startTimer();
- // Group by partition for efficient updates for both InMemory and DiskBased structures.
- fileGroups.stream().collect(Collectors.groupingBy(HoodieFileGroup::getPartitionPath)).forEach((partition, value) -> {
- if (!isPartitionAvailableInStore(partition)) {
- if (bootstrapIndex.useIndex()) {
- try (BootstrapIndex.IndexReader reader = bootstrapIndex.createReader()) {
- LOG.info("Bootstrap Index available for partition " + partition);
- List<BootstrapFileMapping> sourceFileMappings =
- reader.getSourceFileMappingForPartition(partition);
- addBootstrapBaseFileMapping(sourceFileMappings.stream()
- .map(s -> new BootstrapBaseFileMapping(new HoodieFileGroupId(s.getPartitionPath(),
- s.getFileId()), s.getBootstrapFileStatus())));
+ writeLock.lock();
+ try {
Review Comment:
Why we still need these locks?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/PriorityBasedFileSystemView.java:
##########
@@ -222,6 +222,16 @@ public Stream<HoodieFileGroup> getAllFileGroups(String partitionPath) {
return execute(partitionPath, preferredView::getAllFileGroups, secondaryView::getAllFileGroups);
}
+ @Override
+ public boolean isPartitionAvailableInStoreForTest(String partitionPath) {
+ return execute(partitionPath, preferredView::isPartitionAvailableInStoreForTest, secondaryView::isPartitionAvailableInStoreForTest);
+ }
Review Comment:
We do not add APIs just for tesing purpose.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -305,8 +305,13 @@ void removeFileGroupsInPendingClustering(Stream<Pair<HoodieFileGroupId, HoodieIn
*/
@Override
Stream<HoodieFileGroup> fetchAllStoredFileGroups(String partition) {
- final List<HoodieFileGroup> fileGroups = new ArrayList<>(partitionToFileGroupsMap.get(partition));
- return fileGroups.stream();
+ readLock.lock();
+ try {
+ final List<HoodieFileGroup> fileGroups = new ArrayList<>(partitionToFileGroupsMap.get(partition));
+ return fileGroups.stream();
+ } finally {
+ readLock.unlock();
+ }
Review Comment:
Confused by all these lock changes.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java:
##########
@@ -491,10 +491,13 @@ public Pair<Boolean, List<CleanFileInfo>> getDeletePaths(String partitionPath, O
Pair<Boolean, List<CleanFileInfo>> deletePaths;
if (policy == HoodieCleaningPolicy.KEEP_LATEST_COMMITS) {
deletePaths = getFilesToCleanKeepingLatestCommits(partitionPath, earliestCommitToRetain);
+ fileSystemView.cleanFileGroupsInPartitions(Collections.singletonList(partitionPath));
} else if (policy == HoodieCleaningPolicy.KEEP_LATEST_FILE_VERSIONS) {
deletePaths = getFilesToCleanKeepingLatestVersions(partitionPath);
+ fileSystemView.cleanFileGroupsInPartitions(Collections.singletonList(partitionPath));
} else if (policy == HoodieCleaningPolicy.KEEP_LATEST_BY_HOURS) {
deletePaths = getFilesToCleanKeepingLatestHours(partitionPath, earliestCommitToRetain);
+ fileSystemView.cleanFileGroupsInPartitions(Collections.singletonList(partitionPath));
Review Comment:
Just move the logic to line 504.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1818193207
Thanks for the contribution @zhuanshenbsj1 , I'm not sure why the read/write lock changes are introduced, can we just implement another interface `getAllFileGroupsStatelessly(String partition)` on fs view similar with `getAllFileGroups(String partiton)`, in this method it does 2 steps to load a file group.
- if the partition is already in the file group, just return it;
- if it is not, load the file group on the fly, but we do not cache it in the file group mapping of the fs view.
The clean can just use this method for cleaning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817899220
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 85d14854ac69b515c4557d18f8b09d1debb5fb9a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005)
* 356bb49429ab96b078dbf5bd968418f01729619d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1801053081
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d10a137bff419d0d4befb5dac8380ac0bf0f12f8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705)
* 4dcb8f7bea46847202c2444e1a99901484239f4f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718)
* bb60d3f2fe5737fc43a700bcc6c37806fe48868a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1799266722
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d10a137bff419d0d4befb5dac8380ac0bf0f12f8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1798831519
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d10a137bff419d0d4befb5dac8380ac0bf0f12f8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1822690429
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 476071649c9cf7048435217bd30d246b18d9be65 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097)
* fc27baa8c2df9135bc6e4b0d14e50a127ecb434f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1822595088
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 03e698c23b11e2b1891b755ddae1e6431796680d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038)
* 476071649c9cf7048435217bd30d246b18d9be65 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1825471045
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6024de8ab05dd38e9cdb58afc10b70991542c392 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122)
* bc5aa1807fab83dd00391b1dfb86d4e213692c7c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1402973135
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/RocksDbBasedFileSystemView.java:
##########
@@ -553,6 +553,10 @@ protected void removeReplacedFileIdsAtInstants(Set<String> instants) {
);
}
+ protected boolean hasReplacedFilesInPartition(String partitionPath) {
+ throw new UnsupportedOperationException("isReplacedFileExistWithinSpecifiedPartition() is not supported for RocksDbBasedFileSystemView!");
Review Comment:
We must support it correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the memory usage of timeline server for table service [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1826287327
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
}, {
"hash" : "8723081c68a37fa68b9665fad562d84543ed8533",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21143",
"triggerID" : "8723081c68a37fa68b9665fad562d84543ed8533",
"triggerType" : "PUSH"
}, {
"hash" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144",
"triggerID" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"triggerType" : "PUSH"
}, {
"hash" : "3d733303d36f8f9531337a15d6dc265b8a0b5cff",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144",
"triggerID" : "1826280731",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 3d733303d36f8f9531337a15d6dc265b8a0b5cff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21144)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1823907225
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 35fed0de0587b411f9470e1c69db43501df5a725 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1817810093
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 47200120dd37ebaee77c583628bfddac1f564b3b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967)
* b4aebce1c5d648e0c06d2330a930eaa3e24466e7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003)
* 45443c1149800a6cbe40b0fd8a4080ac7d4f91c4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "zhuanshenbsj1 (via GitHub)" <gi...@apache.org>.
zhuanshenbsj1 commented on code in PR #10002:
URL: https://github.com/apache/hudi/pull/10002#discussion_r1399249455
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -305,8 +305,13 @@ void removeFileGroupsInPendingClustering(Stream<Pair<HoodieFileGroupId, HoodieIn
*/
@Override
Stream<HoodieFileGroup> fetchAllStoredFileGroups(String partition) {
- final List<HoodieFileGroup> fileGroups = new ArrayList<>(partitionToFileGroupsMap.get(partition));
- return fileGroups.stream();
+ readLock.lock();
+ try {
+ final List<HoodieFileGroup> fileGroups = new ArrayList<>(partitionToFileGroupsMap.get(partition));
+ return fileGroups.stream();
+ } finally {
+ readLock.unlock();
+ }
Review Comment:
> Confused by all these lock changes.
To Ensure thread safety for partitionToFileGroupsMap. However, some read locks can indeed be removed because the partitionToFileGroupsMap is ConcurrentHashMap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7041] Optimize the mem usage of partitionToFileGroupsMap during the cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10002:
URL: https://github.com/apache/hudi/pull/10002#issuecomment-1825940137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20705",
"triggerID" : "d10a137bff419d0d4befb5dac8380ac0bf0f12f8",
"triggerType" : "PUSH"
}, {
"hash" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20718",
"triggerID" : "4dcb8f7bea46847202c2444e1a99901484239f4f",
"triggerType" : "PUSH"
}, {
"hash" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20722",
"triggerID" : "bb60d3f2fe5737fc43a700bcc6c37806fe48868a",
"triggerType" : "PUSH"
}, {
"hash" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=20967",
"triggerID" : "47200120dd37ebaee77c583628bfddac1f564b3b",
"triggerType" : "PUSH"
}, {
"hash" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21003",
"triggerID" : "b4aebce1c5d648e0c06d2330a930eaa3e24466e7",
"triggerType" : "PUSH"
}, {
"hash" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21004",
"triggerID" : "45443c1149800a6cbe40b0fd8a4080ac7d4f91c4",
"triggerType" : "PUSH"
}, {
"hash" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21005",
"triggerID" : "85d14854ac69b515c4557d18f8b09d1debb5fb9a",
"triggerType" : "PUSH"
}, {
"hash" : "356bb49429ab96b078dbf5bd968418f01729619d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21006",
"triggerID" : "356bb49429ab96b078dbf5bd968418f01729619d",
"triggerType" : "PUSH"
}, {
"hash" : "11c765e0318e6d410143da323c3962aba366fbc5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21007",
"triggerID" : "11c765e0318e6d410143da323c3962aba366fbc5",
"triggerType" : "PUSH"
}, {
"hash" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21008",
"triggerID" : "0d2b1cf6af86fed4821e9ec5b41e8477ad915b64",
"triggerType" : "PUSH"
}, {
"hash" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21038",
"triggerID" : "03e698c23b11e2b1891b755ddae1e6431796680d",
"triggerType" : "PUSH"
}, {
"hash" : "476071649c9cf7048435217bd30d246b18d9be65",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21097",
"triggerID" : "476071649c9cf7048435217bd30d246b18d9be65",
"triggerType" : "PUSH"
}, {
"hash" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21100",
"triggerID" : "fc27baa8c2df9135bc6e4b0d14e50a127ecb434f",
"triggerType" : "PUSH"
}, {
"hash" : "35fed0de0587b411f9470e1c69db43501df5a725",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21121",
"triggerID" : "35fed0de0587b411f9470e1c69db43501df5a725",
"triggerType" : "PUSH"
}, {
"hash" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21122",
"triggerID" : "6024de8ab05dd38e9cdb58afc10b70991542c392",
"triggerType" : "PUSH"
}, {
"hash" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21139",
"triggerID" : "bc5aa1807fab83dd00391b1dfb86d4e213692c7c",
"triggerType" : "PUSH"
}, {
"hash" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142",
"triggerID" : "a400e5e76089c0af8236fc938ea6b432d8c39da7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a400e5e76089c0af8236fc938ea6b432d8c39da7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21142)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org