You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/12/15 13:00:24 UTC
[GitHub] [hudi] onlywangyh opened a new pull request, #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
onlywangyh opened a new pull request, #7323:
URL: https://github.com/apache/hudi/pull/7323
## What is the purpose of the pull request
fix https://github.com/apache/hudi/issues/7298
Fix query partition path add invalid input path. In function getPartitionPathsWithPrefixes will fetch list of all partitions path that whose relative partition paths match the given prefixes. It caused a unexpected path add and throw exception in hive. Like this:
Table has partition 2 partitions:
year=2022/month=08/day=30
year=2022back/month=08/day=31
When query "year=2022", we only want get the path 2022 not the 2022back.
Expect paths _[year=2022/month=08/day=30]_.
actual paths [year=2022/month=08/day=30, year=2022back/month=08/day=30]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar commented on a diff in pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by "bvaradar (via GitHub)" <gi...@apache.org>.
bvaradar commented on code in PR #7323:
URL: https://github.com/apache/hudi/pull/7323#discussion_r1121086268
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -146,7 +146,7 @@ protected Option<HoodieRecord<HoodieMetadataPayload>> getRecordByKey(String key,
@Override
public List<String> getPartitionPathsWithPrefixes(List<String> prefixes) throws IOException {
return getAllPartitionPaths().stream()
- .filter(p -> prefixes.stream().anyMatch(p::startsWith))
+ .filter(p -> prefixes.stream().anyMatch(queryPaths -> p.startsWith(queryPaths + "/") || queryPaths.equals(p)))
Review Comment:
@onlywangyh : Is the current behavior in Hive differs from Parquet Hive and Presto table ? If so, can we change in the caller side (for e:g HiveHoodieTableFileIndex) instead of this place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on a diff in pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on code in PR #7323:
URL: https://github.com/apache/hudi/pull/7323#discussion_r1035646763
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -146,7 +146,7 @@ protected Option<HoodieRecord<HoodieMetadataPayload>> getRecordByKey(String key,
@Override
public List<String> getPartitionPathsWithPrefixes(List<String> prefixes) throws IOException {
return getAllPartitionPaths().stream()
- .filter(p -> prefixes.stream().anyMatch(p::startsWith))
+ .filter(p -> prefixes.stream().anyMatch(queryPaths -> p.startsWith(queryPaths + "/") || queryPaths.equals(p)))
Review Comment:
I have changed this condition try to covering all production scenarios. This method we want get all matchedPartitionPaths
When this partition is "/year=2022/month=08/day=30", the query relative partition path prefixes may be these value :
1、a empty path like "". In this scenarios we should filter all paths return as matched path
2、a prefixes partition path like "/year=2022";
3、a absolute path like "/year=2022/month=08/day=30"
If prefixes is empty ,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on a diff in pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on code in PR #7323:
URL: https://github.com/apache/hudi/pull/7323#discussion_r1035603153
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -146,7 +146,7 @@ protected Option<HoodieRecord<HoodieMetadataPayload>> getRecordByKey(String key,
@Override
public List<String> getPartitionPathsWithPrefixes(List<String> prefixes) throws IOException {
return getAllPartitionPaths().stream()
- .filter(p -> prefixes.stream().anyMatch(p::startsWith))
+ .filter(p -> prefixes.stream().anyMatch(queryPaths -> p.startsWith(queryPaths + "/") || queryPaths.equals(p)))
Review Comment:
I had changed the condition try to support more scenarios.
This method will return a match partition path in hudi. When this table has a partition like [/inc_day=20221120/opcode=501, /inc_day=20221120/opcode=50, /inc_day=20221120-back/opcode=5000], and the query path prefixes may be a few of the following possibilities:
1) a empty path like "";
2) a part of partition like "/inc_day=20221120";
3) a absoulty path like "/inc_day=20221120/opcode=50";
If we just use startWith filter match paths, we will reurn a list with unnecessary partiton paths. Like this:
prefixes="/inc_day=20221120/opcode=50"
matchedPartitionPaths=[/inc_day=20221120/opcode=50, /inc_day=20221120/opcode=501]
or
prefixes="/inc_day=20221120"
matchedPartitionPaths=[/inc_day=20221120/opcode=50, /inc_day=20221120/opcode=501, /inc_day=20221120-back/opcode=5000]
While in most of scenarios the matchedPartitionPaths contains unnecessary partiton paths is right. But in hive will caused a java.lang.RuntimeException: Invalid input path. So we want make the matchedPartitionPaths exclude these unnecessary partiton path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1353073653
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450",
"triggerID" : "1336710901",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451",
"triggerID" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"triggerType" : "PUSH"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484",
"triggerID" : "1339448543",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13719",
"triggerID" : "1350722794",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13727",
"triggerID" : "1351636981",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1352996791",
"triggerType" : "MANUAL"
}, {
"hash" : "0000",
"status" : "CANCELED",
"url" : "TBD",
"triggerID" : "1352996791",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13765",
"triggerID" : "1353028577",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0000 Unknown: [CANCELED](TBD)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1339448543
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1331584812
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ff7723a4143159db1873f2a453338f2b5dd88bbd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311)
* 0695162ef193c9b8f2df9deb66ff7eaab8325608 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: Update HoodieBackedTableMetadata.java
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1330220608
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ff7723a4143159db1873f2a453338f2b5dd88bbd UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1352996791
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1351231027
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450",
"triggerID" : "1336710901",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451",
"triggerID" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"triggerType" : "PUSH"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484",
"triggerID" : "1339448543",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13719",
"triggerID" : "1350722794",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a788a5320de3adb3de45751ba628dadfb12ab154 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13719)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1350724977
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450",
"triggerID" : "1336710901",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451",
"triggerID" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"triggerType" : "PUSH"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484",
"triggerID" : "1339448543",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13719",
"triggerID" : "1350722794",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a788a5320de3adb3de45751ba628dadfb12ab154 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13719)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1336740381
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450",
"triggerID" : "1336710901",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0695162ef193c9b8f2df9deb66ff7eaab8325608 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on a diff in pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on code in PR #7323:
URL: https://github.com/apache/hudi/pull/7323#discussion_r1035597947
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -146,7 +146,7 @@ protected Option<HoodieRecord<HoodieMetadataPayload>> getRecordByKey(String key,
@Override
public List<String> getPartitionPathsWithPrefixes(List<String> prefixes) throws IOException {
return getAllPartitionPaths().stream()
- .filter(p -> prefixes.stream().anyMatch(p::startsWith))
+ .filter(p -> prefixes.stream().anyMatch(queryPaths -> p.startsWith(queryPaths + "/") || queryPaths.equals(p)))
Review Comment:
This method will return a match partition path in hudi. When this table has a partition like [/inc_day=20221120/opcode=501, /inc_day=20221120/opcode=50, /inc_day=20221120-back/opcode=5000], and the query path prefixes may be a few of the following possibilities:
1) a empty path like "";
2) a part of partition like "/inc_day=20221120";
3) a absoulty path like "/inc_day=20221120/opcode=50";
If we just use startWith filter match paths, we will reurn a list with unnecessary partiton paths. Like this:
prefixes="/inc_day=20221120/opcode=50"
matchedPartitionPaths=[/inc_day=20221120/opcode=50, /inc_day=20221120/opcode=501]
or
prefixes="/inc_day=20221120"
matchedPartitionPaths=[/inc_day=20221120/opcode=50, /inc_day=20221120/opcode=501, /inc_day=20221120-back/opcode=5000]
While in most of scenarios the matchedPartitionPaths contains unnecessary partiton paths is right. But in hive will caused a java.lang.RuntimeException: Invalid input path. So we want make the matchedPartitionPaths exclude these unnecessary partiton path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1357594850
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1336744553
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450",
"triggerID" : "1336710901",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0695162ef193c9b8f2df9deb66ff7eaab8325608 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450)
* a788a5320de3adb3de45751ba628dadfb12ab154 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1331670871
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0695162ef193c9b8f2df9deb66ff7eaab8325608 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1330239154
@codope Do you have time to help review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1339535670
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450",
"triggerID" : "1336710901",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451",
"triggerID" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"triggerType" : "PUSH"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484",
"triggerID" : "1339448543",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a788a5320de3adb3de45751ba628dadfb12ab154 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-5276][WIP] Exclude unnecessary partiton paths return [hudi]
Posted by "bvaradar (via GitHub)" <gi...@apache.org>.
bvaradar commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1747363762
Closing this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #7323:
URL: https://github.com/apache/hudi/pull/7323#discussion_r1073075839
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -152,7 +153,8 @@ protected Option<HoodieRecord<HoodieMetadataPayload>> getRecordByKey(String key,
@Override
public List<String> getPartitionPathsWithPrefixes(List<String> prefixes) throws IOException {
return getAllPartitionPaths().stream()
- .filter(p -> prefixes.stream().anyMatch(p::startsWith))
+ .filter(p -> prefixes.stream().anyMatch(queryPaths ->
Review Comment:
@onlywangyh this is not the right place to fix this -- this should be fixed on the caller side: if caller specifies prefix w/o a slash this method will match it appropriately. Instead, caller should have specified a path ending with a slash to signal that it doesn't want any other folders to be matched
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on a diff in pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on code in PR #7323:
URL: https://github.com/apache/hudi/pull/7323#discussion_r1035597947
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -146,7 +146,7 @@ protected Option<HoodieRecord<HoodieMetadataPayload>> getRecordByKey(String key,
@Override
public List<String> getPartitionPathsWithPrefixes(List<String> prefixes) throws IOException {
return getAllPartitionPaths().stream()
- .filter(p -> prefixes.stream().anyMatch(p::startsWith))
+ .filter(p -> prefixes.stream().anyMatch(queryPaths -> p.startsWith(queryPaths + "/") || queryPaths.equals(p)))
Review Comment:
This method will return a match partition path in hudi. When this table has a partition like [/inc_day=20221120/opcode=501, /inc_day=20221120/opcode=50, /inc_day=20221120-back/opcode=5000], and the query path prefixes may be a few of the following possibilities:
1) a empty path like "";
2) a part of partition like "/inc_day=20221120";
3) a absoulty path like "/inc_day=20221120/opcode=50";
If we just use startWith filter match paths, we will reurn a list with unnecessary partiton paths. Like this:
prefixes="/inc_day=20221120/opcode=50"
matchedPartitionPaths=[/inc_day=20221120/opcode=50, /inc_day=20221120/opcode=501]
or
prefixes="/inc_day=20221120"
matchedPartitionPaths=[/inc_day=20221120/opcode=50, /inc_day=20221120/opcode=501, /inc_day=20221120-back/opcode=5000]
While in most of scenarios the matchedPartitionPaths contains unnecessary partiton paths is right. But in hive will caused a java.lang.RuntimeException: Invalid input path. So we want make the matchedPartitionPaths exclude these unnecessary partiton path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1351636981
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh closed pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh closed pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
URL: https://github.com/apache/hudi/pull/7323
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1336747079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450",
"triggerID" : "1336710901",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451",
"triggerID" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a788a5320de3adb3de45751ba628dadfb12ab154 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1359152712
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
codope commented on code in PR #7323:
URL: https://github.com/apache/hudi/pull/7323#discussion_r1034655436
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -146,7 +146,7 @@ protected Option<HoodieRecord<HoodieMetadataPayload>> getRecordByKey(String key,
@Override
public List<String> getPartitionPathsWithPrefixes(List<String> prefixes) throws IOException {
return getAllPartitionPaths().stream()
- .filter(p -> prefixes.stream().anyMatch(p::startsWith))
+ .filter(p -> prefixes.stream().anyMatch(queryPaths -> p.startsWith(queryPaths + "/") || queryPaths.equals(p)))
Review Comment:
This alters the condition by assuming things such as "/" will be separator and it will be present. Such conditions may not be true for many production scenarios. I think we should not change the condition here. This method is just meant to check for prefixes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on a diff in pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on code in PR #7323:
URL: https://github.com/apache/hudi/pull/7323#discussion_r1035646763
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -146,7 +146,7 @@ protected Option<HoodieRecord<HoodieMetadataPayload>> getRecordByKey(String key,
@Override
public List<String> getPartitionPathsWithPrefixes(List<String> prefixes) throws IOException {
return getAllPartitionPaths().stream()
- .filter(p -> prefixes.stream().anyMatch(p::startsWith))
+ .filter(p -> prefixes.stream().anyMatch(queryPaths -> p.startsWith(queryPaths + "/") || queryPaths.equals(p)))
Review Comment:
I have changed this condition try to covering all production scenarios. This method we want get all matchedPartitionPaths
When this partition is "/year=2022/month=08/day=30", the query relative partition path prefixes may be these value :
1、a empty path like "". In this scenarios we should filter all paths return as matched path
2、a prefixes partition path like "/year=2022";
3、a absolute path like "/year=2022/month=08/day=30"
If prefixes is empty ,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1330647085
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ff7723a4143159db1873f2a453338f2b5dd88bbd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-5276][WIP] Exclude unnecessary partiton paths return [hudi]
Posted by "bvaradar (via GitHub)" <gi...@apache.org>.
bvaradar closed pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
URL: https://github.com/apache/hudi/pull/7323
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1331588828
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ff7723a4143159db1873f2a453338f2b5dd88bbd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311)
* 0695162ef193c9b8f2df9deb66ff7eaab8325608 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: Update HoodieBackedTableMetadata.java
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1330225815
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ff7723a4143159db1873f2a453338f2b5dd88bbd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on pull request #7323: [HUDI-5276] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1336710901
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1350722794
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1351729148
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450",
"triggerID" : "1336710901",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451",
"triggerID" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"triggerType" : "PUSH"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484",
"triggerID" : "1339448543",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13719",
"triggerID" : "1350722794",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13727",
"triggerID" : "1351636981",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a788a5320de3adb3de45751ba628dadfb12ab154 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13719) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13727)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1351856005
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450",
"triggerID" : "1336710901",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451",
"triggerID" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"triggerType" : "PUSH"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484",
"triggerID" : "1339448543",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13719",
"triggerID" : "1350722794",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13727",
"triggerID" : "1351636981",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a788a5320de3adb3de45751ba628dadfb12ab154 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13719) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13727)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] onlywangyh commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
onlywangyh commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1353028577
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7323: [HUDI-5276][WIP] Exclude unnecessary partiton paths return
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7323:
URL: https://github.com/apache/hudi/pull/7323#issuecomment-1339545915
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13311",
"triggerID" : "ff7723a4143159db1873f2a453338f2b5dd88bbd",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13336",
"triggerID" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"triggerType" : "PUSH"
}, {
"hash" : "0695162ef193c9b8f2df9deb66ff7eaab8325608",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13450",
"triggerID" : "1336710901",
"triggerType" : "MANUAL"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451",
"triggerID" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"triggerType" : "PUSH"
}, {
"hash" : "a788a5320de3adb3de45751ba628dadfb12ab154",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484",
"triggerID" : "1339448543",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a788a5320de3adb3de45751ba628dadfb12ab154 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13451) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13484)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org