You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "xuzifu666 (via GitHub)" <gi...@apache.org> on 2024/05/06 12:47:46 UTC
[PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
xuzifu666 opened a new pull request, #11161:
URL: https://github.com/apache/hudi/pull/11161
### Change Logs
Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups
### Impact
low
### Risk level (write none, low medium or high below)
low
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none"._
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11161:
URL: https://github.com/apache/hudi/pull/11161#issuecomment-2097741186
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23703",
"triggerID" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"triggerType" : "PUSH"
}, {
"hash" : "920a8f421b2d6650c8b2451af2038346a2906343",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23725",
"triggerID" : "920a8f421b2d6650c8b2451af2038346a2906343",
"triggerType" : "PUSH"
}, {
"hash" : "9b7057d990f1d6de45a27c3f6b47b467c9007c5b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23732",
"triggerID" : "9b7057d990f1d6de45a27c3f6b47b467c9007c5b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9b7057d990f1d6de45a27c3f6b47b467c9007c5b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23732)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11161:
URL: https://github.com/apache/hudi/pull/11161#issuecomment-2096029242
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23703",
"triggerID" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50008938bd209ca6dedc42f7fa616d7df952df4b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23703)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11161:
URL: https://github.com/apache/hudi/pull/11161#issuecomment-2096278083
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23703",
"triggerID" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50008938bd209ca6dedc42f7fa616d7df952df4b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23703)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11161:
URL: https://github.com/apache/hudi/pull/11161#issuecomment-2097430015
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23703",
"triggerID" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"triggerType" : "PUSH"
}, {
"hash" : "920a8f421b2d6650c8b2451af2038346a2906343",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "920a8f421b2d6650c8b2451af2038346a2906343",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50008938bd209ca6dedc42f7fa616d7df952df4b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23703)
* 920a8f421b2d6650c8b2451af2038346a2906343 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11161:
URL: https://github.com/apache/hudi/pull/11161#issuecomment-2097544407
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23703",
"triggerID" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"triggerType" : "PUSH"
}, {
"hash" : "920a8f421b2d6650c8b2451af2038346a2906343",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23725",
"triggerID" : "920a8f421b2d6650c8b2451af2038346a2906343",
"triggerType" : "PUSH"
}, {
"hash" : "9b7057d990f1d6de45a27c3f6b47b467c9007c5b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9b7057d990f1d6de45a27c3f6b47b467c9007c5b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 920a8f421b2d6650c8b2451af2038346a2906343 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23725)
* 9b7057d990f1d6de45a27c3f6b47b467c9007c5b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #11161:
URL: https://github.com/apache/hudi/pull/11161
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11161:
URL: https://github.com/apache/hudi/pull/11161#issuecomment-2097467988
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23703",
"triggerID" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"triggerType" : "PUSH"
}, {
"hash" : "920a8f421b2d6650c8b2451af2038346a2906343",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23725",
"triggerID" : "920a8f421b2d6650c8b2451af2038346a2906343",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 920a8f421b2d6650c8b2451af2038346a2906343 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23725)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11161:
URL: https://github.com/apache/hudi/pull/11161#issuecomment-2096012513
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50008938bd209ca6dedc42f7fa616d7df952df4b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #11161:
URL: https://github.com/apache/hudi/pull/11161#discussion_r1591700013
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -307,6 +307,10 @@ void removeFileGroupsInPendingClustering(Stream<Pair<HoodieFileGroupId, HoodieIn
*/
@Override
Stream<HoodieFileGroup> fetchAllStoredFileGroups(String partition) {
+ if (!isPartitionAvailableInStore(partition)) {
Review Comment:
We can `partitionToFileGroupsMap.get(partition)` first then decide if it is empty or null to eliminate additional lookup for the cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "xuzifu666 (via GitHub)" <gi...@apache.org>.
xuzifu666 commented on code in PR #11161:
URL: https://github.com/apache/hudi/pull/11161#discussion_r1591794126
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -307,6 +307,10 @@ void removeFileGroupsInPendingClustering(Stream<Pair<HoodieFileGroupId, HoodieIn
*/
@Override
Stream<HoodieFileGroup> fetchAllStoredFileGroups(String partition) {
+ if (!isPartitionAvailableInStore(partition)) {
Review Comment:
Had changed，do you mean modify like current way？ @danny0405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #11161:
URL: https://github.com/apache/hudi/pull/11161#discussion_r1591700013
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemView.java:
##########
@@ -307,6 +307,10 @@ void removeFileGroupsInPendingClustering(Stream<Pair<HoodieFileGroupId, HoodieIn
*/
@Override
Stream<HoodieFileGroup> fetchAllStoredFileGroups(String partition) {
+ if (!isPartitionAvailableInStore(partition)) {
Review Comment:
We can `partitionToFileGroupsMap.get(partition)` first then decide if it is now to eliminate one lookup for the cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7720] Fix HoodieTableFileSystemView NPE in fetchAllStoredFileGroups [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11161:
URL: https://github.com/apache/hudi/pull/11161#issuecomment-2097554582
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23703",
"triggerID" : "50008938bd209ca6dedc42f7fa616d7df952df4b",
"triggerType" : "PUSH"
}, {
"hash" : "920a8f421b2d6650c8b2451af2038346a2906343",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23725",
"triggerID" : "920a8f421b2d6650c8b2451af2038346a2906343",
"triggerType" : "PUSH"
}, {
"hash" : "9b7057d990f1d6de45a27c3f6b47b467c9007c5b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23732",
"triggerID" : "9b7057d990f1d6de45a27c3f6b47b467c9007c5b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 920a8f421b2d6650c8b2451af2038346a2906343 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23725)
* 9b7057d990f1d6de45a27c3f6b47b467c9007c5b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23732)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org