You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "ChestnutQiang (via GitHub)" <gi...@apache.org> on 2023/04/13 06:04:21 UTC
[GitHub] [hudi] ChestnutQiang opened a new pull request, #8448: [HUDI-6069] Fixing If the primary key is a non-string type, and metas…
ChestnutQiang opened a new pull request, #8448:
URL: https://github.com/apache/hudi/pull/8448
…tore table is used for filtering, taking minValue to forcibly convert to String will error (#8444)
Change Logs
Change the way metadata table obtains minValue string strong-aroundmethod
Impact
metadata table skip data
Risk level (write none, low medium or high below)
low
Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x ] Change Logs and Impact were stated clearly
- [x] Adequate tests were added if applicable
- [x] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8448: [HUDI-6069] Fixing If the primary key is a non-string type, and metas…
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8448:
URL: https://github.com/apache/hudi/pull/8448#issuecomment-1506432574
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "41ec05b72d53dfa11d216a2546d7fa5d70b69fd6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16303",
"triggerID" : "41ec05b72d53dfa11d216a2546d7fa5d70b69fd6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 41ec05b72d53dfa11d216a2546d7fa5d70b69fd6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16303)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8448: [HUDI-6069] Fixing If the primary key is a non-string type, and metas…
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8448:
URL: https://github.com/apache/hudi/pull/8448#issuecomment-1506490054
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "41ec05b72d53dfa11d216a2546d7fa5d70b69fd6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16303",
"triggerID" : "41ec05b72d53dfa11d216a2546d7fa5d70b69fd6",
"triggerType" : "PUSH"
}, {
"hash" : "5b5de9d429205781af5200a103b546c38c56973e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5b5de9d429205781af5200a103b546c38c56973e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 41ec05b72d53dfa11d216a2546d7fa5d70b69fd6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16303)
* 5b5de9d429205781af5200a103b546c38c56973e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] ChestnutQiang commented on a diff in pull request #8448: [HUDI-6069] Fixing If the primary key is a non-string type, and metas…
Posted by "ChestnutQiang (via GitHub)" <gi...@apache.org>.
ChestnutQiang commented on code in PR #8448:
URL: https://github.com/apache/hudi/pull/8448#discussion_r1169894611
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bloom/HoodieBloomIndex.java:
##########
@@ -234,8 +234,8 @@ protected List<Pair<String, BloomIndexFileInfo>> loadColumnRangesFromMetaIndex(
new BloomIndexFileInfo(
FSUtils.getFileId(entry.getKey().getRight()),
// NOTE: Here we assume that the type of the primary key field is string
- (String) unwrapStatisticValueWrapper(entry.getValue().getMinValue()),
- (String) unwrapStatisticValueWrapper(entry.getValue().getMaxValue())
+ unwrapStatisticValueWrapper(entry.getValue().getMinValue()).toString(),
+ unwrapStatisticValueWrapper(entry.getValue().getMaxValue()).toString()
Review Comment:
@codope Can you tell me what I need to change? thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] ChestnutQiang commented on a diff in pull request #8448: [HUDI-6069] Fixing If the primary key is a non-string type, and metas…
Posted by "ChestnutQiang (via GitHub)" <gi...@apache.org>.
ChestnutQiang commented on code in PR #8448:
URL: https://github.com/apache/hudi/pull/8448#discussion_r1165086554
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bloom/HoodieBloomIndex.java:
##########
@@ -234,8 +234,8 @@ protected List<Pair<String, BloomIndexFileInfo>> loadColumnRangesFromMetaIndex(
new BloomIndexFileInfo(
FSUtils.getFileId(entry.getKey().getRight()),
// NOTE: Here we assume that the type of the primary key field is string
- (String) unwrapStatisticValueWrapper(entry.getValue().getMinValue()),
- (String) unwrapStatisticValueWrapper(entry.getValue().getMaxValue())
+ unwrapStatisticValueWrapper(entry.getValue().getMinValue()).toString(),
+ unwrapStatisticValueWrapper(entry.getValue().getMaxValue()).toString()
Review Comment:
Thank you very much, I think if the primary key is null, an exception should be thrown inside this method. After I modified it like this, all other reads and writes were normal. Even if the primary key is bigint
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-6069] Fixing If the primary key is a non-string type, and metas… [hudi]
Posted by "bvaradar (via GitHub)" <gi...@apache.org>.
bvaradar commented on PR #8448:
URL: https://github.com/apache/hudi/pull/8448#issuecomment-1747405785
@chestnutqiang : Can you resolve the conflict. I will take a look at this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8448: [HUDI-6069] Fixing If the primary key is a non-string type, and metas…
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8448:
URL: https://github.com/apache/hudi/pull/8448#issuecomment-1506720695
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "41ec05b72d53dfa11d216a2546d7fa5d70b69fd6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16303",
"triggerID" : "41ec05b72d53dfa11d216a2546d7fa5d70b69fd6",
"triggerType" : "PUSH"
}, {
"hash" : "5b5de9d429205781af5200a103b546c38c56973e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16305",
"triggerID" : "5b5de9d429205781af5200a103b546c38c56973e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5b5de9d429205781af5200a103b546c38c56973e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16305)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8448: [HUDI-6069] Fixing If the primary key is a non-string type, and metas…
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8448:
URL: https://github.com/apache/hudi/pull/8448#issuecomment-1506499663
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "41ec05b72d53dfa11d216a2546d7fa5d70b69fd6",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16303",
"triggerID" : "41ec05b72d53dfa11d216a2546d7fa5d70b69fd6",
"triggerType" : "PUSH"
}, {
"hash" : "5b5de9d429205781af5200a103b546c38c56973e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16305",
"triggerID" : "5b5de9d429205781af5200a103b546c38c56973e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 41ec05b72d53dfa11d216a2546d7fa5d70b69fd6 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16303)
* 5b5de9d429205781af5200a103b546c38c56973e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16305)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #8448: [HUDI-6069] Fixing If the primary key is a non-string type, and metas…
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #8448:
URL: https://github.com/apache/hudi/pull/8448#discussion_r1165043872
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bloom/HoodieBloomIndex.java:
##########
@@ -234,8 +234,8 @@ protected List<Pair<String, BloomIndexFileInfo>> loadColumnRangesFromMetaIndex(
new BloomIndexFileInfo(
FSUtils.getFileId(entry.getKey().getRight()),
// NOTE: Here we assume that the type of the primary key field is string
- (String) unwrapStatisticValueWrapper(entry.getValue().getMinValue()),
- (String) unwrapStatisticValueWrapper(entry.getValue().getMaxValue())
+ unwrapStatisticValueWrapper(entry.getValue().getMinValue()).toString(),
+ unwrapStatisticValueWrapper(entry.getValue().getMaxValue()).toString()
Review Comment:
Could potentially result in `null` dereference.
Also, we have made a note of the assumption that primary key is supposed to be string.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8448: [HUDI-6069] Fixing If the primary key is a non-string type, and metas…
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8448:
URL: https://github.com/apache/hudi/pull/8448#issuecomment-1506424948
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "41ec05b72d53dfa11d216a2546d7fa5d70b69fd6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "41ec05b72d53dfa11d216a2546d7fa5d70b69fd6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 41ec05b72d53dfa11d216a2546d7fa5d70b69fd6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org