You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/08/30 02:04:40 UTC
[GitHub] [hudi] wuwenchi opened a new pull request, #6539: [HUDI-4739] Wrong value returned when key's length equals 1
wuwenchi opened a new pull request, #6539:
URL: https://github.com/apache/hudi/pull/6539
### Change Logs
_Describe context and summary for this change. Highlight if any code was copied._
### Impact
_Describe any public API or user-facing feature change or any performance impact._
**Risk level: none | low | medium | high**
_Choose one. If medium or high, explain what verification was done to mitigate the risks._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wuwenchi commented on a diff in pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
wuwenchi commented on code in PR #6539:
URL: https://github.com/apache/hudi/pull/6539#discussion_r958074279
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:
##########
@@ -73,21 +73,16 @@ public static String getPartitionPathFromGenericRecord(GenericRecord genericReco
*/
public static String[] extractRecordKeys(String recordKey) {
String[] fieldKV = recordKey.split(",");
- if (fieldKV.length == 1) {
- return fieldKV;
- } else {
- // a complex key
- return Arrays.stream(fieldKV).map(kv -> {
- final String[] kvArray = kv.split(":");
- if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
- return null;
- } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
- return "";
- } else {
- return kvArray[1];
- }
- }).toArray(String[]::new);
- }
+ return Arrays.stream(fieldKV).map(kv -> {
+ final String[] kvArray = kv.split(":");
+ if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
+ return null;
+ } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
+ return "";
+ } else {
+ return kvArray[1];
+ }
+ }).toArray(String[]::new);
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wuwenchi commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
wuwenchi commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1236233752
@nsivabalan Thanks, updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1236241444
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11040",
"triggerID" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"triggerType" : "PUSH"
}, {
"hash" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1236240737",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0e648e21e15d21e537914ef8dbec051ea761f76e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11040)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1231637928
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11032",
"triggerID" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"triggerType" : "PUSH"
}, {
"hash" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11040",
"triggerID" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0e648e21e15d21e537914ef8dbec051ea761f76e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11040)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #6539:
URL: https://github.com/apache/hudi/pull/6539#discussion_r958079651
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:
##########
@@ -73,21 +73,16 @@ public static String getPartitionPathFromGenericRecord(GenericRecord genericReco
*/
public static String[] extractRecordKeys(String recordKey) {
String[] fieldKV = recordKey.split(",");
- if (fieldKV.length == 1) {
- return fieldKV;
- } else {
- // a complex key
- return Arrays.stream(fieldKV).map(kv -> {
- final String[] kvArray = kv.split(":");
- if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
- return null;
- } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
- return "";
- } else {
- return kvArray[1];
- }
- }).toArray(String[]::new);
- }
+ return Arrays.stream(fieldKV).map(kv -> {
+ final String[] kvArray = kv.split(":");
+ if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
+ return null;
+ } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
+ return "";
+ } else {
+ return kvArray[1];
+ }
+ }).toArray(String[]::new);
Review Comment:
Why we generate a key like `name:val`, can we avoid that ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1231236145
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11032",
"triggerID" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 822e071498bbe67aaaced421c27cdffb8e9e6584 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11032)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 merged pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
danny0405 merged PR #6539:
URL: https://github.com/apache/hudi/pull/6539
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1236240975
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0e648e21e15d21e537914ef8dbec051ea761f76e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] XuQianJin-Stars commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1236240737
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1231069104
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11032",
"triggerID" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 822e071498bbe67aaaced421c27cdffb8e9e6584 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11032)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1231066421
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 822e071498bbe67aaaced421c27cdffb8e9e6584 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1236242522
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11040",
"triggerID" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"triggerType" : "PUSH"
}, {
"hash" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11040",
"triggerID" : "1236240737",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0e648e21e15d21e537914ef8dbec051ea761f76e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11040)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #6539:
URL: https://github.com/apache/hudi/pull/6539#discussion_r962460133
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:
##########
@@ -73,21 +73,16 @@ public static String getPartitionPathFromGenericRecord(GenericRecord genericReco
*/
public static String[] extractRecordKeys(String recordKey) {
String[] fieldKV = recordKey.split(",");
- if (fieldKV.length == 1) {
- return fieldKV;
- } else {
- // a complex key
- return Arrays.stream(fieldKV).map(kv -> {
- final String[] kvArray = kv.split(":");
- if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
- return null;
- } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
- return "";
- } else {
- return kvArray[1];
- }
- }).toArray(String[]::new);
- }
+ return Arrays.stream(fieldKV).map(kv -> {
+ final String[] kvArray = kv.split(":");
+ if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
+ return null;
+ } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
+ return "";
+ } else {
+ return kvArray[1];
+ }
+ }).toArray(String[]::new);
Review Comment:
Thanks, generally we should not use `Complex` key generators for single field primary key, but the fix makes the logic more robust,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wuwenchi commented on a diff in pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
wuwenchi commented on code in PR #6539:
URL: https://github.com/apache/hudi/pull/6539#discussion_r958351246
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:
##########
@@ -73,21 +73,16 @@ public static String getPartitionPathFromGenericRecord(GenericRecord genericReco
*/
public static String[] extractRecordKeys(String recordKey) {
String[] fieldKV = recordKey.split(",");
- if (fieldKV.length == 1) {
- return fieldKV;
- } else {
- // a complex key
- return Arrays.stream(fieldKV).map(kv -> {
- final String[] kvArray = kv.split(":");
- if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
- return null;
- } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
- return "";
- } else {
- return kvArray[1];
- }
- }).toArray(String[]::new);
- }
+ return Arrays.stream(fieldKV).map(kv -> {
+ final String[] kvArray = kv.split(":");
+ if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
+ return null;
+ } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
+ return "";
+ } else {
+ return kvArray[1];
+ }
+ }).toArray(String[]::new);
Review Comment:
Here's what I know so far. If it is delete type data, hudi will use `DeleteRecord#create` to generate `DeleteRecord` according to the data. This class records the `HoodieKey` and then writes it to the avro log file. When the log is read, data of type `HoodieAvroRecord` is generated, and the `getRecordKey` method is called to obtain the primary key information of the data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1231246179
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11032",
"triggerID" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"triggerType" : "PUSH"
}, {
"hash" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11040",
"triggerID" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 822e071498bbe67aaaced421c27cdffb8e9e6584 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11032)
* 0e648e21e15d21e537914ef8dbec051ea761f76e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11040)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #6539:
URL: https://github.com/apache/hudi/pull/6539#discussion_r958250709
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:
##########
@@ -73,21 +73,16 @@ public static String getPartitionPathFromGenericRecord(GenericRecord genericReco
*/
public static String[] extractRecordKeys(String recordKey) {
String[] fieldKV = recordKey.split(",");
- if (fieldKV.length == 1) {
- return fieldKV;
- } else {
- // a complex key
- return Arrays.stream(fieldKV).map(kv -> {
- final String[] kvArray = kv.split(":");
- if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
- return null;
- } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
- return "";
- } else {
- return kvArray[1];
- }
- }).toArray(String[]::new);
- }
+ return Arrays.stream(fieldKV).map(kv -> {
+ final String[] kvArray = kv.split(":");
+ if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
+ return null;
+ } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
+ return "";
+ } else {
+ return kvArray[1];
+ }
+ }).toArray(String[]::new);
Review Comment:
Can you elaborate a little more for the details, a delete record still has a primary key right ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1231241305
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11032",
"triggerID" : "822e071498bbe67aaaced421c27cdffb8e9e6584",
"triggerType" : "PUSH"
}, {
"hash" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0e648e21e15d21e537914ef8dbec051ea761f76e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 822e071498bbe67aaaced421c27cdffb8e9e6584 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11032)
* 0e648e21e15d21e537914ef8dbec051ea761f76e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wuwenchi commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
wuwenchi commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1231120563
@danny0405 Can you help review it? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #6539:
URL: https://github.com/apache/hudi/pull/6539#discussion_r958009785
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:
##########
@@ -73,21 +73,16 @@ public static String getPartitionPathFromGenericRecord(GenericRecord genericReco
*/
public static String[] extractRecordKeys(String recordKey) {
String[] fieldKV = recordKey.split(",");
- if (fieldKV.length == 1) {
- return fieldKV;
- } else {
- // a complex key
- return Arrays.stream(fieldKV).map(kv -> {
- final String[] kvArray = kv.split(":");
- if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
- return null;
- } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
- return "";
- } else {
- return kvArray[1];
- }
- }).toArray(String[]::new);
- }
+ return Arrays.stream(fieldKV).map(kv -> {
+ final String[] kvArray = kv.split(":");
+ if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
+ return null;
+ } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
+ return "";
+ } else {
+ return kvArray[1];
+ }
+ }).toArray(String[]::new);
Review Comment:
@wuwenchi could you add a unit test for the util method considering the fixed case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wuwenchi commented on a diff in pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
wuwenchi commented on code in PR #6539:
URL: https://github.com/apache/hudi/pull/6539#discussion_r958111163
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:
##########
@@ -73,21 +73,16 @@ public static String getPartitionPathFromGenericRecord(GenericRecord genericReco
*/
public static String[] extractRecordKeys(String recordKey) {
String[] fieldKV = recordKey.split(",");
- if (fieldKV.length == 1) {
- return fieldKV;
- } else {
- // a complex key
- return Arrays.stream(fieldKV).map(kv -> {
- final String[] kvArray = kv.split(":");
- if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
- return null;
- } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
- return "";
- } else {
- return kvArray[1];
- }
- }).toArray(String[]::new);
- }
+ return Arrays.stream(fieldKV).map(kv -> {
+ final String[] kvArray = kv.split(":");
+ if (kvArray[1].equals(NULL_RECORDKEY_PLACEHOLDER)) {
+ return null;
+ } else if (kvArray[1].equals(EMPTY_RECORDKEY_PLACEHOLDER)) {
+ return "";
+ } else {
+ return kvArray[1];
+ }
+ }).toArray(String[]::new);
Review Comment:
When a delete record is obtained, only `recordKey` is recorded in avro, and the fixed format of `recordKey` is `key:value`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #6539: [HUDI-4739] Wrong value returned when key's length equals 1
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #6539:
URL: https://github.com/apache/hudi/pull/6539#issuecomment-1236227724
can you fill in the PR template (change logs, impact etc)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org