You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/04/13 02:08:15 UTC
[GitHub] [hudi] xicm opened a new pull request, #5308: HUDI-3862 Some default configurations in HoodieHBaseIndexConfig do not work
xicm opened a new pull request, #5308:
URL: https://github.com/apache/hudi/pull/5308
…t work
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
HUDI-3862 Some default configurations in HoodieHBaseIndexConfig do not work
## Brief change log
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieConfig.java
## Verify this pull request
This change added tests and can be verified as follows:
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5308: [HUDI-3862] Fix default configurations of HoodieHBaseIndexConfig
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5308:
URL: https://github.com/apache/hudi/pull/5308#issuecomment-1097905227
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031",
"triggerID" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"triggerType" : "PUSH"
}, {
"hash" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8036",
"triggerID" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"triggerType" : "PUSH"
}, {
"hash" : "4082b4566643bf3a37ae7b924182e96e2057cb81",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8043",
"triggerID" : "4082b4566643bf3a37ae7b924182e96e2057cb81",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4082b4566643bf3a37ae7b924182e96e2057cb81 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8043)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on a diff in pull request #5308: [HUDI-3862] Fix default configurations of HoodieHBaseIndexConfig
Posted by GitBox <gi...@apache.org>.
xicm commented on code in PR #5308:
URL: https://github.com/apache/hudi/pull/5308#discussion_r849203953
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -1453,43 +1453,43 @@ public String getHbaseTableName() {
}
public int getHbaseIndexGetBatchSize() {
- return getInt(HoodieHBaseIndexConfig.GET_BATCH_SIZE);
+ return getIntOrDefault(HoodieHBaseIndexConfig.GET_BATCH_SIZE);
Review Comment:
Thanks for your comments, I updated the pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5308: HUDI-3862 Some default configurations in HoodieHBaseIndexConfig do not work
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5308:
URL: https://github.com/apache/hudi/pull/5308#issuecomment-1097550162
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031",
"triggerID" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"triggerType" : "PUSH"
}, {
"hash" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4833349933c202632b3230f3db85ed3dac3a47cc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031)
* 192c99c64209bfd34220c0ef6c9acd47926f2117 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5308: HUDI-3862 Some default configurations in HoodieHBaseIndexConfig do not work
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5308:
URL: https://github.com/apache/hudi/pull/5308#issuecomment-1097480449
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031",
"triggerID" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4833349933c202632b3230f3db85ed3dac3a47cc Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5308: HUDI-3862 Some default configurations in HoodieHBaseIndexConfig do not work
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5308:
URL: https://github.com/apache/hudi/pull/5308#issuecomment-1097543379
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031",
"triggerID" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4833349933c202632b3230f3db85ed3dac3a47cc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5308: HUDI-3862 Some default configurations in HoodieHBaseIndexConfig do not work
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5308:
URL: https://github.com/apache/hudi/pull/5308#issuecomment-1097551621
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031",
"triggerID" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"triggerType" : "PUSH"
}, {
"hash" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8036",
"triggerID" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4833349933c202632b3230f3db85ed3dac3a47cc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031)
* 192c99c64209bfd34220c0ef6c9acd47926f2117 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8036)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #5308: HUDI-3862 Some default configurations in HoodieHBaseIndexConfig do not work
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #5308:
URL: https://github.com/apache/hudi/pull/5308#discussion_r849137873
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -1453,43 +1453,43 @@ public String getHbaseTableName() {
}
public int getHbaseIndexGetBatchSize() {
- return getInt(HoodieHBaseIndexConfig.GET_BATCH_SIZE);
+ return getIntOrDefault(HoodieHBaseIndexConfig.GET_BATCH_SIZE);
Review Comment:
Thanks for reporting the issue. Actually, all of these changes are not needed if the `HoodieHBaseIndexConfig` defaults are set properly. It looks like that the root cause is that the builder of `HoodieWriteConfig` does not set defaults for `HoodieHBaseIndexConfig` in `setDefaults()`, like other Hudi config class. Could you fix that and revert all getter changes instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua merged pull request #5308: [HUDI-3862] Fix default configurations of HoodieHBaseIndexConfig
Posted by GitBox <gi...@apache.org>.
yihua merged PR #5308:
URL: https://github.com/apache/hudi/pull/5308
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5308: [HUDI-3862] Fix default configurations of HoodieHBaseIndexConfig
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5308:
URL: https://github.com/apache/hudi/pull/5308#issuecomment-1097692750
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031",
"triggerID" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"triggerType" : "PUSH"
}, {
"hash" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8036",
"triggerID" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"triggerType" : "PUSH"
}, {
"hash" : "4082b4566643bf3a37ae7b924182e96e2057cb81",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4082b4566643bf3a37ae7b924182e96e2057cb81",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 192c99c64209bfd34220c0ef6c9acd47926f2117 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8036)
* 4082b4566643bf3a37ae7b924182e96e2057cb81 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5308: HUDI-3862 Some default configurations in HoodieHBaseIndexConfig do not work
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5308:
URL: https://github.com/apache/hudi/pull/5308#issuecomment-1097478746
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4833349933c202632b3230f3db85ed3dac3a47cc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5308: [HUDI-3862] Fix default configurations of HoodieHBaseIndexConfig
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5308:
URL: https://github.com/apache/hudi/pull/5308#issuecomment-1097635030
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031",
"triggerID" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"triggerType" : "PUSH"
}, {
"hash" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8036",
"triggerID" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 192c99c64209bfd34220c0ef6c9acd47926f2117 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8036)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5308: [HUDI-3862] Fix default configurations of HoodieHBaseIndexConfig
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5308:
URL: https://github.com/apache/hudi/pull/5308#issuecomment-1097696171
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8031",
"triggerID" : "4833349933c202632b3230f3db85ed3dac3a47cc",
"triggerType" : "PUSH"
}, {
"hash" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8036",
"triggerID" : "192c99c64209bfd34220c0ef6c9acd47926f2117",
"triggerType" : "PUSH"
}, {
"hash" : "4082b4566643bf3a37ae7b924182e96e2057cb81",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8043",
"triggerID" : "4082b4566643bf3a37ae7b924182e96e2057cb81",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 192c99c64209bfd34220c0ef6c9acd47926f2117 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8036)
* 4082b4566643bf3a37ae7b924182e96e2057cb81 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8043)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org