You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "jonvex (via GitHub)" <gi...@apache.org> on 2024/11/21 16:43:26 UTC
[PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
jonvex opened a new pull request, #12307:
URL: https://github.com/apache/hudi/pull/12307
### Change Logs
meta client fails due to both timeline layout options being empty, now we will throw a proper error.
### Impact
better error message for users
test expected tablenotfoundexception when the table is not ready, so now it will be the expected exception
### Risk level (write none, low medium or high below)
low
### Documentation Update
N/A
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12307:
URL: https://github.com/apache/hudi/pull/12307#issuecomment-2491752599
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "61987e0e9b50cafc510aac777ca441c1f19e9756",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "61987e0e9b50cafc510aac777ca441c1f19e9756",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 61987e0e9b50cafc510aac777ca441c1f19e9756 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12307:
URL: https://github.com/apache/hudi/pull/12307#issuecomment-2491757480
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "61987e0e9b50cafc510aac777ca441c1f19e9756",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=1814",
"triggerID" : "61987e0e9b50cafc510aac777ca441c1f19e9756",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 61987e0e9b50cafc510aac777ca441c1f19e9756 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=1814)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #12307:
URL: https://github.com/apache/hudi/pull/12307#discussion_r1857207184
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -176,8 +177,10 @@ protected HoodieTableMetaClient(HoodieStorage storage, String basePath, boolean
checkArgument(layoutVersion.get().compareTo(tableConfigVersion.get()) >= 0,
"Layout Version defined in hoodie properties has higher version (" + tableConfigVersion.get()
+ ") than the one passed in config (" + layoutVersion.get() + ")");
+ } else if (layoutVersion.isEmpty() && tableConfigVersion.isEmpty()) {
Review Comment:
I think it is just when the table has not been initialized and it is passed in as empty into the constructor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #12307:
URL: https://github.com/apache/hudi/pull/12307#discussion_r1857204629
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -176,8 +177,10 @@ protected HoodieTableMetaClient(HoodieStorage storage, String basePath, boolean
checkArgument(layoutVersion.get().compareTo(tableConfigVersion.get()) >= 0,
"Layout Version defined in hoodie properties has higher version (" + tableConfigVersion.get()
+ ") than the one passed in config (" + layoutVersion.get() + ")");
+ } else if (layoutVersion.isEmpty() && tableConfigVersion.isEmpty()) {
Review Comment:
In what scenario both layoutVersion and tableConfigVersion are empty?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #12307:
URL: https://github.com/apache/hudi/pull/12307#discussion_r1872244128
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -176,8 +177,10 @@ protected HoodieTableMetaClient(HoodieStorage storage, String basePath, boolean
checkArgument(layoutVersion.get().compareTo(tableConfigVersion.get()) >= 0,
"Layout Version defined in hoodie properties has higher version (" + tableConfigVersion.get()
+ ") than the one passed in config (" + layoutVersion.get() + ")");
+ } else if (layoutVersion.isEmpty() && tableConfigVersion.isEmpty()) {
+ throw new TableNotFoundException("Table does not exist");
Review Comment:
Should we say the table config is corrupted instead of `TableNotFoundException`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #12307:
URL: https://github.com/apache/hudi/pull/12307#discussion_r1855573497
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -176,8 +177,10 @@ protected HoodieTableMetaClient(HoodieStorage storage, String basePath, boolean
checkArgument(layoutVersion.get().compareTo(tableConfigVersion.get()) >= 0,
"Layout Version defined in hoodie properties has higher version (" + tableConfigVersion.get()
+ ") than the one passed in config (" + layoutVersion.get() + ")");
+ } else if (layoutVersion.isEmpty() && tableConfigVersion.isEmpty()) {
+ throw new TableNotFoundException("Table does not exist");
}
- this.timelineLayoutVersion = layoutVersion.orElseGet(() -> tableConfig.getTimelineLayoutVersion().get());
+ this.timelineLayoutVersion = layoutVersion.orElseGet(tableConfigVersion::get);
Review Comment:
We can discuss tomorrow if your mind isn't changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #12307:
URL: https://github.com/apache/hudi/pull/12307#discussion_r1853387703
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -176,8 +177,10 @@ protected HoodieTableMetaClient(HoodieStorage storage, String basePath, boolean
checkArgument(layoutVersion.get().compareTo(tableConfigVersion.get()) >= 0,
"Layout Version defined in hoodie properties has higher version (" + tableConfigVersion.get()
+ ") than the one passed in config (" + layoutVersion.get() + ")");
+ } else if (layoutVersion.isEmpty() && tableConfigVersion.isEmpty()) {
+ throw new TableNotFoundException("Table does not exist");
}
- this.timelineLayoutVersion = layoutVersion.orElseGet(() -> tableConfig.getTimelineLayoutVersion().get());
+ this.timelineLayoutVersion = layoutVersion.orElseGet(tableConfigVersion::get);
Review Comment:
I think the issue is that when `reloadTableConfig` is called then timelineLayoutVersion is not updated. We should load timelineLayoutversion within reloadTableConig.
```
public synchronized void reloadTableConfig() {
this.tableConfig = new HoodieTableConfig(this.storage, metaPath,
this.tableConfig.getRecordMergeMode(), this.tableConfig.getKeyGeneratorClassName(), this.tableConfig.getRecordMergeStrategyId());
reloadTimelineLayout();
}
private void reloadTimelineLayout() {
this.timelineLayoutVersion = tableConfig.getTimelineLayoutVersion().get();
this.timelineLayout = TimelineLayout.fromVersion(timelineLayoutVersion);
this.timelinePath = timelineLayout.getTimelinePath(this.basePath);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex merged PR #12307:
URL: https://github.com/apache/hudi/pull/12307
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #12307:
URL: https://github.com/apache/hudi/pull/12307#issuecomment-2491921418
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "61987e0e9b50cafc510aac777ca441c1f19e9756",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=1814",
"triggerID" : "61987e0e9b50cafc510aac777ca441c1f19e9756",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 61987e0e9b50cafc510aac777ca441c1f19e9756 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=1814)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8548] fail create meta client if table does not exist and proper configs not set [hudi]
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #12307:
URL: https://github.com/apache/hudi/pull/12307#discussion_r1855573448
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -176,8 +177,10 @@ protected HoodieTableMetaClient(HoodieStorage storage, String basePath, boolean
checkArgument(layoutVersion.get().compareTo(tableConfigVersion.get()) >= 0,
"Layout Version defined in hoodie properties has higher version (" + tableConfigVersion.get()
+ ") than the one passed in config (" + layoutVersion.get() + ")");
+ } else if (layoutVersion.isEmpty() && tableConfigVersion.isEmpty()) {
+ throw new TableNotFoundException("Table does not exist");
}
- this.timelineLayoutVersion = layoutVersion.orElseGet(() -> tableConfig.getTimelineLayoutVersion().get());
+ this.timelineLayoutVersion = layoutVersion.orElseGet(tableConfigVersion::get);
Review Comment:
take a look at the stack trace from the jira ticket
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org