You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "empcl (via GitHub)" <gi...@apache.org> on 2023/08/31 16:46:31 UTC
[GitHub] [hudi] empcl opened a new pull request, #9592: automatically create a database when using the flink catalog dfs mode
empcl opened a new pull request, #9592:
URL: https://github.com/apache/hudi/pull/9592
x
### Change Logs
_automatically create a database when using the flink catalog dfs mode._
### Impact
_automatically create a database when using the flink catalog dfs mode._
### Risk level (write none, low medium or high below)
_none._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9592: automatically create a database when using the flink catalog dfs mode
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9592:
URL: https://github.com/apache/hudi/pull/9592#issuecomment-1701777353
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c961be19038e5600f418ef660b7ede740cef76c6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19581",
"triggerID" : "c961be19038e5600f418ef660b7ede740cef76c6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c961be19038e5600f418ef660b7ede740cef76c6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19581)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9592: automatically create a database when using the flink catalog dfs mode
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9592:
URL: https://github.com/apache/hudi/pull/9592#issuecomment-1701446824
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c961be19038e5600f418ef660b7ede740cef76c6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19581",
"triggerID" : "c961be19038e5600f418ef660b7ede740cef76c6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c961be19038e5600f418ef660b7ede740cef76c6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19581)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] empcl commented on a diff in pull request #9592: automatically create a database when using the flink catalog dfs mode
Posted by "empcl (via GitHub)" <gi...@apache.org>.
empcl commented on code in PR #9592:
URL: https://github.com/apache/hudi/pull/9592#discussion_r1312612204
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/catalog/TestHoodieCatalog.java:
##########
Review Comment:
because there is already a judgment，`org.apache.hudi.table.catalog.TestHoodieCatalog#testDatabaseExists`
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/catalog/TestHoodieCatalog.java:
##########
Review Comment:
because there is already a judgment，`org.apache.hudi.table.catalog.TestHoodieCatalog#testDatabaseExists`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9592: automatically create a database when using the flink catalog dfs mode
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9592:
URL: https://github.com/apache/hudi/pull/9592#issuecomment-1701434747
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c961be19038e5600f418ef660b7ede740cef76c6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c961be19038e5600f418ef660b7ede740cef76c6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c961be19038e5600f418ef660b7ede740cef76c6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 merged pull request #9592: [HUDI-6818] Create a database automatically when using the flink catalog dfs mode
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #9592:
URL: https://github.com/apache/hudi/pull/9592
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9592: automatically create a database when using the flink catalog dfs mode
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9592:
URL: https://github.com/apache/hudi/pull/9592#issuecomment-1702574292
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c961be19038e5600f418ef660b7ede740cef76c6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19581",
"triggerID" : "c961be19038e5600f418ef660b7ede740cef76c6",
"triggerType" : "PUSH"
}, {
"hash" : "702653a08249790e738497e49ddc9970613e2343",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19600",
"triggerID" : "702653a08249790e738497e49ddc9970613e2343",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 702653a08249790e738497e49ddc9970613e2343 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19600)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #9592: automatically create a database when using the flink catalog dfs mode
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #9592:
URL: https://github.com/apache/hudi/pull/9592#discussion_r1312603674
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/catalog/TestHoodieCatalog.java:
##########
Review Comment:
Can you add a test case where default database is created?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #9592: automatically create a database when using the flink catalog dfs mode
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #9592:
URL: https://github.com/apache/hudi/pull/9592#discussion_r1312444984
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/catalog/HoodieCatalog.java:
##########
@@ -125,6 +125,16 @@ public void open() throws CatalogException {
} catch (IOException e) {
throw new CatalogException(String.format("Checking catalog path %s exists exception.", catalogPathStr), e);
}
+
+ if (!databaseExists(getDefaultDatabase())) {
+ LOG.info("Creating database {} automatically because it does not exist.", getDefaultDatabase());
+ Path dbPath = new Path(catalogPath, getDefaultDatabase());
Review Comment:
Can we write a test case for it in `TestHoodieCatalog`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] empcl commented on a diff in pull request #9592: automatically create a database when using the flink catalog dfs mode
Posted by "empcl (via GitHub)" <gi...@apache.org>.
empcl commented on code in PR #9592:
URL: https://github.com/apache/hudi/pull/9592#discussion_r1312545914
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/catalog/TestHoodieCatalog.java:
##########
Review Comment:
@danny0405 Hello, currently in the test cases, we should not manually create the caatalog+db path, but instead create the db directory by calling the open() method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9592: automatically create a database when using the flink catalog dfs mode
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9592:
URL: https://github.com/apache/hudi/pull/9592#issuecomment-1702170818
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c961be19038e5600f418ef660b7ede740cef76c6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19581",
"triggerID" : "c961be19038e5600f418ef660b7ede740cef76c6",
"triggerType" : "PUSH"
}, {
"hash" : "702653a08249790e738497e49ddc9970613e2343",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19600",
"triggerID" : "702653a08249790e738497e49ddc9970613e2343",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c961be19038e5600f418ef660b7ede740cef76c6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19581)
* 702653a08249790e738497e49ddc9970613e2343 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19600)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9592: automatically create a database when using the flink catalog dfs mode
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9592:
URL: https://github.com/apache/hudi/pull/9592#issuecomment-1702164906
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c961be19038e5600f418ef660b7ede740cef76c6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19581",
"triggerID" : "c961be19038e5600f418ef660b7ede740cef76c6",
"triggerType" : "PUSH"
}, {
"hash" : "702653a08249790e738497e49ddc9970613e2343",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "702653a08249790e738497e49ddc9970613e2343",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c961be19038e5600f418ef660b7ede740cef76c6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19581)
* 702653a08249790e738497e49ddc9970613e2343 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org