You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "Zouxxyy (via GitHub)" <gi...@apache.org> on 2024/01/15 04:13:12 UTC
[PR] [core][spark] Support create database with location, comment, props using hive catalog [incubator-paimon]
Zouxxyy opened a new pull request, #2698:
URL: https://github.com/apache/incubator-paimon/pull/2698
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close #2619
support create database with location, comment, props
```sql
CREATE DATABASE db LOCATION 'x' COMMENT 'x' WITH DBPROPERTIES ('k1' = 'v1', 'k2' = 'v2')
```
Note:
- for hive catalog: store all props(location, comment, other props) to hms
- for filesystem catalog: when props is not empty, do nothings but give a warn `Currently filesystem catalog can't store database properties, discard properties: {}`, because filesystem catalog can't store meta. Besides, because db's location in filesystem catalog is stable, **when create db location 'xxx' with filesystem catalog, an exception is thrown**: `Cannot specify location for a database when using fileSystem catalog`
<!-- What is the purpose of the change -->
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][spark] Support create database with location, comment, props using hive catalog [incubator-paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #2698:
URL: https://github.com/apache/incubator-paimon/pull/2698#discussion_r1452967252
##########
paimon-core/src/main/java/org/apache/paimon/catalog/Catalog.java:
##########
@@ -71,17 +71,33 @@ default Optional<MetastoreClient.Factory> metastoreClientFactory(Identifier iden
*/
boolean databaseExists(String databaseName);
+ default void createDatabase(String name, boolean ignoreIfExists)
Review Comment:
document this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][spark] Support create database with location, comment, props using hive catalog [incubator-paimon]
Posted by "YannByron (via GitHub)" <gi...@apache.org>.
YannByron commented on PR #2698:
URL: https://github.com/apache/incubator-paimon/pull/2698#issuecomment-1893043342
just leave two tiny comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][spark] Support create database with location, comment, props using hive catalog [incubator-paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #2698:
URL: https://github.com/apache/incubator-paimon/pull/2698
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][spark] Support create database with location, comment, props using hive catalog [incubator-paimon]
Posted by "YannByron (via GitHub)" <gi...@apache.org>.
YannByron commented on code in PR #2698:
URL: https://github.com/apache/incubator-paimon/pull/2698#discussion_r1452905049
##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -105,10 +106,23 @@ public void createDatabase(String name, boolean ignoreIfExists)
}
throw new DatabaseAlreadyExistException(name);
}
+ createDatabaseImpl(name, properties);
+ }
- createDatabaseImpl(name);
+ @Override
+ public Map<String, String> loadDatabaseProperties(String name)
+ throws DatabaseNotExistException {
+ if (isSystemDatabase(name)) {
+ return Collections.emptyMap();
+ }
+ if (!databaseExists(name)) {
+ throw new DatabaseNotExistException(name);
+ }
+ return loadDatabasePropertiesImpl(name);
}
+ protected abstract Map<String, String> loadDatabasePropertiesImpl(String name);
Review Comment:
I prefer to name this method `loadDatabaseProperties0` if this is required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][spark] Support create database with location, comment, props using hive catalog [incubator-paimon]
Posted by "YannByron (via GitHub)" <gi...@apache.org>.
YannByron commented on code in PR #2698:
URL: https://github.com/apache/incubator-paimon/pull/2698#discussion_r1452902910
##########
paimon-core/src/main/java/org/apache/paimon/catalog/Catalog.java:
##########
@@ -81,7 +81,33 @@ default Optional<MetastoreClient.Factory> metastoreClientFactory(Identifier iden
* @throws DatabaseAlreadyExistException if the given database already exists and ignoreIfExists
* is false
*/
- void createDatabase(String name, boolean ignoreIfExists) throws DatabaseAlreadyExistException;
Review Comment:
the repeated doc is not necessary. merge them as the following one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][spark] Support create database with location, comment, props using hive catalog [incubator-paimon]
Posted by "Zouxxyy (via GitHub)" <gi...@apache.org>.
Zouxxyy commented on code in PR #2698:
URL: https://github.com/apache/incubator-paimon/pull/2698#discussion_r1452956660
##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -105,10 +106,23 @@ public void createDatabase(String name, boolean ignoreIfExists)
}
throw new DatabaseAlreadyExistException(name);
}
+ createDatabaseImpl(name, properties);
+ }
- createDatabaseImpl(name);
+ @Override
+ public Map<String, String> loadDatabaseProperties(String name)
+ throws DatabaseNotExistException {
+ if (isSystemDatabase(name)) {
+ return Collections.emptyMap();
+ }
+ if (!databaseExists(name)) {
+ throw new DatabaseNotExistException(name);
+ }
+ return loadDatabasePropertiesImpl(name);
}
+ protected abstract Map<String, String> loadDatabasePropertiesImpl(String name);
Review Comment:
Just for unified code style, like createDatabase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core][spark] Support create database with location, comment, props using hive catalog [incubator-paimon]
Posted by "YannByron (via GitHub)" <gi...@apache.org>.
YannByron commented on PR #2698:
URL: https://github.com/apache/incubator-paimon/pull/2698#issuecomment-1893114659
LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org