You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@amoro.apache.org by "XBaith (via GitHub)" <gi...@apache.org> on 2024/05/10 02:27:35 UTC
[PR] [AMORO-2823] update catalog meta and corresponding terminal session [amoro]
XBaith opened a new pull request, #2824:
URL: https://github.com/apache/amoro/pull/2824
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://amoro.apache.org/how-to-contribute/
2. If the PR is related to an issue in https://github.com/apache/amoro/issues, add '[AMORO-XXXX]' in your PR title, e.g., '[AMORO-XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][AMORO-XXXX] Your PR title ...'.
-->
## Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about its use case.
2. If you fix a bug, you can clarify why it is a bug.
3. Use Fix/Resolve/Close #{ISSUE_NUMBER} to link this PR to its related issue
-->
Close #2823
## Brief change log
<!--
Clearly describe the changes made in modules, classes, methods, etc.
-->
- fix bugs
## How was this patch tested?
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] Run test locally before making a pull request
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2823] update catalog meta and corresponding terminal session [amoro]
Posted by "XBaith (via GitHub)" <gi...@apache.org>.
XBaith commented on code in PR #2824:
URL: https://github.com/apache/amoro/pull/2824#discussion_r1597343567
##########
amoro-ams/amoro-ams-server/src/main/java/org/apache/amoro/server/terminal/TerminalManager.java:
##########
@@ -370,7 +380,16 @@ private void applyClientProperties(CatalogMeta catalogMeta) {
catalogMeta.putToCatalogProperties(
CatalogMetaProperties.KEY_WAREHOUSE, catalogMeta.getCatalogName());
} else if (!catalogMeta.getCatalogProperties().containsKey(CatalogProperties.CATALOG_IMPL)) {
- catalogMeta.putToCatalogProperties("type", catalogType);
+ if (Sets.newHashSet(
Review Comment:
I see. I updated the code, PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2823] update catalog meta and corresponding terminal session [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong commented on code in PR #2824:
URL: https://github.com/apache/amoro/pull/2824#discussion_r1597331981
##########
amoro-ams/amoro-ams-server/src/main/java/org/apache/amoro/server/terminal/TerminalManager.java:
##########
@@ -370,7 +380,16 @@ private void applyClientProperties(CatalogMeta catalogMeta) {
catalogMeta.putToCatalogProperties(
CatalogMetaProperties.KEY_WAREHOUSE, catalogMeta.getCatalogName());
} else if (!catalogMeta.getCatalogProperties().containsKey(CatalogProperties.CATALOG_IMPL)) {
- catalogMeta.putToCatalogProperties("type", catalogType);
+ if (Sets.newHashSet(
Review Comment:
According to the code here, If the `catalog-impl` property exists, we will skip to set the type, so it will be fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2823] update catalog meta and corresponding terminal session [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong commented on code in PR #2824:
URL: https://github.com/apache/amoro/pull/2824#discussion_r1597340718
##########
amoro-ams/amoro-ams-server/src/main/java/org/apache/amoro/server/terminal/TerminalManager.java:
##########
@@ -370,7 +380,16 @@ private void applyClientProperties(CatalogMeta catalogMeta) {
catalogMeta.putToCatalogProperties(
CatalogMetaProperties.KEY_WAREHOUSE, catalogMeta.getCatalogName());
} else if (!catalogMeta.getCatalogProperties().containsKey(CatalogProperties.CATALOG_IMPL)) {
- catalogMeta.putToCatalogProperties("type", catalogType);
+ if (Sets.newHashSet(
Review Comment:
What I mean is we may better construct the required properties when the catalog is created and persist them in the sysdb rather than put them every time we use them. `AMS_URI` may be different because it may change when AMS reboots.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2823] update catalog meta and corresponding terminal session [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong commented on code in PR #2824:
URL: https://github.com/apache/amoro/pull/2824#discussion_r1597344550
##########
amoro-ams/amoro-ams-server/src/main/java/org/apache/amoro/server/terminal/TerminalManager.java:
##########
@@ -370,7 +380,16 @@ private void applyClientProperties(CatalogMeta catalogMeta) {
catalogMeta.putToCatalogProperties(
CatalogMetaProperties.KEY_WAREHOUSE, catalogMeta.getCatalogName());
} else if (!catalogMeta.getCatalogProperties().containsKey(CatalogProperties.CATALOG_IMPL)) {
- catalogMeta.putToCatalogProperties("type", catalogType);
+ if (Sets.newHashSet(
Review Comment:
Thanks for the update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2823] update catalog meta and corresponding terminal session [amoro]
Posted by "XBaith (via GitHub)" <gi...@apache.org>.
XBaith commented on code in PR #2824:
URL: https://github.com/apache/amoro/pull/2824#discussion_r1597334228
##########
amoro-ams/amoro-ams-server/src/main/java/org/apache/amoro/server/terminal/TerminalManager.java:
##########
@@ -370,7 +380,16 @@ private void applyClientProperties(CatalogMeta catalogMeta) {
catalogMeta.putToCatalogProperties(
CatalogMetaProperties.KEY_WAREHOUSE, catalogMeta.getCatalogName());
} else if (!catalogMeta.getCatalogProperties().containsKey(CatalogProperties.CATALOG_IMPL)) {
- catalogMeta.putToCatalogProperties("type", catalogType);
+ if (Sets.newHashSet(
Review Comment:
Do you mean we can add the property when build catalog instance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2823] update catalog meta and corresponding terminal session [amoro]
Posted by "XBaith (via GitHub)" <gi...@apache.org>.
XBaith commented on code in PR #2824:
URL: https://github.com/apache/amoro/pull/2824#discussion_r1597331897
##########
amoro-ams/amoro-ams-server/src/main/java/org/apache/amoro/server/terminal/TerminalManager.java:
##########
@@ -121,17 +124,24 @@ public String executeScript(String terminalId, String catalog, String script) {
TerminalSessionFactory.SessionConfigOptions.catalogProperty(catalog, key), value);
}
- TerminalSessionContext context;
synchronized (sessionMapLock) {
- sessionMap.computeIfAbsent(
+ sessionMap.compute(
sessionId,
- id ->
- new TerminalSessionContext(
- id, metaStore, executionPool, sessionFactory, configuration));
-
- context = sessionMap.get(sessionId);
+ (id, ctx) -> {
Review Comment:
Yes, because I found that session configuration was not update after changing server catalog configurations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2823] update catalog meta and corresponding terminal session [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong commented on code in PR #2824:
URL: https://github.com/apache/amoro/pull/2824#discussion_r1597328063
##########
amoro-ams/amoro-ams-server/src/main/java/org/apache/amoro/server/terminal/TerminalManager.java:
##########
@@ -121,17 +124,24 @@ public String executeScript(String terminalId, String catalog, String script) {
TerminalSessionFactory.SessionConfigOptions.catalogProperty(catalog, key), value);
}
- TerminalSessionContext context;
synchronized (sessionMapLock) {
- sessionMap.computeIfAbsent(
+ sessionMap.compute(
sessionId,
- id ->
- new TerminalSessionContext(
- id, metaStore, executionPool, sessionFactory, configuration));
-
- context = sessionMap.get(sessionId);
+ (id, ctx) -> {
Review Comment:
Changes here seem to fix another issue.
##########
amoro-ams/amoro-ams-server/src/main/java/org/apache/amoro/server/terminal/TerminalManager.java:
##########
@@ -370,7 +380,16 @@ private void applyClientProperties(CatalogMeta catalogMeta) {
catalogMeta.putToCatalogProperties(
CatalogMetaProperties.KEY_WAREHOUSE, catalogMeta.getCatalogName());
} else if (!catalogMeta.getCatalogProperties().containsKey(CatalogProperties.CATALOG_IMPL)) {
- catalogMeta.putToCatalogProperties("type", catalogType);
+ if (Sets.newHashSet(
Review Comment:
How about setting the `catalog-impl` properties automatically when creating the Glue catalog?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2823] update catalog meta and corresponding terminal session [amoro]
Posted by "XBaith (via GitHub)" <gi...@apache.org>.
XBaith commented on code in PR #2824:
URL: https://github.com/apache/amoro/pull/2824#discussion_r1597331773
##########
amoro-ams/amoro-ams-server/src/main/java/org/apache/amoro/server/terminal/TerminalManager.java:
##########
@@ -370,7 +380,16 @@ private void applyClientProperties(CatalogMeta catalogMeta) {
catalogMeta.putToCatalogProperties(
CatalogMetaProperties.KEY_WAREHOUSE, catalogMeta.getCatalogName());
} else if (!catalogMeta.getCatalogProperties().containsKey(CatalogProperties.CATALOG_IMPL)) {
- catalogMeta.putToCatalogProperties("type", catalogType);
+ if (Sets.newHashSet(
Review Comment:
Setting the `type` and `catalog-impl` properties at the same time will cause a conflict exception for Iceberg catalog
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2823] update catalog meta and corresponding terminal session [amoro]
Posted by "XBaith (via GitHub)" <gi...@apache.org>.
XBaith merged PR #2824:
URL: https://github.com/apache/amoro/pull/2824
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org