You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@amoro.apache.org by "zhoujinsong (via GitHub)" <gi...@apache.org> on 2024/09/19 09:50:18 UTC
[PR] [AMORO-3210] Support loading flink mixed catalog without ams url [amoro]
zhoujinsong opened a new pull request, #3211:
URL: https://github.com/apache/amoro/pull/3211
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://amoro.apache.org/how-to-contribute/
2. If the PR is related to an issue in https://github.com/apache/amoro/issues, add '[AMORO-XXXX]' in your PR title, e.g., '[AMORO-XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][AMORO-XXXX] Your PR title ...'.
-->
## Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about its use case.
2. If you fix a bug, you can clarify why it is a bug.
3. Use Fix/Resolve/Close #{ISSUE_NUMBER} to link this PR to its related issue
-->
Close #3210.
## Brief change log
<!--
Clearly describe the changes made in modules, classes, methods, etc.
-->
- Support loading Flink mixed catalog without ams url
- Support loading Flink unified catalog without ams url
- Remove some duplicated codes in CatalogUtil
## How was this patch tested?
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] Run test locally before making a pull request
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3210] Support loading flink mixed catalog without ams url [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong merged PR #3211:
URL: https://github.com/apache/amoro/pull/3211
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3210] Support loading flink mixed catalog without ams url [amoro]
Posted by "Aireed (via GitHub)" <gi...@apache.org>.
Aireed commented on PR #3211:
URL: https://github.com/apache/amoro/pull/3211#issuecomment-2362680727
Here, the definition of the catalog's property is completely unrestricted. I suggest keeping the original configuration validation. What do others think? @zhongqishang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3210] Support loading flink mixed catalog without ams url [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong commented on PR #3211:
URL: https://github.com/apache/amoro/pull/3211#issuecomment-2362698197
> Here, the definition of the catalog's property is completely unrestricted. I suggest keeping the original configuration validation. What do others think? @zhongqishang
Thanks for the review, I have made some changes based on the comments.
I removed the catalog properties limit from the catalog factory.
I hope to open up more catalog properties at this stage. I feel that it may not be possible to list all the catalog properties we wish to support for the time being.
And I found the same implementation in Iceberg and Paimon Flink catalog factory:
https://github.com/apache/iceberg/blob/main/flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalogFactory.java#L133
https://github.com/apache/paimon/blob/master/paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/FlinkCatalogFactory.java#L47
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3210] Support loading flink mixed catalog without ams url [amoro]
Posted by "zhoujinsong (via GitHub)" <gi...@apache.org>.
zhoujinsong commented on code in PR #3211:
URL: https://github.com/apache/amoro/pull/3211#discussion_r1767905586
##########
amoro-format-mixed/amoro-mixed-flink/amoro-mixed-flink-common/src/main/java/org/apache/amoro/flink/table/UnifiedDynamicTableFactory.java:
##########
@@ -42,12 +42,12 @@
* AmoroDynamicTableFactory is a factory for creating dynamic table sources and sinks. It implements
Review Comment:
Fixed.
##########
amoro-format-iceberg/src/test/java/org/apache/amoro/catalog/BasicMixedCatalog.java:
##########
@@ -214,7 +215,7 @@ public Map<String, String> properties() {
protected TableMeta getMixedTableMeta(TableIdentifier identifier) {
TableMeta tableMeta;
try {
- tableMeta = getClient().getTable(MixedFormatCatalogUtil.amsTaleId(identifier));
+ tableMeta = getClient().getTable(CatalogUtil.amsTaleId(identifier));
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-3210] Support loading flink mixed catalog without ams url [amoro]
Posted by "Aireed (via GitHub)" <gi...@apache.org>.
Aireed commented on code in PR #3211:
URL: https://github.com/apache/amoro/pull/3211#discussion_r1767843472
##########
amoro-format-iceberg/src/test/java/org/apache/amoro/catalog/BasicMixedCatalog.java:
##########
@@ -214,7 +215,7 @@ public Map<String, String> properties() {
protected TableMeta getMixedTableMeta(TableIdentifier identifier) {
TableMeta tableMeta;
try {
- tableMeta = getClient().getTable(MixedFormatCatalogUtil.amsTaleId(identifier));
+ tableMeta = getClient().getTable(CatalogUtil.amsTaleId(identifier));
Review Comment:
`tableMeta = getClient().getTable(CatalogUtil.amsTaleId(identifier));`
I think `amsTaleId` should be `amsTableId`, previously it was probably misspelled.
##########
amoro-format-mixed/amoro-mixed-flink/amoro-mixed-flink-common/src/main/java/org/apache/amoro/flink/table/UnifiedDynamicTableFactory.java:
##########
@@ -42,12 +42,12 @@
* AmoroDynamicTableFactory is a factory for creating dynamic table sources and sinks. It implements
Review Comment:
replace AmoroDynamicTableFactory with UnifiedDynamicTableFactory in comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org