You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@amoro.apache.org by "Aireed (via GitHub)" <gi...@apache.org> on 2024/05/10 04:25:39 UTC
[PR] [AMORO-2819]Spark cannot execute the "alter table set identifier field" command on tables in Iceberg format. [amoro]
Aireed opened a new pull request, #2825:
URL: https://github.com/apache/amoro/pull/2825
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://amoro.apache.org/how-to-contribute/
2. If the PR is related to an issue in https://github.com/apache/amoro/issues, add '[AMORO-XXXX]' in your PR title, e.g., '[AMORO-XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][AMORO-XXXX] Your PR title ...'.
-->
## Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about its use case.
2. If you fix a bug, you can clarify why it is a bug.
3. Use Fix/Resolve/Close #{ISSUE_NUMBER} to link this PR to its related issue
-->
Close #2819 .
## Brief change log
<!--
Clearly describe the changes made in modules, classes, methods, etc.
-->
The cause:
- UnifiedSessionCatalog is neither SparkCatalog nor SparkSessionCatalog of Icebergļ¼ so it doesn't match catalogAndIdentifier, and can't get the physical plan
solution:
1. copy the iceberg ExtendedDataSourceV2Strategy and override ` IcebergCatalogAndIdentifier` as AmoroExtendedDataSourceV2Strategy
2. replace ExtendedDataSourceV2Strategy with AmoroExtendedDataSourceV2Strategy in ArcticSparkExtensions
Tips:
1. ArcticSparkExtensions inject arctic extension and iceberg extension, so We don't need to configure the Iceberg extension in spark.sql.extensions in the future.
## How was this patch tested?
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] Run test locally before making a pull request
## Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2819]Spark cannot execute the "alter table set identifier field" command on tables in Iceberg format in unified catalog [amoro]
Posted by "Aireed (via GitHub)" <gi...@apache.org>.
Aireed closed pull request #2825: [AMORO-2819]Spark cannot execute the "alter table set identifier field" command on tables in Iceberg format in unified catalog
URL: https://github.com/apache/amoro/pull/2825
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2819]Spark cannot execute the "alter table set identifier field" command on tables in Iceberg format in unified catalog [amoro]
Posted by "Aireed (via GitHub)" <gi...@apache.org>.
Aireed closed pull request #2825: [AMORO-2819]Spark cannot execute the "alter table set identifier field" command on tables in Iceberg format in unified catalog
URL: https://github.com/apache/amoro/pull/2825
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2819]Spark cannot execute the "alter table set identifier field" command on tables in Iceberg format in unified catalog [amoro]
Posted by "baiyangtx (via GitHub)" <gi...@apache.org>.
baiyangtx commented on PR #2825:
URL: https://github.com/apache/amoro/pull/2825#issuecomment-2128677581
This PR may cause conflict with https://github.com/apache/amoro/pull/2849.
I suggest that we can wait #2849 is merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2819]Spark cannot execute the "alter table set identifier field" command on tables in Iceberg format. [amoro]
Posted by "Aireed (via GitHub)" <gi...@apache.org>.
Aireed commented on PR #2825:
URL: https://github.com/apache/amoro/pull/2825#issuecomment-2103838101
cc @baiyangtx PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AMORO-2819]Spark cannot execute the "alter table set identifier field" command on tables in Iceberg format in unified catalog [amoro]
Posted by "Aireed (via GitHub)" <gi...@apache.org>.
Aireed commented on PR #2825:
URL: https://github.com/apache/amoro/pull/2825#issuecomment-2144833401
i reope
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org