You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "ulysses-you (via GitHub)" <gi...@apache.org> on 2024/08/23 02:01:07 UTC
[PR] [core] Fix caching catalog case sensitive [paimon]
ulysses-you opened a new pull request, #4043:
URL: https://github.com/apache/paimon/pull/4043
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
The CacaingCatalog should respect the wrapped catalog case sensitive, otherwise it would case wrong result.
```
create table t (c1 int);
drop table T;
create table t (c1 int);
[TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `default`.`T` because it already exists.
```
<!-- What is the purpose of the change -->
### Tests
<!-- List UT and IT cases to verify this change -->
add test
### API and Format
<!-- Does this change affect API or storage format -->
no
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix caching catalog case sensitive [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #4043:
URL: https://github.com/apache/paimon/pull/4043#discussion_r1728374415
##########
paimon-spark/paimon-spark-common/src/test/scala/org/apache/paimon/spark/sql/DDLTestBase.scala:
##########
@@ -456,4 +456,12 @@ abstract class DDLTestBase extends PaimonSparkTestBase {
}.getMessage
assert(error.contains("Unsupported partition transform"))
}
+
+ test("Fix caching catalog case sensitive") {
Review Comment:
I ran this test in the master branch, it passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix caching catalog case sensitive [paimon]
Posted by "ulysses-you (via GitHub)" <gi...@apache.org>.
ulysses-you commented on PR #4043:
URL: https://github.com/apache/paimon/pull/4043#issuecomment-2309200065
@JingsongLi thank you for the context, I sse the option about allow_upper_case which has different default value for different backend engine and catalog, which makes me a bit confused before.. What do you think about this new option ? I think it at least can resovle some issues mannually and I see the iceberg also has a dedicated option for caching catalog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix caching catalog case sensitive [paimon]
Posted by "ulysses-you (via GitHub)" <gi...@apache.org>.
ulysses-you commented on code in PR #4043:
URL: https://github.com/apache/paimon/pull/4043#discussion_r1728385278
##########
paimon-spark/paimon-spark-common/src/test/scala/org/apache/paimon/spark/sql/DDLTestBase.scala:
##########
@@ -456,4 +456,12 @@ abstract class DDLTestBase extends PaimonSparkTestBase {
}.getMessage
assert(error.contains("Unsupported partition transform"))
}
+
+ test("Fix caching catalog case sensitive") {
Review Comment:
oh, sorry for the missleading. It seems the cache future is disabled in tests.. I will modify the test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix caching catalog case sensitive [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #4043:
URL: https://github.com/apache/paimon/pull/4043#issuecomment-2309174517
> MacOS filesystem is case insensitive, so the FileSystemCatalog becomes insensitive implicitly.. anyway, I add one more option to choose if make caching catalog case insensitive.
I see, actually we had this option before, which was later changed to `allow-upper-case`. There's an odd history behind this: since Flink doesn't natively support case-insensitivity, we cannot store uppercase objects in a case-insensitive storage because if Flink requests an object in uppercase, but the storage returns it in lowercase due to its case-insensitive nature, Flink would fail with an error stating that the object cannot be found.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix caching catalog case sensitive [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #4043:
URL: https://github.com/apache/paimon/pull/4043#issuecomment-2309348987
> @JingsongLi thank you for the context, I see the option about allow_upper_case which has different default value for different backend engine and catalog, which makes me a bit confused before.. What do you think about this new option ? I think it at least can resovle some issues mannually and I see the iceberg also has a dedicated option for caching catalog.
If there is no applicable scenario at present, let's maintain the current status quo for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix caching catalog case sensitive [paimon]
Posted by "ulysses-you (via GitHub)" <gi...@apache.org>.
ulysses-you commented on PR #4043:
URL: https://github.com/apache/paimon/pull/4043#issuecomment-2306760821
MacOS filesystem is case insensitive, so the FileSystemCatalog becomes insensitive implicitly.. anyway, I add one more option to choose if make caching catalog case insensitive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Fix caching catalog case sensitive [paimon]
Posted by "ulysses-you (via GitHub)" <gi...@apache.org>.
ulysses-you closed pull request #4043: [core] Fix caching catalog case sensitive
URL: https://github.com/apache/paimon/pull/4043
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org