You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "zhu3pang (via GitHub)" <gi...@apache.org> on 2024/03/15 03:32:25 UTC
[PR] [PAIMON-1401] [Feature] Implement Purge table for SparkCatalog [incubator-paimon]
zhu3pang opened a new pull request, #3015:
URL: https://github.com/apache/incubator-paimon/pull/3015
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close #1401
<!-- What is the purpose of the change -->
Implement Purge table for SparkCatalog
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Spark] Implement Purge table for SparkCatalog [paimon]
Posted by "zhu3pang (via GitHub)" <gi...@apache.org>.
zhu3pang commented on code in PR #3015:
URL: https://github.com/apache/paimon/pull/3015#discussion_r1545899981
##########
paimon-spark/paimon-spark-3.1/src/test/java/org/apache/paimon/spark/SparkGenericCatalogTest.java:
##########
@@ -105,6 +107,42 @@ public void testCsvTable() {
.containsExactlyInAnyOrder("[1,2,3]", "[4,5,6]");
}
+ @Test
Review Comment:
already moved to DDLTestBase
##########
paimon-spark/paimon-spark-3.1/src/test/java/org/apache/paimon/spark/SparkGenericCatalogTest.java:
##########
@@ -105,6 +107,42 @@ public void testCsvTable() {
.containsExactlyInAnyOrder("[1,2,3]", "[4,5,6]");
}
+ @Test
Review Comment:
already moved to DDLTestBase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Spark] Implement Purge table for SparkCatalog [incubator-paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #3015:
URL: https://github.com/apache/incubator-paimon/pull/3015#discussion_r1528111622
##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -173,6 +173,12 @@ public List<String> listTables(String databaseName) throws DatabaseNotExistExcep
@Override
public void dropTable(Identifier identifier, boolean ignoreIfNotExists)
throws TableNotExistException {
+ dropTable(identifier, ignoreIfNotExists, false);
Review Comment:
ifPurge should be true.
Here should have some test to verify it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Spark] Implement Purge table for SparkCatalog [paimon]
Posted by "zhu3pang (via GitHub)" <gi...@apache.org>.
zhu3pang commented on code in PR #3015:
URL: https://github.com/apache/paimon/pull/3015#discussion_r1545900069
##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -173,6 +173,12 @@ public List<String> listTables(String databaseName) throws DatabaseNotExistExcep
@Override
public void dropTable(Identifier identifier, boolean ignoreIfNotExists)
throws TableNotExistException {
+ dropTable(identifier, ignoreIfNotExists, false);
Review Comment:
already add some tests for it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Spark] Implement Purge table for SparkCatalog [incubator-paimon]
Posted by "YannByron (via GitHub)" <gi...@apache.org>.
YannByron commented on code in PR #3015:
URL: https://github.com/apache/incubator-paimon/pull/3015#discussion_r1527828499
##########
paimon-spark/paimon-spark-3.1/src/test/java/org/apache/paimon/spark/SparkGenericCatalogTest.java:
##########
@@ -105,6 +107,42 @@ public void testCsvTable() {
.containsExactlyInAnyOrder("[1,2,3]", "[4,5,6]");
}
+ @Test
Review Comment:
And I prefer to add these UTs in `DDLTestBase`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Spark] Implement Purge table for SparkCatalog [incubator-paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #3015:
URL: https://github.com/apache/incubator-paimon/pull/3015#issuecomment-2002746646
CC @YannByron
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Spark] Implement Purge table for SparkCatalog [paimon]
Posted by "zhujing1-ola (via GitHub)" <gi...@apache.org>.
zhujing1-ola commented on code in PR #3015:
URL: https://github.com/apache/paimon/pull/3015#discussion_r1544427423
##########
paimon-spark/paimon-spark-3.1/src/test/java/org/apache/paimon/spark/SparkGenericCatalogTest.java:
##########
@@ -105,6 +107,42 @@ public void testCsvTable() {
.containsExactlyInAnyOrder("[1,2,3]", "[4,5,6]");
}
+ @Test
Review Comment:
already moved to DDLTestBase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Spark] Implement Purge table for SparkCatalog [paimon]
Posted by "zhujing1-ola (via GitHub)" <gi...@apache.org>.
zhujing1-ola commented on code in PR #3015:
URL: https://github.com/apache/paimon/pull/3015#discussion_r1544428886
##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -173,6 +173,12 @@ public List<String> listTables(String databaseName) throws DatabaseNotExistExcep
@Override
public void dropTable(Identifier identifier, boolean ignoreIfNotExists)
throws TableNotExistException {
+ dropTable(identifier, ignoreIfNotExists, false);
Review Comment:
already add some tests for it
##########
paimon-spark/paimon-spark-3.1/src/test/java/org/apache/paimon/spark/SparkGenericCatalogTest.java:
##########
@@ -105,6 +107,42 @@ public void testCsvTable() {
.containsExactlyInAnyOrder("[1,2,3]", "[4,5,6]");
}
+ @Test
Review Comment:
already moved to DDLTestBase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Spark] Implement Purge table for SparkCatalog [paimon]
Posted by "zhujing1-ola (via GitHub)" <gi...@apache.org>.
zhujing1-ola commented on code in PR #3015:
URL: https://github.com/apache/paimon/pull/3015#discussion_r1544428886
##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -173,6 +173,12 @@ public List<String> listTables(String databaseName) throws DatabaseNotExistExcep
@Override
public void dropTable(Identifier identifier, boolean ignoreIfNotExists)
throws TableNotExistException {
+ dropTable(identifier, ignoreIfNotExists, false);
Review Comment:
already add some tests for it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Spark] Implement Purge table for SparkCatalog [incubator-paimon]
Posted by "YannByron (via GitHub)" <gi...@apache.org>.
YannByron commented on code in PR #3015:
URL: https://github.com/apache/incubator-paimon/pull/3015#discussion_r1527827190
##########
paimon-spark/paimon-spark-3.1/src/test/java/org/apache/paimon/spark/SparkGenericCatalogTest.java:
##########
@@ -105,6 +107,42 @@ public void testCsvTable() {
.containsExactlyInAnyOrder("[1,2,3]", "[4,5,6]");
}
+ @Test
Review Comment:
I think this UT should can work well in all spark (version >= 3.1). Better to move it to `spark-common`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Spark] Implement Purge table for SparkCatalog [paimon]
Posted by "zhujing1-ola (via GitHub)" <gi...@apache.org>.
zhujing1-ola commented on code in PR #3015:
URL: https://github.com/apache/paimon/pull/3015#discussion_r1544427423
##########
paimon-spark/paimon-spark-3.1/src/test/java/org/apache/paimon/spark/SparkGenericCatalogTest.java:
##########
@@ -105,6 +107,42 @@ public void testCsvTable() {
.containsExactlyInAnyOrder("[1,2,3]", "[4,5,6]");
}
+ @Test
Review Comment:
already moved to DDLTestBase
##########
paimon-spark/paimon-spark-3.1/src/test/java/org/apache/paimon/spark/SparkGenericCatalogTest.java:
##########
@@ -105,6 +107,42 @@ public void testCsvTable() {
.containsExactlyInAnyOrder("[1,2,3]", "[4,5,6]");
}
+ @Test
Review Comment:
already moved to DDLTestBase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org