You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "yabola (via GitHub)" <gi...@apache.org> on 2026/02/13 09:49:12 UTC

[PR] Support SupportsPartitionManagement for HiveTable [kyuubi]

yabola opened a new pull request, #7329:
URL: https://github.com/apache/kyuubi/pull/7329

   <!--
   Thanks for sending a pull request!
   
   Here are some tips for you:
     1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/master/contributing/code/index.html
     2. If the PR is related to an issue in https://github.com/apache/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
     3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
   -->
   
   ### Why are the changes needed?
   <!--
   Please clarify why the changes are needed. For instance,
     1. If you propose a new API, clarify the use case for a new API.
     2. If you fix a bug, you can clarify why it is a bug, and what versions are affected.
   -->
   The Hive Connector V2 needs to fully support partition management operations.
   
   ### How was this patch tested?
   <!--
   If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
   If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
   If tests were not added, please describe why they were not added and/or why it was difficult to add.
   -->
   Add new UT
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#discussion_r2902230532


##########
extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/HiveConnectorUtils.scala:
##########
@@ -28,19 +28,37 @@ import org.apache.spark.internal.Logging
 import org.apache.spark.sql.SparkSession
 import org.apache.spark.sql.catalyst.{InternalRow, TableIdentifier}
 import org.apache.spark.sql.catalyst.catalog.{CatalogTable, CatalogTablePartition}
+import org.apache.spark.sql.catalyst.expressions.{Cast, Expression}
 import org.apache.spark.sql.connector.catalog.TableChange
 import org.apache.spark.sql.connector.catalog.TableChange._
 import org.apache.spark.sql.execution.command.CommandUtils
 import org.apache.spark.sql.execution.datasources.{PartitionDirectory, PartitionedFile}
 import org.apache.spark.sql.hive.execution.HiveFileFormat
 import org.apache.spark.sql.internal.SQLConf
-import org.apache.spark.sql.types.{ArrayType, MapType, StructField, StructType}
+import org.apache.spark.sql.types.{ArrayType, DataType, MapType, StructField, StructType}
 
 import org.apache.kyuubi.util.reflect.{DynClasses, DynConstructors, DynMethods}
 import org.apache.kyuubi.util.reflect.ReflectUtils.invokeAs
 
 object HiveConnectorUtils extends Logging {
 
+  private val castCtor: DynConstructors.Ctor[Expression] =
+    DynConstructors.builder(classOf[Expression])
+      .impl(
+        classOf[Cast],
+        classOf[Expression],
+        classOf[DataType],
+        classOf[Option[_]])
+      .build[Expression]()
+
+  // SPARK-40054, ensuring cross-version compatibility.

Review Comment:
   add fixed version alongside the JIRA ticket id.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#issuecomment-4019723971

   UT does not cover data types other than STRING, could you expand that?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "github-actions[bot] (via GitHub) via notifications" <no...@kyuubi.apache.org>.
github-actions[bot] closed pull request #7329: Support partition management for HiveTable
URL: https://github.com/apache/kyuubi/pull/7329


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "yabola (via GitHub)" <gi...@apache.org>.
yabola commented on code in PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#discussion_r3227039106


##########
extensions/spark/kyuubi-spark-connector-hive/src/test/scala/org/apache/kyuubi/spark/connector/hive/command/DDLCommandTestUtils.scala:
##########
@@ -35,6 +35,8 @@ trait DDLCommandTestUtils extends KyuubiHiveTest {
   // Name of the command as SQL statement, for instance "SHOW PARTITIONS"
   protected def command: String
 
+  protected def catalogName: String = "hive"

Review Comment:
   Strangely, when I ran the tests myself, I had to change catalogName from a val to a def to make V1 use the spark_catalog name. Otherwise, it always resolved to hive.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "yabola (via GitHub)" <gi...@apache.org>.
yabola commented on PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#issuecomment-4047022403

   @pan3793 pan Thank you for the review, I'll address these comments soon.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "yabola (via GitHub)" <gi...@apache.org>.
yabola commented on PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#issuecomment-3982348539

   @pan3793 @cxzl25 Please help review when you have time, thanks~


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#discussion_r2902229691


##########
extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/HiveTable.scala:
##########
@@ -112,4 +116,92 @@ case class HiveTable(
   override def capabilities(): util.Set[TableCapability] = {
     util.EnumSet.of(BATCH_READ, BATCH_WRITE, OVERWRITE_BY_FILTER, OVERWRITE_DYNAMIC)
   }
+
+  override def createPartition(ident: InternalRow, properties: util.Map[String, String]): Unit = {
+    val spec = toPartitionSpec(ident)
+    val location = Option(properties.get(HiveTableProperties.LOCATION)).map(new URI(_))
+    val newPart = CatalogTablePartition(
+      spec,
+      catalogTable.storage.copy(locationUri = location),
+      properties.asScala.toMap)
+    hiveTableCatalog.externalCatalog.createPartitions(
+      catalogTable.database,
+      catalogTable.identifier.table,
+      Seq(newPart),
+      ignoreIfExists = false)
+  }
+
+  override def dropPartition(ident: InternalRow): Boolean = {
+    try {
+      hiveTableCatalog.externalCatalog.dropPartitions(
+        catalogTable.database,
+        catalogTable.identifier.table,
+        Seq(toPartitionSpec(ident)),
+        ignoreIfNotExists = false,
+        purge = false,
+        retainData = false)
+      true
+    } catch {
+      case _: NoSuchPartitionException => false
+    }
+  }
+
+  override def replacePartitionMetadata(
+      ident: InternalRow,
+      properties: util.Map[String, String]): Unit = {
+    throw new UnsupportedOperationException("Replace partition is not supported")

Review Comment:
   ```suggestion
       throw new UnsupportedOperationException("Replace partition metadata is not supported")
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "yabola (via GitHub)" <gi...@apache.org>.
yabola commented on code in PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#discussion_r3226990927


##########
extensions/spark/kyuubi-spark-connector-hive/src/test/scala/org/apache/kyuubi/spark/connector/hive/command/PartitionManagementSuite.scala:
##########
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.spark.connector.hive.command
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.analysis.{NoSuchPartitionsException, PartitionsAlreadyExistException}
+import org.apache.spark.sql.connector.catalog.{Identifier, SupportsPartitionManagement, TableCatalog}
+import org.apache.spark.unsafe.types.UTF8String
+
+import org.apache.kyuubi.spark.connector.hive.command.DDLCommandTestUtils.{V1_COMMAND_VERSION, V2_COMMAND_VERSION}
+
+trait PartitionManagementSuite extends DDLCommandTestUtils {
+  override protected def command: String = "PARTITION MANAGEMENT"
+
+  test("create partition") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(s"CREATE TABLE $t (id string, year string, month string) PARTITIONED BY (year, month)")
+      sql(s"ALTER TABLE $t ADD PARTITION (year='2023', month='01')")
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t"),
+        Row("year=2023/month=01") :: Nil)
+      intercept[PartitionsAlreadyExistException] {
+        sql(s"ALTER TABLE $t ADD PARTITION (year='2023', month='01')")
+      }
+    }
+  }
+
+  test("drop partition") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(s"CREATE TABLE $t (id string, year string, month string) PARTITIONED BY (year, month)")
+      sql(s"ALTER TABLE $t ADD PARTITION (year='2023', month='01')")
+      sql(s"ALTER TABLE $t DROP PARTITION (year='2023', month='01')")
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t"),
+        Nil)
+      intercept[NoSuchPartitionsException] {
+        sql(s"ALTER TABLE $t DROP PARTITION (year='9999', month='99')")
+      }
+    }
+  }
+
+  test("show partitions") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(s"CREATE TABLE $t (id string, year string, month string) PARTITIONED BY (year, month)")
+      sql(s"ALTER TABLE $t ADD PARTITION (year='2023', month='01')")
+      sql(s"ALTER TABLE $t ADD PARTITION (year='2023', month='02')")
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t"),
+        Row("year=2023/month=01") :: Row("year=2023/month=02") :: Nil)
+
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t PARTITION (year='2023', month='01')"),
+        Row("year=2023/month=01") :: Nil)
+
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t PARTITION (year='2023')"),
+        Row("year=2023/month=01") :: Row("year=2023/month=02") :: Nil)
+    }
+  }
+}
+
+class PartitionManagementV2Suite extends PartitionManagementSuite {
+  override protected def catalogVersion: String = "Hive V2"
+  override protected def commandVersion: String = V2_COMMAND_VERSION
+
+  test("create partition with location") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(s"CREATE TABLE $t (id string, year string, month string) PARTITIONED BY (year, month)")
+      val loc = "file:///tmp/kyuubi/hive_catalog_part_loc"

Review Comment:
   add `withTempDir` function 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "yabola (via GitHub)" <gi...@apache.org>.
yabola commented on code in PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#discussion_r3227039106


##########
extensions/spark/kyuubi-spark-connector-hive/src/test/scala/org/apache/kyuubi/spark/connector/hive/command/DDLCommandTestUtils.scala:
##########
@@ -35,6 +35,8 @@ trait DDLCommandTestUtils extends KyuubiHiveTest {
   // Name of the command as SQL statement, for instance "SHOW PARTITIONS"
   protected def command: String
 
+  protected def catalogName: String = "hive"

Review Comment:
   When running the tests, I had to change catalogName from a val to a def so that the V1 and V2 suites can override it with different catalog names (spark_catalog for V1 and hive for V2). Otherwise the overridden value never takes effect and the catalog is always resolved to hive.



##########
extensions/spark/kyuubi-spark-connector-hive/src/test/scala/org/apache/kyuubi/spark/connector/hive/command/PartitionManagementSuite.scala:
##########
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.spark.connector.hive.command
+
+import java.io.File
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.analysis.{NoSuchPartitionsException, PartitionsAlreadyExistException}
+import org.apache.spark.sql.connector.catalog.{Identifier, SupportsPartitionManagement, TableCatalog}
+import org.apache.spark.unsafe.types.UTF8String
+
+import org.apache.kyuubi.spark.connector.hive.command.DDLCommandTestUtils.{V1_COMMAND_VERSION, V2_COMMAND_VERSION}
+
+trait PartitionManagementSuite extends DDLCommandTestUtils {
+  override protected def command: String = "PARTITION MANAGEMENT"
+
+  test("drop partition") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(
+        s"""CREATE TABLE $t (id string, name string, year int, dt date)
+           |PARTITIONED BY (name, year, dt)""".stripMargin)
+      sql(s"ALTER TABLE $t ADD PARTITION (name='a', year=2023, dt='2023-01-01')")
+      sql(s"ALTER TABLE $t DROP PARTITION (name='a', year=2023, dt='2023-01-01')")
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t"),
+        Nil)
+      intercept[NoSuchPartitionsException] {
+        sql(s"ALTER TABLE $t DROP PARTITION (name='a', year=9999, dt='9999-12-31')")
+      }
+    }
+  }
+
+  test("show partitions") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(
+        s"""CREATE TABLE $t (id string, name string, year int, dt date)
+           |PARTITIONED BY (name, year, dt)""".stripMargin)
+      sql(s"ALTER TABLE $t ADD PARTITION (name='a', year=2023, dt='2023-01-01')")
+      sql(s"ALTER TABLE $t ADD PARTITION (name='a', year=2023, dt='2023-02-01')")
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t"),
+        Row("name=a/year=2023/dt=2023-01-01") ::
+          Row("name=a/year=2023/dt=2023-02-01") :: Nil)
+
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t PARTITION (name='a', year=2023, dt='2023-01-01')"),
+        Row("name=a/year=2023/dt=2023-01-01") :: Nil)
+
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t PARTITION (name='a', year=2023)"),
+        Row("name=a/year=2023/dt=2023-01-01") ::
+          Row("name=a/year=2023/dt=2023-02-01") :: Nil)
+    }
+  }
+}
+
+class PartitionManagementV2Suite extends PartitionManagementSuite {
+  override protected def catalogVersion: String = "Hive V2"
+  override protected def commandVersion: String = V2_COMMAND_VERSION
+
+  test("create partition") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(
+        s"""CREATE TABLE $t (id string, name string, year int, dt date)
+           |PARTITIONED BY (name, year, dt)""".stripMargin)
+      sql(s"ALTER TABLE $t ADD PARTITION (name='a', year=2023, dt='2023-01-01')")
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t"),
+        Row("name=a/year=2023/dt=2023-01-01") :: Nil)
+      intercept[PartitionsAlreadyExistException] {
+        sql(s"ALTER TABLE $t ADD PARTITION (name='a', year=2023, dt='2023-01-01')")
+      }
+      sql(s"INSERT INTO $t PARTITION (name=null, year=null, dt=null) VALUES ('1')")
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t"),
+        Row("name=a/year=2023/dt=2023-01-01") ::
+          Row("name=null/year=null/dt=null") :: Nil)

Review Comment:
   I made a small behavior change here: for null partition values, we now display null instead of __HIVE_DEFAULT_PARTITION__. This is consistent with Iceberg's behavior, but differs from DataSource V1. Please take a look and let me know if this is acceptable.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#discussion_r2902224740


##########
extensions/spark/kyuubi-spark-connector-hive/src/test/scala/org/apache/kyuubi/spark/connector/hive/command/PartitionManagementSuite.scala:
##########
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.spark.connector.hive.command
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.analysis.{NoSuchPartitionsException, PartitionsAlreadyExistException}
+import org.apache.spark.sql.connector.catalog.{Identifier, SupportsPartitionManagement, TableCatalog}
+import org.apache.spark.unsafe.types.UTF8String
+
+import org.apache.kyuubi.spark.connector.hive.command.DDLCommandTestUtils.{V1_COMMAND_VERSION, V2_COMMAND_VERSION}
+
+trait PartitionManagementSuite extends DDLCommandTestUtils {
+  override protected def command: String = "PARTITION MANAGEMENT"
+
+  test("create partition") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(s"CREATE TABLE $t (id string, year string, month string) PARTITIONED BY (year, month)")
+      sql(s"ALTER TABLE $t ADD PARTITION (year='2023', month='01')")
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t"),
+        Row("year=2023/month=01") :: Nil)
+      intercept[PartitionsAlreadyExistException] {
+        sql(s"ALTER TABLE $t ADD PARTITION (year='2023', month='01')")
+      }
+    }
+  }
+
+  test("drop partition") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(s"CREATE TABLE $t (id string, year string, month string) PARTITIONED BY (year, month)")
+      sql(s"ALTER TABLE $t ADD PARTITION (year='2023', month='01')")
+      sql(s"ALTER TABLE $t DROP PARTITION (year='2023', month='01')")
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t"),
+        Nil)
+      intercept[NoSuchPartitionsException] {
+        sql(s"ALTER TABLE $t DROP PARTITION (year='9999', month='99')")
+      }
+    }
+  }
+
+  test("show partitions") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(s"CREATE TABLE $t (id string, year string, month string) PARTITIONED BY (year, month)")
+      sql(s"ALTER TABLE $t ADD PARTITION (year='2023', month='01')")
+      sql(s"ALTER TABLE $t ADD PARTITION (year='2023', month='02')")
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t"),
+        Row("year=2023/month=01") :: Row("year=2023/month=02") :: Nil)
+
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t PARTITION (year='2023', month='01')"),
+        Row("year=2023/month=01") :: Nil)
+
+      checkAnswer(
+        sql(s"SHOW PARTITIONS $t PARTITION (year='2023')"),
+        Row("year=2023/month=01") :: Row("year=2023/month=02") :: Nil)
+    }
+  }
+}
+
+class PartitionManagementV2Suite extends PartitionManagementSuite {
+  override protected def catalogVersion: String = "Hive V2"
+  override protected def commandVersion: String = V2_COMMAND_VERSION
+
+  test("create partition with location") {
+    withNamespaceAndTable("ns", "tbl") { t =>
+      sql(s"CREATE TABLE $t (id string, year string, month string) PARTITIONED BY (year, month)")
+      val loc = "file:///tmp/kyuubi/hive_catalog_part_loc"

Review Comment:
   create a temp dir instead of relying on global `/tmp`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "yabola (via GitHub)" <gi...@apache.org>.
yabola commented on PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#issuecomment-4789596706

   @pan3793 Hi~ please help review this when you have time~


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "github-actions[bot] (via GitHub) via notifications" <no...@kyuubi.apache.org>.
github-actions[bot] commented on PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#issuecomment-6536441440

   Thanks for the PR! This PR is being closed due to inactivity. This isn't a judgement on the merit of the PR in any way. If this is still an issue with the latest version of Kyuubi, please reopen it and ask a committer to remove the Stale tag!
   
   Thank you for using Kyuubi!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#issuecomment-4019716517

   please test `null` partition value cases


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support SupportsPartitionManagement for HiveTable [kyuubi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#issuecomment-3901374900

   ## [Codecov](https://app.codecov.io/gh/apache/kyuubi/pull/7329?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `60 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 0.00%. Comparing base ([`2037881`](https://app.codecov.io/gh/apache/kyuubi/commit/2037881ff7c2a32efdcc22bca3e427eb5c0780b1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`60ef239`](https://app.codecov.io/gh/apache/kyuubi/commit/60ef23966146e3fea95586cecc14159c5faad8cb?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 2 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/kyuubi/pull/7329?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [...apache/kyuubi/spark/connector/hive/HiveTable.scala](https://app.codecov.io/gh/apache/kyuubi/pull/7329?src=pr&el=tree&filepath=extensions%2Fspark%2Fkyuubi-spark-connector-hive%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fkyuubi%2Fspark%2Fconnector%2Fhive%2FHiveTable.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZXh0ZW5zaW9ucy9zcGFyay9reXV1Ymktc3BhcmstY29ubmVjdG9yLWhpdmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1Ymkvc3BhcmsvY29ubmVjdG9yL2hpdmUvSGl2ZVRhYmxlLnNjYWxh) | 0.00% | [57 Missing :warning: ](https://app.codecov.io/gh/apache/kyuubi/pull/7329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...ubi/spark/connector/hive/HiveTableProperties.scala](https://app.codecov.io/gh/apache/kyuubi/pull/7329?src=pr&el=tree&filepath=extensions%2Fspark%2Fkyuubi-spark-connector-hive%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fkyuubi%2Fspark%2Fconnector%2Fhive%2FHiveTableProperties.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZXh0ZW5zaW9ucy9zcGFyay9reXV1Ymktc3BhcmstY29ubmVjdG9yLWhpdmUvc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1Ymkvc3BhcmsvY29ubmVjdG9yL2hpdmUvSGl2ZVRhYmxlUHJvcGVydGllcy5zY2FsYQ==) | 0.00% | [3 Missing :warning: ](https://app.codecov.io/gh/apache/kyuubi/pull/7329?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@          Coverage Diff           @@
   ##           master   #7329   +/-   ##
   ======================================
     Coverage    0.00%   0.00%           
   ======================================
     Files         698     699    +1     
     Lines       43656   43716   +60     
     Branches     5896    5897    +1     
   ======================================
   - Misses      43656   43716   +60     
   ```
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/kyuubi/pull/7329?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [PR] Support partition management for HiveTable [kyuubi]

Posted by "yabola (via GitHub)" <gi...@apache.org>.
yabola commented on PR #7329:
URL: https://github.com/apache/kyuubi/pull/7329#issuecomment-4431235028

   @pan3793 Very sorry for the late update. Was busy with another project. Comments are now addressed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org