You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/26 03:05:10 UTC
[GitHub] [spark] AngersZhuuuu opened a new pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
AngersZhuuuu opened a new pull request #34388:
URL: https://github.com/apache/spark/pull/34388
### What changes were proposed in this pull request?
In this pr we use shim to wrap all common hive client call apis to make all calls are controlled by shim.
### Why are the changes needed?
Wrap all hive client calls to make it better collect related metrics
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existed UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951962494
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49091/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951835009
**[Test build #144619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144619/testReport)** for PR 34388 at commit [`889b0b5`](https://github.com/apache/spark/commit/889b0b59141398f6ddfea02fd6f0b6d0dba5e7cc).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951667572
**[Test build #144617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144617/testReport)** for PR 34388 at commit [`7013bde`](https://github.com/apache/spark/commit/7013bdee17576166e30141f714d250831db845a7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952542439
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49110/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on a change in pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
sunchao commented on a change in pull request #34388:
URL: https://github.com/apache/spark/pull/34388#discussion_r736899783
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -484,6 +549,111 @@ private[client] class Shim_v0_12 extends Shim with Logging {
override def getDatabaseOwnerName(db: Database): String = ""
override def setDatabaseOwnerName(db: Database, owner: String): Unit = {}
+
+ override def createDatabase(hive: Hive, db: Database, ignoreIfExists: Boolean): Unit = {
+ hive.createDatabase(db, ignoreIfExists)
+ }
+
+ override def dropDatabase(
+ hive: Hive,
+ name: String,
+ deleteData: Boolean,
+ ignoreUnknownDb: Boolean,
+ cascade: Boolean): Unit = {
+ hive.dropDatabase(name, deleteData, ignoreUnknownDb, cascade)
+ }
+
+ override def alterDatabase(hive: Hive, dbName: String, d: Database): Unit = {
+ hive.alterDatabase(dbName, d)
+ }
+
+ override def getDatabase(hive: Hive, dbName: String): Database = {
+ hive.getDatabase(dbName)
+ }
+
+ override def getAllDatabases(hive: Hive): JList[String] = {
+ hive.getAllDatabases
+ }
+
+ override def getDatabasesByPattern(hive: Hive, pattern: String): JList[String] = {
Review comment:
maybe it's better to return `Seq[String]`?
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -484,6 +549,111 @@ private[client] class Shim_v0_12 extends Shim with Logging {
override def getDatabaseOwnerName(db: Database): String = ""
override def setDatabaseOwnerName(db: Database, owner: String): Unit = {}
+
+ override def createDatabase(hive: Hive, db: Database, ignoreIfExists: Boolean): Unit = {
+ hive.createDatabase(db, ignoreIfExists)
+ }
+
+ override def dropDatabase(
+ hive: Hive,
+ name: String,
+ deleteData: Boolean,
+ ignoreUnknownDb: Boolean,
+ cascade: Boolean): Unit = {
+ hive.dropDatabase(name, deleteData, ignoreUnknownDb, cascade)
+ }
+
+ override def alterDatabase(hive: Hive, dbName: String, d: Database): Unit = {
+ hive.alterDatabase(dbName, d)
+ }
+
+ override def getDatabase(hive: Hive, dbName: String): Database = {
+ hive.getDatabase(dbName)
+ }
+
+ override def getAllDatabases(hive: Hive): JList[String] = {
+ hive.getAllDatabases
+ }
+
+ override def getDatabasesByPattern(hive: Hive, pattern: String): JList[String] = {
+ hive.getDatabasesByPattern(pattern)
+ }
+
+ override def databaseExists(hive: Hive, dbName: String): Boolean = {
+ hive.databaseExists(dbName)
+ }
+
+ override def createTable(hive: Hive, table: Table, ifNotExists: Boolean): Unit = {
+ hive.createTable(table, ifNotExists)
+ }
+
+ override def getTable(
+ hive: Hive,
+ dbName: String,
+ tableName: String,
+ throwException: Boolean): Table = {
+ hive.getTable(dbName, tableName, throwException)
+ }
+
+ override def getTablesByPattern(hive: Hive, dbName: String, pattern: String): JList[String] = {
+ hive.getTablesByPattern(dbName, pattern)
+ }
+
+ override def getAllTables(hive: Hive, dbName: String): JList[String] = {
+ hive.getAllTables(dbName)
+ }
+
+ override def dropTable(hive: Hive, dbName: String, tableName: String): Unit = {
+ hive.dropTable(dbName, tableName)
+ }
+
+ override def getPartition(
+ hive: Hive,
+ table: Table,
+ partSpec: JMap[String, String],
+ forceCreate: Boolean): Partition = {
+ hive.getPartition(table, partSpec, forceCreate)
+ }
+
+ override def getPartitions(
+ hive: Hive,
+ table: Table,
+ partSpec: JMap[String, String]): JList[Partition] = {
+ hive.getPartitions(table, partSpec)
+ }
+
+ override def getPartitionNames(
+ hive: Hive,
+ dbName: String,
+ tblName: String,
+ max: Short): JList[String] = {
+ hive.getPartitionNames(dbName, tblName, max)
+ }
+
+ override def getPartitionNames(
+ hive: Hive,
+ dbName: String,
+ tblName: String,
+ partSpec: JMap[String, String],
+ max: Short): JList[String] = {
+ hive.getPartitionNames(dbName, tblName, partSpec, max)
+ }
+
+ override def renamePartition(
+ hive: Hive,
+ tbl: Table,
Review comment:
nit: maybe rename this to `table` to be consistent
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -484,6 +549,111 @@ private[client] class Shim_v0_12 extends Shim with Logging {
override def getDatabaseOwnerName(db: Database): String = ""
override def setDatabaseOwnerName(db: Database, owner: String): Unit = {}
+
+ override def createDatabase(hive: Hive, db: Database, ignoreIfExists: Boolean): Unit = {
+ hive.createDatabase(db, ignoreIfExists)
+ }
+
+ override def dropDatabase(
+ hive: Hive,
+ name: String,
Review comment:
nit: `dbName`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951924921
**[Test build #144621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144621/testReport)** for PR 34388 at commit [`a668f50`](https://github.com/apache/spark/commit/a668f504d82340229f6f1cc34e6bf79096cd18a1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952032756
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49091/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952497190
**[Test build #144640 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144640/testReport)** for PR 34388 at commit [`a888ae7`](https://github.com/apache/spark/commit/a888ae7597262cb384f51611aa63fbba79db3337).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952546597
**[Test build #144640 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144640/testReport)** for PR 34388 at commit [`a888ae7`](https://github.com/apache/spark/commit/a888ae7597262cb384f51611aa63fbba79db3337).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951562783
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951798883
**[Test build #144617 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144617/testReport)** for PR 34388 at commit [`7013bde`](https://github.com/apache/spark/commit/7013bdee17576166e30141f714d250831db845a7).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951560545
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49078/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952032756
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49091/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952547318
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144640/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951780056
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49089/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34388:
URL: https://github.com/apache/spark/pull/34388#discussion_r737059752
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -484,6 +549,111 @@ private[client] class Shim_v0_12 extends Shim with Logging {
override def getDatabaseOwnerName(db: Database): String = ""
override def setDatabaseOwnerName(db: Database, owner: String): Unit = {}
+
+ override def createDatabase(hive: Hive, db: Database, ignoreIfExists: Boolean): Unit = {
+ hive.createDatabase(db, ignoreIfExists)
+ }
+
+ override def dropDatabase(
+ hive: Hive,
+ name: String,
+ deleteData: Boolean,
+ ignoreUnknownDb: Boolean,
+ cascade: Boolean): Unit = {
+ hive.dropDatabase(name, deleteData, ignoreUnknownDb, cascade)
+ }
+
+ override def alterDatabase(hive: Hive, dbName: String, d: Database): Unit = {
+ hive.alterDatabase(dbName, d)
+ }
+
+ override def getDatabase(hive: Hive, dbName: String): Database = {
+ hive.getDatabase(dbName)
+ }
+
+ override def getAllDatabases(hive: Hive): JList[String] = {
+ hive.getAllDatabases
+ }
+
+ override def getDatabasesByPattern(hive: Hive, pattern: String): JList[String] = {
+ hive.getDatabasesByPattern(pattern)
+ }
+
+ override def databaseExists(hive: Hive, dbName: String): Boolean = {
+ hive.databaseExists(dbName)
+ }
+
+ override def createTable(hive: Hive, table: Table, ifNotExists: Boolean): Unit = {
+ hive.createTable(table, ifNotExists)
+ }
+
+ override def getTable(
+ hive: Hive,
+ dbName: String,
+ tableName: String,
+ throwException: Boolean): Table = {
+ hive.getTable(dbName, tableName, throwException)
+ }
+
+ override def getTablesByPattern(hive: Hive, dbName: String, pattern: String): JList[String] = {
+ hive.getTablesByPattern(dbName, pattern)
+ }
+
+ override def getAllTables(hive: Hive, dbName: String): JList[String] = {
+ hive.getAllTables(dbName)
+ }
+
+ override def dropTable(hive: Hive, dbName: String, tableName: String): Unit = {
+ hive.dropTable(dbName, tableName)
+ }
+
+ override def getPartition(
+ hive: Hive,
+ table: Table,
+ partSpec: JMap[String, String],
+ forceCreate: Boolean): Partition = {
+ hive.getPartition(table, partSpec, forceCreate)
+ }
+
+ override def getPartitions(
+ hive: Hive,
+ table: Table,
+ partSpec: JMap[String, String]): JList[Partition] = {
+ hive.getPartitions(table, partSpec)
+ }
+
+ override def getPartitionNames(
+ hive: Hive,
+ dbName: String,
+ tblName: String,
+ max: Short): JList[String] = {
+ hive.getPartitionNames(dbName, tblName, max)
+ }
+
+ override def getPartitionNames(
+ hive: Hive,
+ dbName: String,
+ tblName: String,
+ partSpec: JMap[String, String],
+ max: Short): JList[String] = {
+ hive.getPartitionNames(dbName, tblName, partSpec, max)
+ }
+
+ override def renamePartition(
+ hive: Hive,
+ tbl: Table,
Review comment:
done of all such case
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -484,6 +549,111 @@ private[client] class Shim_v0_12 extends Shim with Logging {
override def getDatabaseOwnerName(db: Database): String = ""
override def setDatabaseOwnerName(db: Database, owner: String): Unit = {}
+
+ override def createDatabase(hive: Hive, db: Database, ignoreIfExists: Boolean): Unit = {
+ hive.createDatabase(db, ignoreIfExists)
+ }
+
+ override def dropDatabase(
+ hive: Hive,
+ name: String,
+ deleteData: Boolean,
+ ignoreUnknownDb: Boolean,
+ cascade: Boolean): Unit = {
+ hive.dropDatabase(name, deleteData, ignoreUnknownDb, cascade)
+ }
+
+ override def alterDatabase(hive: Hive, dbName: String, d: Database): Unit = {
+ hive.alterDatabase(dbName, d)
+ }
+
+ override def getDatabase(hive: Hive, dbName: String): Database = {
+ hive.getDatabase(dbName)
+ }
+
+ override def getAllDatabases(hive: Hive): JList[String] = {
+ hive.getAllDatabases
+ }
+
+ override def getDatabasesByPattern(hive: Hive, pattern: String): JList[String] = {
Review comment:
Yea, done for all such case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951551119
**[Test build #144607 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144607/testReport)** for PR 34388 at commit [`2ad345b`](https://github.com/apache/spark/commit/2ad345bc0bcda1f8a5dd69b68972759bd5bd9855).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951836254
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144619/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951516194
**[Test build #144607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144607/testReport)** for PR 34388 at commit [`2ad345b`](https://github.com/apache/spark/commit/2ad345bc0bcda1f8a5dd69b68972759bd5bd9855).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34388:
URL: https://github.com/apache/spark/pull/34388#discussion_r736520113
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -484,6 +549,111 @@ private[client] class Shim_v0_12 extends Shim with Logging {
override def getDatabaseOwnerName(db: Database): String = ""
override def setDatabaseOwnerName(db: Database, owner: String): Unit = {}
+
+ override def createDatabase(hive: Hive, db: Database, ignoreIfExists: Boolean): Unit = {
+ hive.createDatabase(db, ignoreIfExists)
+ }
+
+ override def dropDatabase(
+ hive: Hive,
+ name: String,
+ deleteData: Boolean,
+ ignoreUnknownDb: Boolean,
+ cascade: Boolean): Unit = {
+ hive.dropDatabase(name, deleteData, ignoreUnknownDb, cascade)
+ }
+
+ override def alterDatabase(hive: Hive, dbName: String, d: Database): Unit = {
+ hive.alterDatabase(dbName, d)
+ }
+
+ override def getDatabase(hive: Hive, dbName: String): Database = {
+ hive.getDatabase(dbName)
+ }
+
+ override def getAllDatabases(hive: Hive): JList[String] = {
+ hive.getAllDatabases
+ }
+
+ override def getDatabasesByPattern(hive: Hive, pattern: String): JList[String] = {
+ hive.getDatabasesByPattern(pattern)
+ }
+
+ override def databaseExists(hive: Hive, dbName: String): Boolean = {
+ hive.databaseExists(dbName)
+ }
+
+ override def createTable(hive: Hive, table: Table, ifNotExists: Boolean): Unit = {
+ hive.createTable(table, ifNotExists)
+ }
+
+ override def getTable(
+ hive: Hive,
+ dbName: String,
+ tableName: String,
+ throwException: Boolean): Table = {
+ hive.getTable(dbName, tableName, throwException)
+ }
+
+ override def getTablesByPattern(hive: Hive, dbName: String, pattern: String): JList[String] = {
+ hive.getTablesByPattern(dbName, pattern)
+ }
+
+ override def getAllTables(hive: Hive, dbName: String): JList[String] = {
+ hive.getAllTables(dbName)
Review comment:
Done
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -97,12 +116,50 @@ private[client] sealed abstract class Shim {
def alterPartitions(hive: Hive, tableName: String, newParts: JList[Partition]): Unit
+ def createTable(hive: Hive, table: Table, ifNotExists: Boolean): Unit
+
+ def getTable(
+ hive: Hive,
+ dbName: String,
+ tableName: String,
+ throwException: Boolean = true): Table
+
def getTablesByType(
hive: Hive,
dbName: String,
pattern: String,
tableType: TableType): Seq[String]
+ def getTablesByPattern(hive: Hive, dbName: String, pattern: String): JList[String]
+
+ def getAllTables(hive: Hive, dbName: String): JList[String]
+
+ def dropTable(hive: Hive, dbName: String, tableName: String): Unit
+
+ def getPartition(
+ hive: Hive,
+ table: Table,
+ partSpec: JMap[String, String],
+ forceCreate: Boolean): Partition
+
+ def getPartitions(
+ hive: Hive,
+ table: Table,
+ partSpec: JMap[String, String]): JList[Partition]
+
+ def getPartitionNames(
+ hive: Hive,
+ dbName: String,
+ tblName: String,
+ max: Short): JList[String]
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951781223
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49087/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951775563
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49087/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34388:
URL: https://github.com/apache/spark/pull/34388#discussion_r736515709
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -484,6 +549,111 @@ private[client] class Shim_v0_12 extends Shim with Logging {
override def getDatabaseOwnerName(db: Database): String = ""
override def setDatabaseOwnerName(db: Database, owner: String): Unit = {}
+
+ override def createDatabase(hive: Hive, db: Database, ignoreIfExists: Boolean): Unit = {
+ hive.createDatabase(db, ignoreIfExists)
+ }
+
+ override def dropDatabase(
+ hive: Hive,
+ name: String,
+ deleteData: Boolean,
+ ignoreUnknownDb: Boolean,
+ cascade: Boolean): Unit = {
+ hive.dropDatabase(name, deleteData, ignoreUnknownDb, cascade)
+ }
+
+ override def alterDatabase(hive: Hive, dbName: String, d: Database): Unit = {
+ hive.alterDatabase(dbName, d)
+ }
+
+ override def getDatabase(hive: Hive, dbName: String): Database = {
+ hive.getDatabase(dbName)
+ }
+
+ override def getAllDatabases(hive: Hive): JList[String] = {
+ hive.getAllDatabases
+ }
+
+ override def getDatabasesByPattern(hive: Hive, pattern: String): JList[String] = {
+ hive.getDatabasesByPattern(pattern)
+ }
+
+ override def databaseExists(hive: Hive, dbName: String): Boolean = {
+ hive.databaseExists(dbName)
+ }
+
+ override def createTable(hive: Hive, table: Table, ifNotExists: Boolean): Unit = {
+ hive.createTable(table, ifNotExists)
+ }
+
+ override def getTable(
+ hive: Hive,
+ dbName: String,
+ tableName: String,
+ throwException: Boolean): Table = {
+ hive.getTable(dbName, tableName, throwException)
+ }
+
+ override def getTablesByPattern(hive: Hive, dbName: String, pattern: String): JList[String] = {
+ hive.getTablesByPattern(dbName, pattern)
+ }
+
+ override def getAllTables(hive: Hive, dbName: String): JList[String] = {
+ hive.getAllTables(dbName)
Review comment:
```suggestion
hive.getAllTables(dbName)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952620977
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #34388:
URL: https://github.com/apache/spark/pull/34388
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951924921
**[Test build #144621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144621/testReport)** for PR 34388 at commit [`a668f50`](https://github.com/apache/spark/commit/a668f504d82340229f6f1cc34e6bf79096cd18a1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952005030
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49091/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952542439
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49110/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952497190
**[Test build #144640 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144640/testReport)** for PR 34388 at commit [`a888ae7`](https://github.com/apache/spark/commit/a888ae7597262cb384f51611aa63fbba79db3337).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951823964
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952052062
**[Test build #144621 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144621/testReport)** for PR 34388 at commit [`a668f50`](https://github.com/apache/spark/commit/a668f504d82340229f6f1cc34e6bf79096cd18a1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951517930
**[Test build #144609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144609/testReport)** for PR 34388 at commit [`ac11fef`](https://github.com/apache/spark/commit/ac11fef275d9ab05328a2fd4199130bb083a0fe5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951553001
**[Test build #144609 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144609/testReport)** for PR 34388 at commit [`ac11fef`](https://github.com/apache/spark/commit/ac11fef275d9ab05328a2fd4199130bb083a0fe5).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34388:
URL: https://github.com/apache/spark/pull/34388#discussion_r736515376
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -97,12 +116,50 @@ private[client] sealed abstract class Shim {
def alterPartitions(hive: Hive, tableName: String, newParts: JList[Partition]): Unit
+ def createTable(hive: Hive, table: Table, ifNotExists: Boolean): Unit
+
+ def getTable(
+ hive: Hive,
+ dbName: String,
+ tableName: String,
+ throwException: Boolean = true): Table
+
def getTablesByType(
hive: Hive,
dbName: String,
pattern: String,
tableType: TableType): Seq[String]
+ def getTablesByPattern(hive: Hive, dbName: String, pattern: String): JList[String]
+
+ def getAllTables(hive: Hive, dbName: String): JList[String]
+
+ def dropTable(hive: Hive, dbName: String, tableName: String): Unit
+
+ def getPartition(
+ hive: Hive,
+ table: Table,
+ partSpec: JMap[String, String],
+ forceCreate: Boolean): Partition
+
+ def getPartitions(
+ hive: Hive,
+ table: Table,
+ partSpec: JMap[String, String]): JList[Partition]
+
+ def getPartitionNames(
+ hive: Hive,
+ dbName: String,
+ tblName: String,
+ max: Short): JList[String]
Review comment:
###
```suggestion
hive: Hive,
dbName: String,
tblName: String,
max: Short): JList[String]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952053621
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144621/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951734208
**[Test build #144619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144619/testReport)** for PR 34388 at commit [`889b0b5`](https://github.com/apache/spark/commit/889b0b59141398f6ddfea02fd6f0b6d0dba5e7cc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952547318
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144640/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951667572
**[Test build #144617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144617/testReport)** for PR 34388 at commit [`7013bde`](https://github.com/apache/spark/commit/7013bdee17576166e30141f714d250831db845a7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951516194
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951734208
**[Test build #144619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/144619/testReport)** for PR 34388 at commit [`889b0b5`](https://github.com/apache/spark/commit/889b0b59141398f6ddfea02fd6f0b6d0dba5e7cc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951836254
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144619/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951821705
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49089/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951823964
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #34388:
URL: https://github.com/apache/spark/pull/34388#discussion_r737059694
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala
##########
@@ -484,6 +549,111 @@ private[client] class Shim_v0_12 extends Shim with Logging {
override def getDatabaseOwnerName(db: Database): String = ""
override def setDatabaseOwnerName(db: Database, owner: String): Unit = {}
+
+ override def createDatabase(hive: Hive, db: Database, ignoreIfExists: Boolean): Unit = {
+ hive.createDatabase(db, ignoreIfExists)
+ }
+
+ override def dropDatabase(
+ hive: Hive,
+ name: String,
Review comment:
Done, and handle all of such case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952514767
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49110/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952053621
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144621/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951726898
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49087/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951517928
ping @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951535695
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49078/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951562783
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34388: [SPARK-37115][SQL] HiveClientImple use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-951781223
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49087/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34388: [SPARK-37115][SQL] HiveClientImpl should use shim to wrap all hive client calls
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34388:
URL: https://github.com/apache/spark/pull/34388#issuecomment-952537568
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49110/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org