You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/02/28 09:13:38 UTC

[GitHub] [spark] AngersZhuuuu opened a new pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

AngersZhuuuu opened a new pull request #31680:
URL: https://github.com/apache/spark/pull/31680


   ### What changes were proposed in this pull request?
   When SparkContext is initialed, if we want to start SparkSession, when we call 
   `SparkSession.buillder.enableHiveSupport().getOrCreate()`, the SparkSession we created won't have hive support since 
   we have't reset existed SC's conf's `spark.sql.catalogImplementation`.
   
   Also, if we use existed SparkSession or default SparkSession, we need to check if the catalog implementation  is same.
   
   In this pr we do two thinks:
   
   1. When SparkContext is existed, reset `spark.sql.catalogImplementation` according to current configuration.
   2. When use existed SparkSession and defaultSession, we should check if catalog implementation is same.
   
   
   ### Why are the changes needed?
   We should respect `enableHiveSupport`
   
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   
   ### How was this patch tested?
   WIP
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810449836


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136716/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787721078


   **[Test build #135572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135572/testReport)** for PR 31680 at commit [`f73421a`](https://github.com/apache/spark/commit/f73421ae2df131faeb2509083099ecbbd645a7d0).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r584495945



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
##########
@@ -946,7 +946,11 @@ object SparkSession extends Logging {
           SparkContext.getOrCreate(sparkConf)
           // Do not update `SparkConf` for existing `SparkContext`, as it's shared by all sessions.
         }
-
+        // We should reset `spark.sql.catalogImplementation` according to current requirement.
+        if (sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "in-memory" &&
+          sparkConf.get(CATALOG_IMPLEMENTATION) == "hive") {

Review comment:
       > I think if sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "hive", I think user have set this before start SparkSession.
   
   `CATALOG_IMPLEMENTATION` is a static configuration, so we can change the value that a user sets before starting a SparkContext?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810449836


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136716/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787729728


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135572/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787816901


   > > ```sql
   > >            SparkSession.sessionStateClassName(sharedState.conf),
   > > ```
   > > 
   > > 
   > > seems correct to me
   > 
   > Hmmmm
   > 
   > ```
   > private[spark] object HiveUtils extends Logging {
   > 
   >   def withHiveExternalCatalog(sc: SparkContext): SparkContext = {
   >     sc.conf.set(CATALOG_IMPLEMENTATION.key, "hive")
   >     sc
   >   }
   > ```
   
   Can't we use `SparkSession.builder().enableHiveSupport().sparkContext(sc).getOrCreate()` in `HiveContext`？


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810321030


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41304/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787464897


   > ```sql
   >            SparkSession.sessionStateClassName(sharedState.conf),
   > ```
   > 
   > seems correct to me
   
   LGTM, I will try this way.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809291710


   **[Test build #136636 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136636/testReport)** for PR 31680 at commit [`f9024fe`](https://github.com/apache/spark/commit/f9024fecda1c75d631b1b8bd5b478c8ceae9de2f).
    * This patch **fails PySpark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810092729


   **[Test build #136692 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136692/testReport)** for PR 31680 at commit [`858390b`](https://github.com/apache/spark/commit/858390be282e313af43c3ac7c7ef9f87d4972b0a).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809306676


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136636/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787431445


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40134/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810207995


   **[Test build #136717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136717/testReport)** for PR 31680 at commit [`3d86d79`](https://github.com/apache/spark/commit/3d86d795f4c3f61bc9eaca77492d379d1d66ea48).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810207995






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r604080765



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,21 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {
+    val conf = new SparkConf().setMaster("local").setAppName("SPARK-34568")
+    val sc = SparkContext.getOrCreate(conf)
+    val catalog = sc.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION)
+    try {
+      sc.conf.set(StaticSQLConf.CATALOG_IMPLEMENTATION, "in-memory")
+      val sparkSession = SparkSession.builder().enableHiveSupport().sparkContext(sc).getOrCreate()
+      assert(
+        sparkSession.sparkContext.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION) == "in-memory")
+      assert(sparkSession.sharedState.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION) == "hive")

Review comment:
       > ditto
   
   Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809961950


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41274/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810251144


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41298/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r584469068



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
##########
@@ -946,7 +946,11 @@ object SparkSession extends Logging {
           SparkContext.getOrCreate(sparkConf)
           // Do not update `SparkConf` for existing `SparkContext`, as it's shared by all sessions.
         }
-
+        // We should reset `spark.sql.catalogImplementation` according to current requirement.
+        if (sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "in-memory" &&
+          sparkConf.get(CATALOG_IMPLEMENTATION) == "hive") {

Review comment:
       > What if `sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "hive" && sparkConf.get(CATALOG_IMPLEMENTATION) == "in-memory"`?
   
   I think if `sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "hive"`, I think user have set this before start SparkSession.
   
   And we have a method `HiveUtils. withHiveExternalCatalog` we should respect this too ==




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r584497705



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
##########
@@ -946,7 +946,11 @@ object SparkSession extends Logging {
           SparkContext.getOrCreate(sparkConf)
           // Do not update `SparkConf` for existing `SparkContext`, as it's shared by all sessions.
         }
-
+        // We should reset `spark.sql.catalogImplementation` according to current requirement.
+        if (sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "in-memory" &&
+          sparkConf.get(CATALOG_IMPLEMENTATION) == "hive") {

Review comment:
       > > I think if sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "hive", I think user have set this before start SparkSession.
   > 
   > `CATALOG_IMPLEMENTATION` is a static configuration, so we can change the value that a user sets before starting a SparkContext?
   
   But if we start SparkContext and then call `HiveUtils. withHiveExternalCatalog()` then use this SparkContext to start SparkSession?
   
   Seems our test framework is build like this. and you can see my test result before the last finished one.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809926159


   **[Test build #136692 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136692/testReport)** for PR 31680 at commit [`858390b`](https://github.com/apache/spark/commit/858390be282e313af43c3ac7c7ef9f87d4972b0a).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r604066109



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,21 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {
+    val conf = new SparkConf().setMaster("local").setAppName("SPARK-34568")
+    val sc = SparkContext.getOrCreate(conf)
+    val catalog = sc.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION)
+    try {
+      sc.conf.set(StaticSQLConf.CATALOG_IMPLEMENTATION, "in-memory")
+      val sparkSession = SparkSession.builder().enableHiveSupport().sparkContext(sc).getOrCreate()
+      assert(
+        sparkSession.sparkContext.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION) == "in-memory")

Review comment:
       nit: ===

##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,21 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {
+    val conf = new SparkConf().setMaster("local").setAppName("SPARK-34568")
+    val sc = SparkContext.getOrCreate(conf)
+    val catalog = sc.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION)
+    try {
+      sc.conf.set(StaticSQLConf.CATALOG_IMPLEMENTATION, "in-memory")
+      val sparkSession = SparkSession.builder().enableHiveSupport().sparkContext(sc).getOrCreate()
+      assert(
+        sparkSession.sparkContext.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION) == "in-memory")
+      assert(sparkSession.sharedState.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION) == "hive")

Review comment:
       ditto

##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,21 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {
+    val conf = new SparkConf().setMaster("local").setAppName("SPARK-34568")

Review comment:
       nit: disable ui

##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,21 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {

Review comment:
       the test name and the PR title are too general, can we describe the situation?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809989821


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41279/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809990977


   **[Test build #136701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136701/testReport)** for PR 31680 at commit [`b733d1e`](https://github.com/apache/spark/commit/b733d1e74be281b2851d05e8bf599867b6708135).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r604059281



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,17 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {
+    val conf = new SparkConf().setMaster("local").setAppName("SPARK-34568")
+    val sc = SparkContext.getOrCreate(conf)
+    val catalog = sc.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION)
+    sc.conf.set(StaticSQLConf.CATALOG_IMPLEMENTATION, "in-memory")

Review comment:
       > try ... finally?
   
   Done

##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/test/TestHive.scala
##########
@@ -685,3 +685,10 @@ private[hive] object HiveTestJars {
     targetFile
   }
 }
+
+private[hive] object TestHiveUtils {
+  def withHiveExternalCatalog(sc: SparkContext): SparkContext = {

Review comment:
       > Could you move this method into `TestHiveContext`?
   
   Sure, more suitable place.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809860396


   **[Test build #136681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136681/testReport)** for PR 31680 at commit [`bfdd5eb`](https://github.com/apache/spark/commit/bfdd5eb9b10613b09e1fac2ce45c41936cf113f2).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810255894


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41298/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810447109


   **[Test build #136717 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136717/testReport)** for PR 31680 at commit [`3d86d79`](https://github.com/apache/spark/commit/3d86d795f4c3f61bc9eaca77492d379d1d66ea48).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809961916


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41274/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809325509


   **[Test build #136638 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136638/testReport)** for PR 31680 at commit [`2afad3b`](https://github.com/apache/spark/commit/2afad3b82cef36abecd4d32d14cb8736d878d49d).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809955112


   **[Test build #136697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136697/testReport)** for PR 31680 at commit [`6c114e1`](https://github.com/apache/spark/commit/6c114e125df5d5b3024a57b02c416c85ed6f9949).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787438416


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40134/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809933240






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810150296


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136697/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809878016


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41263/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810150296


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136697/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809150805


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41216/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810255937


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41298/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809695251


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41241/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787471624


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40139/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787603295


   **[Test build #135572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135572/testReport)** for PR 31680 at commit [`f73421a`](https://github.com/apache/spark/commit/f73421ae2df131faeb2509083099ecbbd645a7d0).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810449255






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810190461


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136701/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809306676


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136636/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809198047


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41219/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809198068


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41219/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787631580


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40153/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810315062


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41304/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787438416


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40134/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809999225


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41282/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809986253






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809340670


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136638/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu edited a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu edited a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787463830


   > > When SparkContext is initialed, if we want to start SparkSession, when we call `SparkSession.builder.enableHiveSupport().getOrCreate()`, the SparkSession we created won't have hive support
   > 
   > Is it true? In this case `enableHiveSupport` will add `spark.sql.catalogImplementation` to the initial configs for `SparkSession` and `SharedState`. And then, the config will be populated in the cloned SparkConf of `SharedState`, which is used to init the externalCatalog.
   
   But sessionStated is initialed according to SparkContext's conf
   ```
     lazy val sessionState: SessionState = {
       parentSessionState
         .map(_.clone(this))
         .getOrElse {
           val state = SparkSession.instantiateSessionState(
             SparkSession.sessionStateClassName(sparkContext.conf),
             self,
             initialSessionOptions)
           state
         }
     }
   ```
   
   Maybe we can just change the conf we used in `SparkSession.sessionStateClassName(sparkContext.conf),`


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809384038


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41213/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787502444


   **[Test build #135558 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135558/testReport)** for PR 31680 at commit [`2d13c15`](https://github.com/apache/spark/commit/2d13c1503f78a094463dde1a6534ddc132599891).
    * This patch **fails SparkR unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809109292






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810449255






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810789864


   thanks, merging to master!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810257355


   **[Test build #136722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136722/testReport)** for PR 31680 at commit [`97dfee2`](https://github.com/apache/spark/commit/97dfee23d840cacad2d194c644cf93f36983c576).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810172342


   **[Test build #136701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136701/testReport)** for PR 31680 at commit [`b733d1e`](https://github.com/apache/spark/commit/b733d1e74be281b2851d05e8bf599867b6708135).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809990977


   **[Test build #136701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136701/testReport)** for PR 31680 at commit [`b733d1e`](https://github.com/apache/spark/commit/b733d1e74be281b2851d05e8bf599867b6708135).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809989821


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41279/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r603808550



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,17 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {

Review comment:
       It seems this test can pass in the current master. Have you checked that?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maropu commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810200636


   Seems fine to me cc: @yaooqinn  @cloud-fan 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809389750


   **[Test build #136659 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136659/testReport)** for PR 31680 at commit [`0ade784`](https://github.com/apache/spark/commit/0ade78468413d929cd0b1ca719ae09a970df5cb8).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809956329


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136681/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809930044


   **[Test build #136694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136694/testReport)** for PR 31680 at commit [`7030e5c`](https://github.com/apache/spark/commit/7030e5c8e94c35c4731fa9aaddea0a55b696d50d).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809993918


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41282/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809955459


   **[Test build #136681 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136681/testReport)** for PR 31680 at commit [`bfdd5eb`](https://github.com/apache/spark/commit/bfdd5eb9b10613b09e1fac2ce45c41936cf113f2).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r604037690



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/test/TestHive.scala
##########
@@ -685,3 +685,10 @@ private[hive] object HiveTestJars {
     targetFile
   }
 }
+
+private[hive] object TestHiveUtils {
+  def withHiveExternalCatalog(sc: SparkContext): SparkContext = {

Review comment:
       Could you move this method into `TestHiveContext`?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787509800


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135558/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809088540


   **[Test build #136638 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136638/testReport)** for PR 31680 at commit [`2afad3b`](https://github.com/apache/spark/commit/2afad3b82cef36abecd4d32d14cb8736d878d49d).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809191355


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41219/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787586508


   > ```sql
   >            SparkSession.sessionStateClassName(sharedState.conf),
   > ```
   > 
   > seems correct to me
   
   Hmmmm 
   ```
   private[spark] object HiveUtils extends Logging {
   
     def withHiveExternalCatalog(sc: SparkContext): SparkContext = {
       sc.conf.set(CATALOG_IMPLEMENTATION.key, "hive")
       sc
     }
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810190461


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136701/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787462880


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135553/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810114352


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136692/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809955112


   **[Test build #136697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136697/testReport)** for PR 31680 at commit [`6c114e1`](https://github.com/apache/spark/commit/6c114e125df5d5b3024a57b02c416c85ed6f9949).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809198068


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41219/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809679840


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41241/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787729728


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135572/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809860396


   **[Test build #136681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136681/testReport)** for PR 31680 at commit [`bfdd5eb`](https://github.com/apache/spark/commit/bfdd5eb9b10613b09e1fac2ce45c41936cf113f2).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809654497


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136659/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809999181


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41282/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan closed pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31680:
URL: https://github.com/apache/spark/pull/31680


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810133836


   **[Test build #136697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136697/testReport)** for PR 31680 at commit [`6c114e1`](https://github.com/apache/spark/commit/6c114e125df5d5b3024a57b02c416c85ed6f9949).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809087041


   **[Test build #136636 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136636/testReport)** for PR 31680 at commit [`f9024fe`](https://github.com/apache/spark/commit/f9024fecda1c75d631b1b8bd5b478c8ceae9de2f).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810321030


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41304/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810267467


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41299/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809934933


   Kubernetes integration test unable to build dist.
   
   exiting with code: 1
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41276/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809901032


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41263/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809901032


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41263/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809695251


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41241/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810436110


   **[Test build #136722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136722/testReport)** for PR 31680 at commit [`97dfee2`](https://github.com/apache/spark/commit/97dfee23d840cacad2d194c644cf93f36983c576).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787421180


   Gentle ping @HyukjinKwon  Since I am not familiar with python and r, hope for some help about python and r.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787427193


   **[Test build #135553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135553/testReport)** for PR 31680 at commit [`5dac115`](https://github.com/apache/spark/commit/5dac115b55393f70c6d944cbe5568ac11a4e17c5).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809956978


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41274/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787509800


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135558/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809926159


   **[Test build #136692 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136692/testReport)** for PR 31680 at commit [`858390b`](https://github.com/apache/spark/commit/858390be282e313af43c3ac7c7ef9f87d4972b0a).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809999225


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41282/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r604080593



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,21 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {
+    val conf = new SparkConf().setMaster("local").setAppName("SPARK-34568")
+    val sc = SparkContext.getOrCreate(conf)
+    val catalog = sc.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION)
+    try {
+      sc.conf.set(StaticSQLConf.CATALOG_IMPLEMENTATION, "in-memory")
+      val sparkSession = SparkSession.builder().enableHiveSupport().sparkContext(sc).getOrCreate()
+      assert(
+        sparkSession.sparkContext.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION) == "in-memory")

Review comment:
       > nit: ===
   
   Done

##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,21 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {
+    val conf = new SparkConf().setMaster("local").setAppName("SPARK-34568")

Review comment:
       > nit: disable ui
   
   Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809389750


   **[Test build #136659 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136659/testReport)** for PR 31680 at commit [`0ade784`](https://github.com/apache/spark/commit/0ade78468413d929cd0b1ca719ae09a970df5cb8).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809933240


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136694/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r603828784



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,17 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {

Review comment:
       > It seems this test can pass in the current master. Have you checked that?
   
   Hmmm, make a mistake, we should check SessionState.., Updated




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810267467


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41299/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787603295


   **[Test build #135572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135572/testReport)** for PR 31680 at commit [`f73421a`](https://github.com/apache/spark/commit/f73421ae2df131faeb2509083099ecbbd645a7d0).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809388775


   Gentle ping @maropu @yaooqinn Current way should be more suitable.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809956329


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136681/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787631580


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40153/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787462880


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135553/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787464549


   ```sql
              SparkSession.sessionStateClassName(sharedState.conf),
   ```
   
   seems correct to me
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809888197


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41263/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809654497


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136659/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809051861


   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810257355


   **[Test build #136722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136722/testReport)** for PR 31680 at commit [`97dfee2`](https://github.com/apache/spark/commit/97dfee23d840cacad2d194c644cf93f36983c576).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r603797915



##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
##########
@@ -34,7 +34,7 @@ class HiveContext private[hive](_sparkSession: SparkSession)
   self =>
 
   def this(sc: SparkContext) = {
-    this(SparkSession.builder().sparkContext(HiveUtils.withHiveExternalCatalog(sc)).getOrCreate())

Review comment:
       > `withHiveExternalCatalog` is used only for tests after this merged, so could you move it into a suitable place?
   
   Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809645750


   **[Test build #136659 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136659/testReport)** for PR 31680 at commit [`0ade784`](https://github.com/apache/spark/commit/0ade78468413d929cd0b1ca719ae09a970df5cb8).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787462635


   > When SparkContext is initialed, if we want to start SparkSession, when we call `SparkSession.builder.enableHiveSupport().getOrCreate()`, the SparkSession we created won't have hive support
   
   Is it true? In this case `enableHiveSupport` will add `spark.sql.catalogImplementation` to the initial configs for `SparkSession` and `SharedState`. And then, the config will be populated in the cloned SparkConf of `SharedState`, which is used to init the externalCatalog.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787456835


   **[Test build #135553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135553/testReport)** for PR 31680 at commit [`5dac115`](https://github.com/apache/spark/commit/5dac115b55393f70c6d944cbe5568ac11a4e17c5).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809087041


   **[Test build #136636 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136636/testReport)** for PR 31680 at commit [`f9024fe`](https://github.com/apache/spark/commit/f9024fecda1c75d631b1b8bd5b478c8ceae9de2f).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787471624


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40139/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r584497705



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
##########
@@ -946,7 +946,11 @@ object SparkSession extends Logging {
           SparkContext.getOrCreate(sparkConf)
           // Do not update `SparkConf` for existing `SparkContext`, as it's shared by all sessions.
         }
-
+        // We should reset `spark.sql.catalogImplementation` according to current requirement.
+        if (sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "in-memory" &&
+          sparkConf.get(CATALOG_IMPLEMENTATION) == "hive") {

Review comment:
       > > I think if sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "hive", I think user have set this before start SparkSession.
   > 
   > `CATALOG_IMPLEMENTATION` is a static configuration, so we can change the value that a user sets before starting a SparkContext?
   
   My expression may be inaccurate. I mean when we call `getOrCreate()` is current existed `SparkContext.conf`'s 
   `CATALOG_IMPLEMENTATION` is `hive`, it means user have decide to use hive ssupport.
   Such as if we start SparkContext and then call `HiveUtils. withHiveExternalCatalog()` then use this SparkContext to start SparkSession?
   
   Seems our test framework is build like this. and you can see my test result before the last finished one.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809059073


   **[Test build #136630 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136630/testReport)** for PR 31680 at commit [`f73421a`](https://github.com/apache/spark/commit/f73421ae2df131faeb2509083099ecbbd645a7d0).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809961950


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41274/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787469119


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40139/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787435631


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40134/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787463529


   **[Test build #135558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135558/testReport)** for PR 31680 at commit [`2d13c15`](https://github.com/apache/spark/commit/2d13c1503f78a094463dde1a6534ddc132599891).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809119022


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41216/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r604081194



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,21 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {

Review comment:
       > the test name and the PR title are too general, can we describe the situation?
   
   Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r603794244



##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
##########
@@ -34,7 +34,7 @@ class HiveContext private[hive](_sparkSession: SparkSession)
   self =>
 
   def this(sc: SparkContext) = {
-    this(SparkSession.builder().sparkContext(HiveUtils.withHiveExternalCatalog(sc)).getOrCreate())

Review comment:
       `withHiveExternalCatalog` is used only for tests after this merged, so could you move it into a suitable place?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810267378






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809933218


   **[Test build #136694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136694/testReport)** for PR 31680 at commit [`7030e5c`](https://github.com/apache/spark/commit/7030e5c8e94c35c4731fa9aaddea0a55b696d50d).
    * This patch **fails to build**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810320976


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41304/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809340670


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136638/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r604064178



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/test/TestHive.scala
##########
@@ -647,6 +647,10 @@ private[hive] object TestHiveContext {
     scratchDir
   }
 
+  def withHiveExternalCatalog(sc: SparkContext): SparkContext = {

Review comment:
       private




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809930044


   **[Test build #136694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136694/testReport)** for PR 31680 at commit [`7030e5c`](https://github.com/apache/spark/commit/7030e5c8e94c35c4731fa9aaddea0a55b696d50d).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809062635


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136633/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810255937


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41298/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809109292






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787463529


   **[Test build #135558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135558/testReport)** for PR 31680 at commit [`2d13c15`](https://github.com/apache/spark/commit/2d13c1503f78a094463dde1a6534ddc132599891).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809088540


   **[Test build #136638 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136638/testReport)** for PR 31680 at commit [`2afad3b`](https://github.com/apache/spark/commit/2afad3b82cef36abecd4d32d14cb8736d878d49d).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r604047420



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
##########
@@ -88,4 +89,17 @@ class HiveSharedStateSuite extends SparkFunSuite {
     assert(!ss.conf.get(WAREHOUSE_PATH).contains(invalidPath),
       "session level conf should be passed to catalog")
   }
+
+  test("SPARK-34568: We should respect enableHiveSupport when initialize SparkSession") {
+    val conf = new SparkConf().setMaster("local").setAppName("SPARK-34568")
+    val sc = SparkContext.getOrCreate(conf)
+    val catalog = sc.conf.get(StaticSQLConf.CATALOG_IMPLEMENTATION)
+    sc.conf.set(StaticSQLConf.CATALOG_IMPLEMENTATION, "in-memory")

Review comment:
       try ... finally?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r604067607



##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/test/TestHive.scala
##########
@@ -647,6 +647,10 @@ private[hive] object TestHiveContext {
     scratchDir
   }
 
+  def withHiveExternalCatalog(sc: SparkContext): SparkContext = {

Review comment:
       > private
   
   Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809954008


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41276/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787470327


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40139/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787463830


   > > When SparkContext is initialed, if we want to start SparkSession, when we call `SparkSession.builder.enableHiveSupport().getOrCreate()`, the SparkSession we created won't have hive support
   > 
   > Is it true? In this case `enableHiveSupport` will add `spark.sql.catalogImplementation` to the initial configs for `SparkSession` and `SharedState`. And then, the config will be populated in the cloned SparkConf of `SharedState`, which is used to init the externalCatalog.
   
   But sessionStated is initialed according to SparkContext's conf
   ```
     lazy val sessionState: SessionState = {
       parentSessionState
         .map(_.clone(this))
         .getOrElse {
           val state = SparkSession.instantiateSessionState(
             SparkSession.sessionStateClassName(sparkContext.conf),
             self,
             initialSessionOptions)
           state
         }
     }
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809119022


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41216/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31680:
URL: https://github.com/apache/spark/pull/31680#discussion_r584461453



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
##########
@@ -946,7 +946,11 @@ object SparkSession extends Logging {
           SparkContext.getOrCreate(sparkConf)
           // Do not update `SparkConf` for existing `SparkContext`, as it's shared by all sessions.
         }
-
+        // We should reset `spark.sql.catalogImplementation` according to current requirement.
+        if (sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "in-memory" &&
+          sparkConf.get(CATALOG_IMPLEMENTATION) == "hive") {

Review comment:
       What if `sparkContext.conf.get(CATALOG_IMPLEMENTATION) == "hive" && sparkConf.get(CATALOG_IMPLEMENTATION) == "in-memory"`?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810197787


   **[Test build #136716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136716/testReport)** for PR 31680 at commit [`8d6548f`](https://github.com/apache/spark/commit/8d6548f5b351d05b8253d7fb12fd5f50fb9889e5).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810114352


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136692/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-787427193


   **[Test build #135553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135553/testReport)** for PR 31680 at commit [`5dac115`](https://github.com/apache/spark/commit/5dac115b55393f70c6d944cbe5568ac11a4e17c5).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31680: [SPARK-34568][SQL] When SparkContext's conf not enable hive, we should respect `enableHiveSupport()` when build SparkSession too

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-810448153


   **[Test build #136716 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136716/testReport)** for PR 31680 at commit [`8d6548f`](https://github.com/apache/spark/commit/8d6548f5b351d05b8253d7fb12fd5f50fb9889e5).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31680: [SPARK-34568][SQL] We should respect enableHiveSupport when initialize SparkSession

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31680:
URL: https://github.com/apache/spark/pull/31680#issuecomment-809062635


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136633/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org