You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/05 22:30:41 UTC

[GitHub] [spark] gatorsmile opened a new pull request #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

gatorsmile opened a new pull request #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815
 
 
   ### What changes were proposed in this pull request?
   Based on the discussion in the mailing list [[Proposal] Modification to Spark's Semantic Versioning Policy](http://apache-spark-developers-list.1001551.n3.nabble.com/Proposal-Modification-to-Spark-s-Semantic-Versioning-Policy-td28938.html) , this PR is to add back the following APIs whose maintenance cost are relatively small.
   
   - HiveContext
   - createExternalTable APIs
   
   ### Why are the changes needed?
   
   Avoid breaking the APIs that are commonly used.
   
   ### Does this PR introduce any user-facing change?
   Adding back the APIs that were removed in 3.0 branch does not introduce the user-facing changes, because Spark 3.0 has not been released. 
   
   ### How was this patch tested?
   
   TODO: add a new test suite for createExternalTable APIs.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile edited a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile edited a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603491190
 
 
   Adoption of Spark 3.0 is much more important than removal of these deprecated methods. I strongly disagree we removed this APIs in Spark 3.0 at the beginning. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596279352
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24278/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596320159
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596319731
 
 
   **[Test build #119546 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119546/testReport)** for PR 27815 at commit [`294337d`](https://github.com/apache/spark/commit/294337dc1e58c04cd400c3851e534de626b816dc).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds the following public classes _(experimental)_:
     * `class HiveContext(SQLContext):`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601537944
 
 
   **[Test build #120078 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120078/testReport)** for PR 27815 at commit [`c3a45e0`](https://github.com/apache/spark/commit/c3a45e0b1da5301f2ebd50ad95feadd782c42e58).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603331280
 
 
   **[Test build #4997 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4997/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603040256
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24956/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603503104
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24998/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603040256
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24956/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596278521
 
 
   Retest this please.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601537944
 
 
   **[Test build #120078 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120078/testReport)** for PR 27815 at commit [`c3a45e0`](https://github.com/apache/spark/commit/c3a45e0b1da5301f2ebd50ad95feadd782c42e58).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r391189171
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   `class HiveContext(SQLContext)` itself is not a simple alias. You know that. This is a whole new layer in Spark 1.6.x age.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596368753
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24287/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595482728
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595482246
 
 
   **[Test build #119423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119423/testReport)** for PR 27815 at commit [`294337d`](https://github.com/apache/spark/commit/294337dc1e58c04cd400c3851e534de626b816dc).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596368748
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601538183
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595482732
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24160/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603039888
 
 
   **[Test build #120242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120242/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603438794
 
 
   **[Test build #4997 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4997/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds the following public classes _(experimental)_:
     * `final class ANOVASelector @Since(\"3.1.0\")(@Since(\"3.1.0\") override val uid: String)`
     * `final class VarianceThresholdSelector @Since(\"3.1.0\")(@Since(\"3.1.0\") override val uid: String)`
     * `    // `case class Foo(i: Int) extends AnyVal` will return type `Int` instead of `Foo`.`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603502417
 
 
   **[Test build #120288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120288/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603503091
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] marmbrus commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
marmbrus commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r391197752
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   If you are just delegating to the "correct"/new implementation, then that sounds like an alias to me. If we are maintaining large parallel/duplication implementation, then that is a different story.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596368748
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596279118
 
 
   **[Test build #119546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119546/testReport)** for PR 27815 at commit [`294337d`](https://github.com/apache/spark/commit/294337dc1e58c04cd400c3851e534de626b816dc).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603508155
 
 
   To eventually remove these deprecated APIs, we need at least build some migration tools for end users to collect the usage of the deprecated APIs. So far, this is missing based on my knowledge. In general, I believe we need a systematic way for reduce the migration pain and make more end users easily upgrade to the latest version of Apache Spark. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595562420
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119423/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603675391
 
 
   @srowen Like the previous releases, we are doing the API auditing and checking all the changes we made in the upcoming 3.0 release. During the auditing, we found it and multiple Spark committers (including @marmbrus @zsxwing @yhuai ) challenged this API change in our offline discussion. Thus, I submitted this PR to add them back.  

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile closed pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile closed pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r391182471
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   I believe @gatorsmile should justify this using the new rubric. We are reviewing this PR.
   Also, cc @rxin since he is the release manager for 3.0.0.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601979545
 
 
   **[Test build #4996 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4996/testReport)** for PR 27815 at commit [`c3a45e0`](https://github.com/apache/spark/commit/c3a45e0b1da5301f2ebd50ad95feadd782c42e58).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603647771
 
 
   Hi, All.
   Although I have been against this reverting PR until now, I'll not be against any longer if @gatorsmile has his reason to support this.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596439170
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595482732
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24160/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603039888
 
 
   **[Test build #120242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120242/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603491190
 
 
   Adoption of Spark 3.0 is much more important than removal of these deprecated methods. I strongly disagreed we removed this APIs in Spark 3.0 at the beginning. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603041613
 
 
   CC @dongjoon-hyun @marmbrus @rxin @srowen This PR is ready for review.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595482246
 
 
   **[Test build #119423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119423/testReport)** for PR 27815 at commit [`294337d`](https://github.com/apache/spark/commit/294337dc1e58c04cd400c3851e534de626b816dc).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596368133
 
 
   **[Test build #119555 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119555/testReport)** for PR 27815 at commit [`ff2d91f`](https://github.com/apache/spark/commit/ff2d91f03d46bc3f67da1980ac5cb9aee61d093b).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r391189171
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   `class HiveContext(SQLContext)` itself is not a simple alias. You know that. This is a whole new layer from Spark 1.6.x age.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595562414
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r391182471
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   I believe @gatorsmile should justify this using the new rubric. We are reviewing this PR.
   Also, cc @rxin since he is the release manager.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596279118
 
 
   **[Test build #119546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119546/testReport)** for PR 27815 at commit [`294337d`](https://github.com/apache/spark/commit/294337dc1e58c04cd400c3851e534de626b816dc).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603503104
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24998/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r390665731
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   Please rethink about this. Although the other removed API are recovered, I believe we should not recover this.
   cc @marmbrus 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595562420
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119423/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603064796
 
 
   **[Test build #120242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120242/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007).
    * This patch **fails due to an unknown error code, -9**.
    * This patch merges cleanly.
    * This patch adds the following public classes _(experimental)_:
     * `final class ANOVASelector @Since(\"3.1.0\")(@Since(\"3.1.0\") override val uid: String)`
     * `final class VarianceThresholdSelector @Since(\"3.1.0\")(@Since(\"3.1.0\") override val uid: String)`
     * `    // `case class Foo(i: Int) extends AnyVal` will return type `Int` instead of `Foo`.`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603040249
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596368753
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24287/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595562414
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile edited a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile edited a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603650408
 
 
   There are three major reasons why I think we should keep HiveContext:
   - SQLContext will not be removed in the near future, because it is part of DSV1 APIs. HiveContext is just the corresponding parts with Hive metastore support. 
   - SQLContext/HiveContext is being used as the function parameter in many internal/public libraries. Replace it by SparkSession is not straightforward. It requires massive updates when end users upgrade Spark from 2.x to 3.x. 
   - SQLContext/HiveContext is being documented in various tutorials and books. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603638140
 
 
   Let me correct the typo and rewrite my original reply to make my point clear. 
   
   Based on the latest discussion in the community, I strongly disagree that we removed these APIs in Spark 3.0 initially. I think we need to add them back. If we want to remove them, we need more public discussions in the community. This PR is just to add the removed APIs back and delay the related discussions to the future releases. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601538189
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24795/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603064916
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120242/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601979545
 
 
   **[Test build #4996 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4996/testReport)** for PR 27815 at commit [`c3a45e0`](https://github.com/apache/spark/commit/c3a45e0b1da5301f2ebd50ad95feadd782c42e58).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601538189
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24795/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603573223
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120288/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603573223
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120288/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] marmbrus commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
marmbrus commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r391187119
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   Look at the code in this PR. I see a bunch of alias methods that call other methods and a few minimal tests. What part of this is hard to maintain?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596320166
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119546/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r391182471
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   I believe @gatorsmile should justify this using the new rubric about why this is `maintenance cost are relatively small`. We are reviewing this PR.
   Also, cc @rxin since he is the release manager for 3.0.0.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595482728
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601562032
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120078/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596279351
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603503091
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596439179
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119555/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603573214
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601562032
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120078/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603650408
 
 
   There are two major reasons why I think we should keep HiveContext:
   - SQLContext will not be removed in the near future, because it is part of DSV1 APIs. HiveContext is just the corresponding parts with Hive metastore support. 
   - SQLContext/HiveContext is being used as the function parameter in many internal/public libraries. Replace it by SparkSession is not straightforward. It requires massive updates when end users upgrade Spark from 2.x to 3.x. 
   - SQLContext/HiveContext is being documented in various tutorials and books. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603064903
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r391196560
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   It seems that we have different ideas on the alias. You are saying that all wrappers are alias, aren't you?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603572648
 
 
   **[Test build #120288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120288/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds the following public classes _(experimental)_:
     * `final class ANOVASelector @Since(\"3.1.0\")(@Since(\"3.1.0\") override val uid: String)`
     * `final class VarianceThresholdSelector @Since(\"3.1.0\")(@Since(\"3.1.0\") override val uid: String)`
     * `    // `case class Foo(i: Int) extends AnyVal` will return type `Int` instead of `Foo`.`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile edited a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile edited a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603650408
 
 
   There are three major reasons why I think we should keep HiveContext:
   - SQLContext will not be removed in the near future, because it is part of [DSV1 APIs](https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala). HiveContext is just the corresponding parts with Hive metastore support. 
   - SQLContext/HiveContext is being used as the function parameter in many internal/public libraries. Replace it by SparkSession is not straightforward. It requires massive updates when end users upgrade Spark from 2.x to 3.x. 
   - SQLContext/HiveContext is being documented in various tutorials and books. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601562027
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603331280
 
 
   **[Test build #4997 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4997/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601561829
 
 
   **[Test build #120078 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120078/testReport)** for PR 27815 at commit [`c3a45e0`](https://github.com/apache/spark/commit/c3a45e0b1da5301f2ebd50ad95feadd782c42e58).
    * This patch **fails due to an unknown error code, -9**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596439170
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603451597
 
 
   I also still strongly believe that we should let this go away from 3.0.0.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603064916
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120242/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603040249
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] marmbrus commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
marmbrus commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r391194663
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   I see 63 lines of code in that file? That class itself looks like an alias... not a whole separate implementation. What am I missing?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601989492
 
 
   **[Test build #4996 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4996/testReport)** for PR 27815 at commit [`c3a45e0`](https://github.com/apache/spark/commit/c3a45e0b1da5301f2ebd50ad95feadd782c42e58).
    * This patch **fails Spark unit tests**.
    * This patch **does not merge cleanly**.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603502417
 
 
   **[Test build #120288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120288/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596279351
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596438165
 
 
   **[Test build #119555 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119555/testReport)** for PR 27815 at commit [`ff2d91f`](https://github.com/apache/spark/commit/ff2d91f03d46bc3f67da1980ac5cb9aee61d093b).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds the following public classes _(experimental)_:
     * `class DeprecatedCreateExternalTableSuite extends SharedSparkSession `

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603064903
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596279352
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24278/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] marmbrus commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
marmbrus commented on a change in pull request #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#discussion_r391168827
 
 

 ##########
 File path: python/pyspark/sql/context.py
 ##########
 @@ -463,6 +481,53 @@ def streams(self):
         return StreamingQueryManager(self._ssql_ctx.streams())
 
 
+class HiveContext(SQLContext):
+    """A variant of Spark SQL that integrates with data stored in Hive.
+
+    Configuration for Hive is read from ``hive-site.xml`` on the classpath.
+    It supports running both SQL and HiveQL commands.
+
+    :param sparkContext: The SparkContext to wrap.
+    :param jhiveContext: An optional JVM Scala HiveContext. If set, we do not instantiate a new
+        :class:`HiveContext` in the JVM, instead we make all calls to this object.
+
+    .. note:: Deprecated in 2.0.0. Use SparkSession.builder.enableHiveSupport().getOrCreate().
 
 Review comment:
   Can you justify this position using the rubric we agreed upon?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596320166
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119546/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601562027
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srowen commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603673135
 
 
   @gatorsmile these changes happened almost a year ago. I'd like to understand why you have advanced these arguments only now. Has something changed?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-595561140
 
 
   **[Test build #119423 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119423/testReport)** for PR 27815 at commit [`294337d`](https://github.com/apache/spark/commit/294337dc1e58c04cd400c3851e534de626b816dc).
    * This patch **fails PySpark unit tests**.
    * This patch merges cleanly.
    * This patch adds the following public classes _(experimental)_:
     * `class HiveContext(SQLContext):`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srowen commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603631097
 
 
   @gatorsmile where did you disagree with this removal originally?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603499858
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596368133
 
 
   **[Test build #119555 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119555/testReport)** for PR 27815 at commit [`ff2d91f`](https://github.com/apache/spark/commit/ff2d91f03d46bc3f67da1980ac5cb9aee61d093b).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-603573214
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-28980][FOLLOW-UP] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596320159
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-601538183
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
URL: https://github.com/apache/spark/pull/27815#issuecomment-596439179
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119555/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org