You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "art000109 (via GitHub)" <gi...@apache.org> on 2026/07/12 09:23:45 UTC

[PR] [MINOR][DOC] Fix to use Hive 4.1 on Hadoop 3.5 in PySpark installation guide [spark]

art000109 opened a new pull request, #57208:
URL: https://github.com/apache/spark/pull/57208

   ### What changes were proposed in this pull request?
   
   This PR updates the outdated Hive version references from `3.2` to `4.1` in the PySpark installation guide
   
   ### Why are the changes needed?
   
   Referencing a commit bf5ccb4 [SPARK-58038]. Apache Hadoop 3.5 supports a minimum of Java 17, whereas Apache Hive supports Java 17 only in versions 4.1.x
   - #57124
   
   ### Does this PR introduce _any_ user-facing change?
   
   No. This is a documentation-only change.
   
   ### How was this patch tested?
   
   Manual review.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   No.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [MINOR][DOCS] Fix to use Hive 4.1 on Hadoop 3.5 in PySpark installation guide [spark]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #57208:
URL: https://github.com/apache/spark/pull/57208#issuecomment-4954795378

   Ya, @pan3793 's review comment is correct. Let me close this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [MINOR][DOCS] Fix to use Hive 4.1 on Hadoop 3.5 in PySpark installation guide [spark]

Posted by "art000109 (via GitHub)" <gi...@apache.org>.
art000109 commented on PR #57208:
URL: https://github.com/apache/spark/pull/57208#issuecomment-4959432736

   > Ya, @pan3793 's review comment is correct. Let me close this.
   
   Okay, thanks. Sorry, my mistake


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [MINOR][DOCS] Fix to use Hive 4.1 on Hadoop 3.5 in PySpark installation guide [spark]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #57208:
URL: https://github.com/apache/spark/pull/57208#issuecomment-4960153786

   No problem at all. Thank you for making a PR, @art000109 !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [MINOR][DOCS] Fix to use Hive 4.1 on Hadoop 3.5 in PySpark installation guide [spark]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #57208:
URL: https://github.com/apache/spark/pull/57208#discussion_r3567604952


##########
python/docs/source/getting_started/install.rst:
##########
@@ -62,7 +62,7 @@ For PySpark with/without a specific Hadoop version, you can install it by using
 
     PYSPARK_HADOOP_VERSION=3 pip install pyspark
 
-The default distribution uses Hadoop 3.5 and Hive 2.3. If users specify different versions of Hadoop, the pip installation automatically
+The default distribution uses Hadoop 3.5 and Hive 4.1. If users specify different versions of Hadoop, the pip installation automatically

Review Comment:
   The current docs are correct, Spark uses Hive 2.3 by default. Upgrading the default Hive version is non-trivial stuff.
   
   https://github.com/apache/spark/blob/9d760a91baa140fcf517a6b754ba75df3eab0f1f/pom.xml#L142



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [MINOR][DOCS] Fix to use Hive 4.1 on Hadoop 3.5 in PySpark installation guide [spark]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #57208: [MINOR][DOCS] Fix to use Hive 4.1 on Hadoop 3.5 in PySpark installation guide
URL: https://github.com/apache/spark/pull/57208


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org