Re: [PR] [SPARK-47151][PYTHON][PS][BUILD] Upgrade to `pandas` 2.2.1 [spark]

["nchammas (via GitHub)" <gi...@apache.org>]

nchammas commented on code in PR #45236:
URL: https://github.com/apache/spark/pull/45236#discussion_r1509657863


##########
python/pyspark/pandas/supported_api_gen.py:
##########
@@ -38,7 +38,7 @@
 MAX_MISSING_PARAMS_SIZE = 5
 COMMON_PARAMETER_SET = {"kwargs", "args", "cls"}
 MODULE_GROUP_MATCH = [(pd, ps), (pdw, psw), (pdg, psg)]
-PANDAS_LATEST_VERSION = "2.2.0"
+PANDAS_LATEST_VERSION = "2.2.1"

Review Comment:
   This issue predates this PR, but I note that this version requirement is more strict than our dev requirement: https://github.com/apache/spark/blob/e46d7d1c60ab82a734bb5c40229d5e72a47dd932/dev/requirements.txt#L8
   
   One consequence of this is that trying to build the Python API docs locally can fail as follows:
   
   ```python
   .../spark/python/pyspark/pandas/supported_api_gen.py:115:
     UserWarning: Warning: pandas 2.2.1 is required; your version is 2.1.4
   ```
   
   We should perhaps align the dev requirement with this Pandas version requirement here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org