You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Jeff Zhang <zj...@gmail.com> on 2021/08/09 09:24:24 UTC

Is the pandas version in doc of using pyarrow in spark wrong

The doc says that the minimum supported pandas version is 0.23.2 which is
only supported in python2.
IIRC, python2 is not supported in pyspark a long time ago. Can any one
confirm whether the doc is wrong and what is the right version of pandas
and pyarrow ?

https://spark.apache.org/docs/latest/api/python/user_guide/arrow_pandas.html#recommended-pandas-and-pyarrow-versions

-- 
Best Regards

Jeff Zhang