[GitHub] [spark] zhengruifeng commented on pull request #42920: [SPARK-45143][PYTHON][CONNECT] Make PySpark compatible with PyArrow 13.0.0

["zhengruifeng (via GitHub)" <gi...@apache.org>]

zhengruifeng commented on PR #42920:
URL: https://github.com/apache/spark/pull/42920#issuecomment-1720329365

   @dongjoon-hyun I don't see failure in [docker build](https://github.com/zhengruifeng/spark/actions/runs/6184148073/job/16787288022)
   
   ```
   #21 [15/19] RUN python3.9 -m pip install numpy pyarrow 'pandas<=2.0.3' scipy unittest-xml-reporting plotly>=4.8 'mlflow>=2.3.1' coverage matplotlib openpyxl 'memory-profiler==0.60.0' 'scikit-learn==1.1.*'
   #21 43.17 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
   #21 DONE 46.2s
   ```
   
   we don't pin the mlflow version, instead we only set a lowerbound `mlflow>=2.3.1`, it looks like mlflow 2.7.0 can work with PyArrow 13.0.0:
   
   `pyarrow [required: >=4.0.0,<14, installed: 13.0.0]`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org