You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2024/04/25 11:58:36 UTC
[PR] [SPARK-47993][PYTHON] Drop Python 3.8 [spark]
HyukjinKwon opened a new pull request, #46228:
URL: https://github.com/apache/spark/pull/46228
### What changes were proposed in this pull request?
This PR proposes to drop Python 3.8 in PySpark.
### Why are the changes needed?
Python 3.8 is EOL in this October (https://devguide.python.org/versions/). Assuming from the release schedule for Spark 4, I think it's better to drop it now before we do it after branchcut, etc.
This also fixes the broken scheduled build (https://github.com/apache/spark/actions/runs/8818780802)
### Does this PR introduce _any_ user-facing change?
Yes, it drops Python 3.8 support.
### How was this patch tested?
The changes will be tested through CI.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47993][PYTHON] Drop Python 3.8 [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #46228:
URL: https://github.com/apache/spark/pull/46228#issuecomment-2078236256
Will fix all up soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47993][PYTHON] Drop Python 3.8 [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #46228:
URL: https://github.com/apache/spark/pull/46228#issuecomment-2078883440
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47993][PYTHON] Drop Python 3.8 [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #46228: [SPARK-47993][PYTHON] Drop Python 3.8
URL: https://github.com/apache/spark/pull/46228
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47993][PYTHON] Drop Python 3.8 [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #46228:
URL: https://github.com/apache/spark/pull/46228#issuecomment-2078021395
Could you take a look at the CI failure, @HyukjinKwon ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47993][PYTHON] Drop Python 3.8 [spark]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on PR #46228:
URL: https://github.com/apache/spark/pull/46228#issuecomment-2077969367
+1 for dropping Python 3.8 support
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47993][PYTHON] Drop Python 3.8 [spark]
Posted by "nchammas (via GitHub)" <gi...@apache.org>.
nchammas commented on PR #46228:
URL: https://github.com/apache/spark/pull/46228#issuecomment-2078215705
`zoneinfo` is [new to Python 3.9][1]. It seems something in the build expects Python 3.9 or newer.
Looks like it's [this line][2], which was added in #46122 a few days ago.
[1]: https://docs.python.org/3/whatsnew/3.9.html#zoneinfo
[2]: https://github.com/apache/spark/blame/d540786d9ceacd7426803ad615f7ab32ec6faf67/python/pyspark/sql/variant_utils.py#L26
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47993][PYTHON] Drop Python 3.8 [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #46228:
URL: https://github.com/apache/spark/pull/46228#issuecomment-2077538849
BTW, I have two comments.
1. We need a vote for this because it's still alive, @HyukjinKwon .
2. I'm not sure `ModuleNotFoundError` could be the reason of dropping a Python version. Do we have any other reason which bans `Python 3.8`?
- https://github.com/apache/spark/actions/runs/8802717653/job/24159244287
```
ModuleNotFoundError: No module named 'zoneinfo'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org