You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "raulcd (via GitHub)" <gi...@apache.org> on 2023/06/21 10:46:14 UTC
[GitHub] [arrow] raulcd opened a new issue, #36199: [Python][CI] Update spark versions tested on our nightly tests
raulcd opened a new issue, #36199:
URL: https://github.com/apache/arrow/issues/36199
### Describe the enhancement requested
It does seem we are using some pretty old and deprecated spark versions on our CI nightly integration builds with spark.
As discussed on this PR comment discussion: https://github.com/apache/arrow/pull/36061#discussion_r1230266720
The following comments were added:
Do we still need Spark 3.1.2?
I think that Spark 3.1 reached EOL:
https://spark.apache.org/versioning-policy.html
> Feature release branches will, generally, be maintained with bug fix releases for a period of 18 months.
And 3.1.1 was released on 2021-03-02:
https://spark.apache.org/news/index.html
> [Spark 3.1.1 released](https://spark.apache.org/news/spark-3-1-1-released.html)
> March 2, 2021
3.2 might reach EOL because 3.2.0 was released on 2021-10-13:
> [Spark 3.2.0 released](https://spark.apache.org/news/spark-3-2-0-released.html)
> October 13, 2021
### Component(s)
Continuous Integration, Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kiszk commented on issue #36199: [Python][CI] Update spark versions tested on our nightly tests
Posted by "kiszk (via GitHub)" <gi...@apache.org>.
kiszk commented on issue #36199:
URL: https://github.com/apache/arrow/issues/36199#issuecomment-1602643968
You are right. [This](https://spark.apache.org/versioning-policy.html) (at the bottom) is the maintenance policy. For 18 months from the release. Based on the policy, the current active Sparks are 3.2.x, 3.3.x, and 3.4.x.
Spark 3.2.x will be retired soon.
> [Spark 3.2.0 released](https://spark.apache.org/news/spark-3-2-0-released.html)
> October 13, 2021
Spark 3.5 will be released after this summer
> August 2023 | Release candidates (RC), voting, etc. until final release passes
So, my suggestion is to test Spark 3.4.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser closed issue #36199: [Python][CI] Update spark versions tested on our nightly tests
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser closed issue #36199: [Python][CI] Update spark versions tested on our nightly tests
URL: https://github.com/apache/arrow/issues/36199
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on issue #36199: [Python][CI] Update spark versions tested on our nightly tests
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #36199:
URL: https://github.com/apache/arrow/issues/36199#issuecomment-1600610942
@kiszk do you know what are the versions we should be testing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org