You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "raulcd (via GitHub)" <gi...@apache.org> on 2023/06/28 09:38:52 UTC
[GitHub] [arrow] raulcd opened a new pull request, #36347: GH-36199: [Python][CI][Spark] Update spark versions used on our nightly tests
raulcd opened a new pull request, #36347:
URL: https://github.com/apache/arrow/pull/36347
### Rationale for this change
We are currently testing with very old or deprecated spark versions.
### What changes are included in this PR?
Update the spark versions to be tested.
### Are these changes tested?
They will via archery
### Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #36347: GH-36199: [Python][CI][Spark] Update spark versions used on our nightly tests
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #36347:
URL: https://github.com/apache/arrow/pull/36347#issuecomment-1671724964
After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit e34d4c93761d4761f0df2626114927c80a81a87f.
There was 1 benchmark result indicating a performance regression:
- Commit Run on `arm64-t4g-linux-compute` at [2023-08-02 00:09:05Z](http://conbench.ursa.dev/compare/runs/4aaabbfed66f4228bffa134995a22073...ef78ecbdab86456db05812eb172a17b4/)
- [`tpch` (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1](http://conbench.ursa.dev/compare/benchmarks/064c972c49867acb8000258990f9fc4e...064c9aa93df178558000420de0982a2c)
The [full Conbench report](https://github.com/apache/arrow/runs/15754693452) has more details. It also includes information about possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #36347: GH-36199: [Python][CI][Spark] Update spark versions used on our nightly tests
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #36347:
URL: https://github.com/apache/arrow/pull/36347#discussion_r1281177096
##########
dev/tasks/tasks.yml:
##########
@@ -1588,9 +1588,9 @@ tasks:
image: conda-python-hdfs
{% endfor %}
-{% for python_version, spark_version, test_pyarrow_only, numpy_version in [("3.8", "v3.1.2", "false", "latest"),
- ("3.9", "v3.2.0", "false", "1.23"),
- ("3.10", "master", "false", "latest")] %}
+{% for python_version, spark_version, test_pyarrow_only, numpy_version in [("3.8", "v3.4.1", "false", "latest"),
+ ("3.10", "v3.4.0", "false", "1.23"),
Review Comment:
Should we use the same version (`v3.4.1` or `v3.4.0`) here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #36347: GH-36199: [Python][CI][Spark] Update spark versions used on our nightly tests
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36347:
URL: https://github.com/apache/arrow/pull/36347#issuecomment-1660287370
Revision: 1a71645808930dd570522e57b4e88e65ff2ae6f9
Submitted crossbow builds: [ursacomputing/crossbow @ actions-32d56069dc](https://github.com/ursacomputing/crossbow/branches/all?query=actions-32d56069dc)
|Task|Status|
|----|------|
|test-conda-python-3.10-spark-v3.4.0|[](https://github.com/ursacomputing/crossbow/actions/runs/5727234324/job/15519217663)|
|test-conda-python-3.11-spark-master|[](https://github.com/ursacomputing/crossbow/actions/runs/5727233972/job/15519216721)|
|test-conda-python-3.8-spark-v3.4.1|[](https://github.com/ursacomputing/crossbow/actions/runs/5727233669/job/15519215905)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser merged pull request #36347: GH-36199: [Python][CI][Spark] Update spark versions used on our nightly tests
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser merged PR #36347:
URL: https://github.com/apache/arrow/pull/36347
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #36347: GH-36199: [Python][CI][Spark] Update spark versions used on our nightly tests
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #36347:
URL: https://github.com/apache/arrow/pull/36347#issuecomment-1611091585
This is currently blocked by: https://github.com/apache/arrow/issues/36332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #36347: GH-36199: [Python][CI][Spark] Update spark versions used on our nightly tests
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #36347:
URL: https://github.com/apache/arrow/pull/36347#issuecomment-1660281054
@github-actions crossbow submit *spark*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #36347: GH-36199: [Python][CI][Spark] Update spark versions used on our nightly tests
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on code in PR #36347:
URL: https://github.com/apache/arrow/pull/36347#discussion_r1281183286
##########
dev/tasks/tasks.yml:
##########
@@ -1588,9 +1588,9 @@ tasks:
image: conda-python-hdfs
{% endfor %}
-{% for python_version, spark_version, test_pyarrow_only, numpy_version in [("3.8", "v3.1.2", "false", "latest"),
- ("3.9", "v3.2.0", "false", "1.23"),
- ("3.10", "master", "false", "latest")] %}
+{% for python_version, spark_version, test_pyarrow_only, numpy_version in [("3.8", "v3.4.1", "false", "latest"),
+ ("3.10", "v3.4.0", "false", "1.23"),
Review Comment:
I thought on covering several versions but now that I look into it again, a patch difference will not make a huge difference so I am ok changing it to:
```suggestion
{% for python_version, spark_version, test_pyarrow_only, numpy_version in [("3.8", "v3.4.1", "false", "latest"),
("3.10", "v3.4.1", "false", "1.23"),
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #36347: GH-36199: [Python][CI][Spark] Update spark versions used on our nightly tests
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36347:
URL: https://github.com/apache/arrow/pull/36347#issuecomment-1611090900
:warning: GitHub issue #36199 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org