You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "pitrou (via GitHub)" <gi...@apache.org> on 2023/10/18 11:52:19 UTC
[PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
pitrou opened a new pull request, #38324:
URL: https://github.com/apache/arrow/pull/38324
### Rationale for this change
gdb on conda-forge frequently conflicts with the newest Python version (example for Python 3.12: https://github.com/conda-forge/gdb-feedstock/issues/66)
### What changes are included in this PR?
Use the system-provided gdb instead of trying to install it from conda-forge.
This works because the Python interpreter used for gdb is entirely independent from the Python interpreter under test, and gdb itself can debug any binary, regardless of binutils version.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #38324:
URL: https://github.com/apache/arrow/pull/38324#discussion_r1363852688
##########
ci/docker/conda-python.dockerfile:
##########
@@ -23,9 +23,11 @@ FROM ${repo}:${arch}-conda-cpp
ARG python=3.8
COPY ci/conda_env_python.txt \
/arrow/ci/
+# If the Python version being tested is the same as the Python used by the system gdb,
+# we need to install the conda-forge gdb instead.
RUN mamba install -q -y \
--file arrow/ci/conda_env_python.txt \
- gdb \
+ $([ "$python" == "3.10" ] && echo "gdb") \
Review Comment:
@kou Does this look ok?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38324:
URL: https://github.com/apache/arrow/pull/38324#issuecomment-1768378244
Revision: cc0ad1f04e404d5ab4d1e43c6af7b21435da7933
Submitted crossbow builds: [ursacomputing/crossbow @ actions-add7835c90](https://github.com/ursacomputing/crossbow/branches/all?query=actions-add7835c90)
|Task|Status|
|----|------|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions/runs/6561053109/job/17819948405)|
|test-conda-python-3.10-cython2|[](https://github.com/ursacomputing/crossbow/actions/runs/6561050411/job/17819939923)|
|test-conda-python-3.10-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions/runs/6561048375/job/17819933408)|
|test-conda-python-3.10-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions/runs/6561054855/job/17819953431)|
|test-conda-python-3.10-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/6561051790/job/17819943810)|
|test-conda-python-3.10-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions/runs/6561053784/job/17819950469)|
|test-conda-python-3.10-spark-v3.5.0|[](https://github.com/ursacomputing/crossbow/actions/runs/6561050752/job/17819940787)|
|test-conda-python-3.10-substrait|[](https://github.com/ursacomputing/crossbow/actions/runs/6561051402/job/17819942911)|
|test-conda-python-3.11|[](https://github.com/ursacomputing/crossbow/actions/runs/6561054130/job/17819951593)|
|test-conda-python-3.11-dask-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/6561050979/job/17819941741)|
|test-conda-python-3.11-dask-upstream_devel|[](https://github.com/ursacomputing/crossbow/actions/runs/6561049469/job/17819936311)|
|test-conda-python-3.11-hypothesis|[](https://github.com/ursacomputing/crossbow/actions/runs/6561054519/job/17819952391)|
|test-conda-python-3.11-pandas-upstream_devel|[](https://github.com/ursacomputing/crossbow/actions/runs/6561055133/job/17819954171)|
|test-conda-python-3.11-spark-master|[](https://github.com/ursacomputing/crossbow/actions/runs/6561049035/job/17819935177)|
|test-conda-python-3.12|[](https://github.com/ursacomputing/crossbow/actions/runs/6561052806/job/17819947153)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions/runs/6561049822/job/17819937324)|
|test-conda-python-3.8-pandas-1.0|[](https://github.com/ursacomputing/crossbow/actions/runs/6561053714/job/17819950147)|
|test-conda-python-3.8-spark-v3.5.0|[](https://github.com/ursacomputing/crossbow/actions/runs/6561052080/job/17819944798)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions/runs/6561050155/job/17819938424)|
|test-conda-python-3.9-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/6561052387/job/17819945715)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #38324:
URL: https://github.com/apache/arrow/pull/38324#issuecomment-1773940690
After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 883a439076233f3f2836f281614ebe01601ccab9.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/17929822108) has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #38324:
URL: https://github.com/apache/arrow/pull/38324#discussion_r1365032217
##########
ci/docker/conda-python.dockerfile:
##########
@@ -23,9 +23,11 @@ FROM ${repo}:${arch}-conda-cpp
ARG python=3.8
COPY ci/conda_env_python.txt \
/arrow/ci/
+# If the Python version being tested is the same as the Python used by the system gdb,
+# we need to install the conda-forge gdb instead.
RUN mamba install -q -y \
--file arrow/ci/conda_env_python.txt \
- gdb \
+ $([ "$python" == "3.10" ] && echo "gdb") \
Review Comment:
Hmm, I'm not sure how to do that in a Dockerfile. Can you help me?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #38324:
URL: https://github.com/apache/arrow/pull/38324#discussion_r1365057834
##########
ci/docker/conda-python.dockerfile:
##########
@@ -23,9 +23,11 @@ FROM ${repo}:${arch}-conda-cpp
ARG python=3.8
COPY ci/conda_env_python.txt \
/arrow/ci/
+# If the Python version being tested is the same as the Python used by the system gdb,
+# we need to install the conda-forge gdb instead.
RUN mamba install -q -y \
--file arrow/ci/conda_env_python.txt \
- gdb \
+ $([ "$python" == "3.10" ] && echo "gdb") \
Review Comment:
Can we try this?
```suggestion
$([ "$python" = $(gdb --batch --eval-command 'python import sys; print(sys.version_info.major, sys.version_info.minor, sep=".")') ] && echo "gdb") \
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou merged PR #38324:
URL: https://github.com/apache/arrow/pull/38324
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #38324:
URL: https://github.com/apache/arrow/pull/38324#discussion_r1364797328
##########
ci/docker/conda-python.dockerfile:
##########
@@ -23,9 +23,11 @@ FROM ${repo}:${arch}-conda-cpp
ARG python=3.8
COPY ci/conda_env_python.txt \
/arrow/ci/
+# If the Python version being tested is the same as the Python used by the system gdb,
+# we need to install the conda-forge gdb instead.
RUN mamba install -q -y \
--file arrow/ci/conda_env_python.txt \
- gdb \
+ $([ "$python" == "3.10" ] && echo "gdb") \
Review Comment:
Will this be broken when we change the base Ubuntu version?
https://github.com/apache/arrow/blob/63840f297df681e710a0b1a822235eca54f1c2fb/ci/docker/conda.dockerfile#L19
Can we detect the Python version that is used by system GDB?
`gdb --batch --eval-command 'python import sys; print(sys.version_info.major, sys.version_info.minor, sep=".")'`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #38324:
URL: https://github.com/apache/arrow/pull/38324#issuecomment-1768287086
@github-actions crossbow submit test-conda-python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #38324:
URL: https://github.com/apache/arrow/pull/38324#issuecomment-1768372694
@github-actions crossbow submit test-conda-python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38324:
URL: https://github.com/apache/arrow/pull/38324#issuecomment-1768291299
Revision: 45bfefb1ca4ab38951c992bd8165c1576fb94394
Submitted crossbow builds: [ursacomputing/crossbow @ actions-598890dc9b](https://github.com/ursacomputing/crossbow/branches/all?query=actions-598890dc9b)
|Task|Status|
|----|------|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions/runs/6560464522/job/17818146769)|
|test-conda-python-3.10-cython2|[](https://github.com/ursacomputing/crossbow/actions/runs/6560465737/job/17818150714)|
|test-conda-python-3.10-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions/runs/6560463118/job/17818141863)|
|test-conda-python-3.10-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions/runs/6560465992/job/17818151682)|
|test-conda-python-3.10-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/6560464124/job/17818145198)|
|test-conda-python-3.10-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions/runs/6560464287/job/17818146033)|
|test-conda-python-3.10-spark-v3.5.0|[](https://github.com/ursacomputing/crossbow/actions/runs/6560462463/job/17818139971)|
|test-conda-python-3.10-substrait|[](https://github.com/ursacomputing/crossbow/actions/runs/6560461951/job/17818138595)|
|test-conda-python-3.11|[](https://github.com/ursacomputing/crossbow/actions/runs/6560461240/job/17818136689)|
|test-conda-python-3.11-dask-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/6560463834/job/17818144364)|
|test-conda-python-3.11-dask-upstream_devel|[](https://github.com/ursacomputing/crossbow/actions/runs/6560463053/job/17818141692)|
|test-conda-python-3.11-hypothesis|[](https://github.com/ursacomputing/crossbow/actions/runs/6560465016/job/17818148189)|
|test-conda-python-3.11-pandas-upstream_devel|[](https://github.com/ursacomputing/crossbow/actions/runs/6560463654/job/17818143632)|
|test-conda-python-3.11-spark-master|[](https://github.com/ursacomputing/crossbow/actions/runs/6560460971/job/17818136164)|
|test-conda-python-3.12|[](https://github.com/ursacomputing/crossbow/actions/runs/6560463356/job/17818142581)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions/runs/6560465242/job/17818149047)|
|test-conda-python-3.8-pandas-1.0|[](https://github.com/ursacomputing/crossbow/actions/runs/6560461616/job/17818137564)|
|test-conda-python-3.8-spark-v3.5.0|[](https://github.com/ursacomputing/crossbow/actions/runs/6560462146/job/17818139075)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions/runs/6560466280/job/17818152455)|
|test-conda-python-3.9-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/6560465519/job/17818149961)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #38324:
URL: https://github.com/apache/arrow/pull/38324#issuecomment-1768350135
Hmm, things are not so simple. All builds pass _except_ when the Python version under test is the same as the one that the system gdb is linked against, in which case the tests fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #38324:
URL: https://github.com/apache/arrow/pull/38324#issuecomment-1770212630
> Do you know why?
I honestly don't understand why. But it only happens when testing Python 3.10 (which is also the Python version installed under conda-forge)...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #38324:
URL: https://github.com/apache/arrow/pull/38324#issuecomment-1770274073
@github-actions crossbow submit test-conda-python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-38323: [CI][Python] Use system gdb on test-conda-python [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38324:
URL: https://github.com/apache/arrow/pull/38324#issuecomment-1770279122
Revision: 53a188778414a589e8d26fb066c1a09ad5e9ed49
Submitted crossbow builds: [ursacomputing/crossbow @ actions-8f00837e89](https://github.com/ursacomputing/crossbow/branches/all?query=actions-8f00837e89)
|Task|Status|
|----|------|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions/runs/6571639129/job/17851191035)|
|test-conda-python-3.10-cython2|[](https://github.com/ursacomputing/crossbow/actions/runs/6571635983/job/17851181715)|
|test-conda-python-3.10-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions/runs/6571640889/job/17851196510)|
|test-conda-python-3.10-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions/runs/6571641546/job/17851198317)|
|test-conda-python-3.10-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/6571639427/job/17851192194)|
|test-conda-python-3.10-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions/runs/6571641508/job/17851198330)|
|test-conda-python-3.10-spark-v3.5.0|[](https://github.com/ursacomputing/crossbow/actions/runs/6571638673/job/17851189586)|
|test-conda-python-3.10-substrait|[](https://github.com/ursacomputing/crossbow/actions/runs/6571637340/job/17851185738)|
|test-conda-python-3.11|[](https://github.com/ursacomputing/crossbow/actions/runs/6571636292/job/17851182362)|
|test-conda-python-3.11-dask-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/6571639692/job/17851192890)|
|test-conda-python-3.11-dask-upstream_devel|[](https://github.com/ursacomputing/crossbow/actions/runs/6571637090/job/17851185216)|
|test-conda-python-3.11-hypothesis|[](https://github.com/ursacomputing/crossbow/actions/runs/6571637600/job/17851186505)|
|test-conda-python-3.11-pandas-upstream_devel|[](https://github.com/ursacomputing/crossbow/actions/runs/6571636836/job/17851184139)|
|test-conda-python-3.11-spark-master|[](https://github.com/ursacomputing/crossbow/actions/runs/6571635806/job/17851181013)|
|test-conda-python-3.12|[](https://github.com/ursacomputing/crossbow/actions/runs/6571638375/job/17851189047)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions/runs/6571640317/job/17851194613)|
|test-conda-python-3.8-pandas-1.0|[](https://github.com/ursacomputing/crossbow/actions/runs/6571640470/job/17851195242)|
|test-conda-python-3.8-spark-v3.5.0|[](https://github.com/ursacomputing/crossbow/actions/runs/6571638094/job/17851187892)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions/runs/6571636589/job/17851183107)|
|test-conda-python-3.9-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions/runs/6571637882/job/17851187116)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org