You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/05/23 09:45:43 UTC
[GitHub] [arrow] AlenkaF opened a new pull request, #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples
AlenkaF opened a new pull request, #13216:
URL: https://github.com/apache/arrow/pull/13216
This PR adds a CI job to test python docstrings with `doctest.`
It can be tested with `docker-compose build conda-python-doctest `.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1138148500
This should be ready to merge.
@raulcd I think Travis failed build is not connected, but am not sure. Can I ask for your opinion on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] amol- closed pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
amol- closed pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
URL: https://github.com/apache/arrow/pull/13216
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] amol- commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
amol- commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1137023792
Seems to be failing some tests at the moment
```
==================================== ERRORS ====================================
_ ERROR collecting opt/conda/envs/arrow/lib/python3.9/site-packages/pyarrow/cuda.py _
opt/conda/envs/arrow/lib/python3.9/site-packages/pyarrow/cuda.py:21: in <module>
from pyarrow._cuda import (Context, IpcMemHandle, CudaBuffer,
E ModuleNotFoundError: No module named 'pyarrow._cuda'
_ ERROR collecting opt/conda/envs/arrow/lib/python3.9/site-packages/pyarrow/substrait.py _
opt/conda/envs/arrow/lib/python3.9/site-packages/pyarrow/substrait.py:18: in <module>
from pyarrow._substrait import ( # noqa
E ModuleNotFoundError: No module named 'pyarrow._substrait'
_ ERROR collecting opt/conda/envs/arrow/lib/python3.9/site-packages/pyarrow/tests/deserialize_buffer.py _
opt/conda/envs/arrow/lib/python3.9/site-packages/pyarrow/tests/deserialize_buffer.py:24: in <module>
with open(sys.argv[1], 'rb') as f:
E FileNotFoundError: [Errno 2] No such file or directory: '-r'
_ ERROR collecting opt/conda/envs/arrow/lib/python3.9/site-packages/pyarrow/tests/read_record_batch.py _
opt/conda/envs/arrow/lib/python3.9/site-packages/pyarrow/tests/read_record_batch.py:24: in <module>
with open(sys.argv[1], 'rb') as f:
E FileNotFoundError: [Errno 2] No such file or directory: '-r'
=============================== warnings summary ===============================
```
I don't think we should be using `sys.argv` in the examples, as that will be the arguments of `pytest`, not the arguments of the example.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
raulcd commented on code in PR #13216:
URL: https://github.com/apache/arrow/pull/13216#discussion_r880506701
##########
docker-compose.yml:
##########
@@ -1055,12 +1055,14 @@ services:
LANG: "C.UTF-8"
BUILD_DOCS_CPP: "ON"
BUILD_DOCS_PYTHON: "ON"
+ PYTEST_ARGS: "--doctest-modules --doctest-cython"
volumes: *conda-volumes
command:
["/arrow/ci/scripts/cpp_build.sh /arrow /build &&
/arrow/ci/scripts/python_build.sh /arrow /build &&
pip install -e /arrow/dev/archery[numpydoc] &&
- archery numpydoc --allow-rule PR01,PR10"]
+ archery numpydoc --allow-rule PR01,PR10
Review Comment:
I think the `&&` will be required to be treated as two different commands. Otherwise both lines will be treated as a single command:
```suggestion
archery numpydoc --allow-rule PR01,PR10 &&
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1138543402
Benchmark runs are scheduled for baseline = 841e905fb461561179f44ad6705146a9939613bc and contender = 7fda23c377657b9004317ebdf2524ee13f783fd3. 7fda23c377657b9004317ebdf2524ee13f783fd3 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/7b38919e896443699f236764970deb49...4429663aadcc4133adb9d624277252a9/)
[Failed :arrow_down:1.24% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/1e4e601d0a8b4b73b86a08a0727df19e...683dcf80d7754eb9a6f51ab77ad05ac8/)
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a1d8819ea221466a9954bf671129a833...df4fa35aa8fe4bbb895eff6afdfa40a8/)
[Finished :arrow_down:0.99% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/3339b5bdca334a9489bca3c5b7084da8...22fb22b1073247078498428daee48000/)
Buildkite builds:
[Finished] [`7fda23c3` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/828)
[Finished] [`7fda23c3` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/827)
[Finished] [`7fda23c3` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/817)
[Finished] [`7fda23c3` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/831)
[Finished] [`841e905f` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/827)
[Failed] [`841e905f` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/826)
[Finished] [`841e905f` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/816)
[Finished] [`841e905f` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/830)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1135452250
CI shows:
```
ValueError: Found errors with docker-compose:
- Service `conda-python-doctest` is defined in `services` but not in `x-hierarchy`
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1135641181
I get this error locally also, not sure if I have to add permission on the file?
```
#6 [3/3] RUN /arrow/ci/scripts/install_doctest_cython.sh ${doctest-cython}
#6 sha256:ee58eeb773027f937695e90f7996eddb8d5709efed5beae96508fe15beda15bd
#6 0.594 mesg: ttyname failed: Inappropriate ioctl for device
#6 1.628 /bin/bash: /arrow/ci/scripts/install_doctest_cython.sh: Permission denied
#6 ERROR: executor failed running [/bin/bash -c -l /arrow/ci/scripts/install_doctest_cython.sh ${doctest-cython}]: exit code: 126
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1135742459
> I get this error locally also, not sure if I have to add permission on the file?
>
I've been able to fix it by changing the permissions from the file and make it executable:
```
$ cd ci/scripts/
$ chmod +x install_doctest_cython.sh
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1135751965
> I am wondering if we can include the doctests in the existing sphinx doc build as a separate step, to avoid adding yet another build in the regular CI (also, setting up the docker and building arrow/pyarrow takes longer as the actual tests). Although for local use, it might be nice you can run it separately.
>
> cc @raulcd
That is a pretty good point. I see benefits and drawbacks on both cases but probably at the point we are right now, were we struggle to get runners on GitHub and docker cache is not working correctly, it might make more sense to add an extra step to the existing Sphinx docs build instead of creating a new CI job as you are suggesting. We have a couple of ideas in mind to extend the capacity of our GitHub runners and improve the Docker caching but they will take some time to be done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1137035155
Yes, those failures are handled in https://github.com/apache/arrow/pull/13199 already. The current plan is to merge that first (soon, once CI is green again), and then update this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1134468733
https://issues.apache.org/jira/browse/ARROW-16018
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1135454063
I am wondering if we can include the doctests in the existing sphinx doc build as a separate step, to avoid adding yet another build in the regular CI (also, setting up the docker and building arrow/pyarrow takes longer as the actual tests). Although for local use, it might be nice you can run it separately.
cc @raulcd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1138269434
> This should be ready to merge. @raulcd I think Travis failed build is not connected, but am not sure. Can I ask for your opinion on this?
The failures on travis don't seem related to the change to me either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13216:
URL: https://github.com/apache/arrow/pull/13216#issuecomment-1135460968
Planning to correct the CI errors asap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org