You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "amoeba (via GitHub)" <gi...@apache.org> on 2024/03/05 22:49:52 UTC
[I] [Python] Error compiling Cython files on Windows during release verification [arrow]
amoeba opened a new issue, #40375:
URL: https://github.com/apache/arrow/issues/40375
### Describe the bug, including details regarding any error messages, version, and platform.
I ran into this while doing release verification for 15.0.1 RC0. The code in the error message is relatively new (PyCapsule) but old enough that it may just be my setup.
On Windows 11 in cmd.exe prompt with Anaconda set up, I ran
```sh
dev\release\verify-release-candidate.bat 15.0.1 0
```
C++ compiled fine and then when it got to building PyArrow, I see:
```sh
-- Running cmake --build for PyArrow
cmake --build . --config release
Microsoft (R) Build Engine version 16.11.2+f32259642 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
1>Checking Build System
Compiling Cython CXX source for _acero...
performance hint: pyarrow\_acero.pyx:511:67: Exception check after calling 'unwrap' will always require the GIL to be
acquired. Declare 'unwrap' as 'noexcept' if you control the definition and you're sure you don't want the function t
o raise exceptions.
performance hint: pyarrow\_acero.pyx:511:36: Exception check after calling 'GetResultValue[shared_ptr[CTable]]' will
always require the GIL to be acquired. Declare 'GetResultValue[shared_ptr[CTable]]' as 'noexcept' if you control the
definition and you're sure you don't want the function to raise exceptions.
Building Custom Rule C:/tmp/arrow-verify-release/apache-arrow-15.0.1/python/CMakeLists.txt
Compiling Cython CXX source for lib...
Error compiling Cython file:
------------------------------------------------------------
...
if schema.release != NULL:
schema.release(schema)
free(schema)
cdef object alloc_c_schema(ArrowSchema** c_schema) noexcept:
^
------------------------------------------------------------
pyarrow\types.pxi:5264:26: noexcept clause is ignored for function returning Python object
Error compiling Cython file:
------------------------------------------------------------
...
if array.release != NULL:
array.release(array)
free(array)
cdef object alloc_c_array(ArrowArray** c_array) noexcept:
^
------------------------------------------------------------
pyarrow\types.pxi:5283:25: noexcept clause is ignored for function returning Python object
Error compiling Cython file:
------------------------------------------------------------
...
if stream.release != NULL:
stream.release(stream)
free(stream)
cdef object alloc_c_stream(ArrowArrayStream** c_stream) noexcept:
^
------------------------------------------------------------
pyarrow\types.pxi:5302:26: noexcept clause is ignored for function returning Python object
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(24
1,5): error MSB8066: Custom build for 'C:\tmp\arrow-verify-release\apache-arrow-15.0.1\python\build\temp.win-amd64-cpyt
hon-38\CMakeFiles\e1d0ace4d7a55ee7d9220e8a19e03e2f\lib_pyx.rule;C:\tmp\arrow-verify-release\apache-arrow-15.0.1\python\
CMakeLists.txt' exited with code 1. [C:\tmp\arrow-verify-release\apache-arrow-15.0.1\python\build\temp.win-amd64-cpytho
n-38\lib_pyx.vcxproj]
```
I noted that the release verification script pins Python 3.8 so right now I'm re-running verification with Python 3.11 in the errors are related to that.
### Component(s)
Python, Release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "mkumatag (via GitHub)" <gi...@apache.org>.
mkumatag commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1983816367
> The other option is to disable the build-isolation, and ensure you have installed the build dependencies up front:
My case is little different, I'm using the [micropipenv](https://github.com/thoth-station/micropipenv) tool to install the packages form the Pipfile.lock got this entry as an indirect dependency for this project. I tried the pinning the cython in the Pipfile itself but couldn't control this version pinning. Not sure what else I need to look into..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1983562182
Googling a bit, I think you should be able to specify "constraints" to pip:
```
# constraints.txt
cython<3.0.9
```
```
$ PIP_CONSTRAINT=constraints.txt pip install pyarrow
```
The other option is to disable the build-isolation, and ensure you have installed the build dependencies up front:
```
pip install cython<3.0.9 numpy setuptools_scm setuptools wheel
pip install pyarrow --no-build-isolation
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "mkumatag (via GitHub)" <gi...@apache.org>.
mkumatag commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1983287929
and also it will be great if we can back port to older branch and cut a release to unblock folks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1983530040
@mkumatag For now, you will have to ensure you are using cython<3.0.9 while installing pyarrow from source.
How are you running into this? Through a `pip install pyarrow`? (I would have to look up how you can pass constraints on build dependencies to pip)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "mkumatag (via GitHub)" <gi...@apache.org>.
mkumatag commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1983285377
any workaround for this one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1981871214
I just merged https://github.com/apache/arrow/pull/40387 which fix the duplicate issue #40386. This one can be closed as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-2013753154
FYI, 15.0.2 was released that includes the fix for this. So the latest pyarrow should again build fine with latest cython
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz closed issue #40375: [Python] Error compiling Cython files on Windows during release verification
URL: https://github.com/apache/arrow/issues/40375
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "mkumatag (via GitHub)" <gi...@apache.org>.
mkumatag commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1984931510
> The PIP_CONSTRAINT approach doesn't work? ( I would assume pipenv is still using pip to actually install the packages, so I would hope it picks up that env variable)
yes, this solution worked for me :) thanks.
```
PIP_CONSTRAINT=constraints.txt micropipenv install
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "amoeba (via GitHub)" <gi...@apache.org>.
amoeba commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1979788914
Using Python 3.11 instead didn't magically fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1981871256
Fixed by https://github.com/apache/arrow/pull/40387
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1983832390
The PIP_CONSTRAINT approach doesn't work? ( I would assume pipenv is still using pip to actually install the packages, so I would hope it picks up that env variable)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Error compiling Cython files on Windows during release verification [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on issue #40375:
URL: https://github.com/apache/arrow/issues/40375#issuecomment-1981813722
It looks like this warning was [added to cython recently](https://github.com/cython/cython/commit/1aec5fe8383a8cc512938f8f0038d6bd4cdae5ff); seems to be present in cython 3.0.9. Since the warning indicates `noexcept` never did anything, I think it's reasonable to just remove the keyword
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org