You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "assignUser (via GitHub)" <gi...@apache.org> on 2023/04/12 11:08:06 UTC
[GitHub] [arrow] assignUser opened a new issue, #35063: [CI][C#] pythonnet requires envvar to be set
assignUser opened a new issue, #35063:
URL: https://github.com/apache/arrow/issues/35063
### Describe the bug, including details regarding any error messages, version, and platform.
https://github.com/ursacomputing/crossbow/actions/runs/4667712326/jobs/8263854333#step:7:67
### Component(s)
C#, Continuous Integration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1505492351
I suppose the error is that `PYTHONNET_PYDLL` isn't correctly set:
```
++ python3 -m find_libpython
+ export PYTHONNET_PYDLL=
+ PYTHONNET_PYDLL=
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1505550230
It seems to be a confusion between brew python and normal python?
```
bash-3.2$ python3.10 -m find_libpython --version
find_libpython 0.3.0
bash-3.2$ python3.10 -m find_libpython --verbose
DEBUG _is_windows = False
DEBUG _is_apple = True
DEBUG _is_mingw = False
DEBUG _is_msys = False
DEBUG _is_posix = True
DEBUG Candidate: /usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/config-3.10-darwin/libpython3.10.dylib
DEBUG Found: /usr/local/Cellar/python@3.10/3.10.10_1/Frameworks/Python.framework/Versions/3.10/Python
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1505523044
The log around the `PYTHONNET_PYDLL` I shared is found on the nuget job: https://github.com/ursacomputing/crossbow/actions/runs/4676295290/jobs/8291018398#step:7:392
which as @assignUser said fails with the same error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1505322416
Thanks, I see the error seems to be this one, adding it for easy of reference:
```
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:02.15] Apache.Arrow.Tests.CDataSchemaPythonTest.ImportSchema [FAIL]
Failed Apache.Arrow.Tests.CDataSchemaPythonTest.ImportSchema [1 ms]
Error Message:
Python.Runtime.BadPythonDllException : Runtime.PythonDLL was not set or does not point to a supported Python runtime DLL. See https://github.com/pythonnet/pythonnet#embedding-python-in-net
---- System.MissingMethodException : Failed to load symbol Py_IncRef: dlsym(RTLD_DEFAULT, Py_IncRef): symbol not found
Stack Trace:
at Python.Runtime.Runtime.TryUsingDll[T](Func`1 op)
at Python.Runtime.Runtime.Initialize(Boolean initSigs)
at Python.Runtime.PythonEngine.Initialize(IEnumerable`1 args, Boolean setSysArgv, Boolean initSigs)
at Python.Runtime.PythonEngine.Initialize(Boolean setSysArgv, Boolean initSigs)
at Python.Runtime.PythonEngine.Initialize()
at Apache.Arrow.Tests.CDataSchemaPythonTest..ctor() in /Users/runner/work/crossbow/crossbow/arrow/csharp/test/Apache.Arrow.Tests/CDataInterfacePythonTests.cs:line 35
at System.RuntimeType.CreateInstanceDefaultCtor(Boolean publicOnly, Boolean wrapExceptions)
----- Inner Stack Trace -----
at Python.Runtime.Platform.PosixLoader.GetFunction(IntPtr dllHandle, String name)
at Python.Runtime.Runtime.Delegates.GetFunctionByName(String functionName, IntPtr libraryHandle)
The active test run was aborted. Reason: Test host process crashed : Unhandled exception. System.InvalidOperationException: GIL must always be released, and it must be released from the same thread that acquired it.
at Python.Runtime.Py.GILState.Finalize()
Failed! - Failed: 1, Passed: 769, Skipped: 0, Total: 770, Duration: 638 ms - Apache.Arrow.Tests.dll (net7.0)
Test Run Aborted with error System.Exception: One or more errors occurred.
---> System.Exception: Unable to read beyond the end of the stream.
at System.IO.BinaryReader.Read7BitEncodedInt()
at System.IO.BinaryReader.ReadString()
at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.LengthPrefixCommunicationChannel.NotifyDataAvailable()
at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.MessageLoopAsync(TcpClient client, ICommunicationChannel channel, Action`1 errorHandler, CancellationToken cancellationToken)
--- End of inner exception stack trace ---.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1504893342
this should have been closed with the merge, right? @kou @assignUser
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1505495081
CC @wjones127
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1505532624
I found the cause of the empty envvar but I don't know what causes it. I ssh'ed into a macos runner and tried to use `find_libpython` but it only openend pythonbahs?? locally it works as expected and just provides the library location.
```
bash-3.2$ python3 -m find_libpython
/Library/Frameworks/Python.framework/Versions/3.11/Pythonbash-3.2$
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wjones127 commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "wjones127 (via GitHub)" <gi...@apache.org>.
wjones127 commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1506013795
Very confused as to how the archery one is failing where it is. When I run this locally:
```
ARCH=arm64v8 archery docker run ubuntu-csharp
```
I find I need to add
```
apt update && apt install -y python3 pip
```
To the docker command. But once I do, the job passes successfully.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1505516709
This also causes the nuget build to fail
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wjones127 closed issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "wjones127 (via GitHub)" <gi...@apache.org>.
wjones127 closed issue #35063: [CI][C#] pythonnet requires envvar to be set
URL: https://github.com/apache/arrow/issues/35063
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1504913149
Ah, yes.
Merge script reported an network error when I merge #35061. So I set milestone manually but forgot to close this .Sorry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou closed issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou closed issue #35063: [CI][C#] pythonnet requires envvar to be set
URL: https://github.com/apache/arrow/issues/35063
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1505086258
This issue was not fixed by the PR, I just opened it while working on the PR. Sorry for the confusion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wjones127 commented on issue #35063: [CI][C#] pythonnet requires envvar to be set
Posted by "wjones127 (via GitHub)" <gi...@apache.org>.
wjones127 commented on issue #35063:
URL: https://github.com/apache/arrow/issues/35063#issuecomment-1506039131
For the verification release, I think why this is messed up in CI and not locally is this line:
https://github.com/apache/arrow/blob/49631057e9cdbf991e11e0be4b9aa0dadf616850/csharp/test/Apache.Arrow.Tests/CDataInterfacePythonTests.cs#L35
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org