You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Roee Shlomo (Jira)" <ji...@apache.org> on 2020/06/28 19:37:00 UTC
[jira] [Updated] (ARROW-9261) [Python][Packaging] S3FileSystem curl
errors in manylinux wheels
[ https://issues.apache.org/jira/browse/ARROW-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roee Shlomo updated ARROW-9261:
-------------------------------
Description:
https://issues.apache.org/jira/browse/ARROW-9109 introduced S3 support in manylinux wheels. However, when trying to use S3FileSystem it fails with
{code:java}
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "pyarrow/_fs.pyx", line 597, in pyarrow._fs.FileSystem.open_input_stream
File "pyarrow/error.pxi", line 122, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
OSError: When reading information for key 'xxxxx' in bucket 'xxxxx': AWS Error [code 99]: curlCode: 77, Problem with the SSL CA cert (path? access rights?) with address{code}
It seems like it can't find the SSL CA cert directory that is installed in the runtime machine (tested on Ubuntu 16.04 and Ubuntu 18.04). It always searches in /etc/pki/tls/certs/ca-bundle.crt probably because the wheels are built on centos, whereas in Ubuntu the path is /etc/ssl/certs/ca-certificates.crt and is different on other distributions.
Reproduce with:
{code:java}
virtualenv -p python3.8 arrowenv
source arrowenv/bin/activate
pip install --extra-index-url https://repo.fury.io/arrow-nightlies/ --pre pyarrow
python -c "from pyarrow.fs import S3FileSystem; fs = S3FileSystem(); fs.open_input_stream('mybucket/myfile')"{code}
was:
https://issues.apache.org/jira/browse/ARROW-9109 introduced S3 support in manylinux wheels. However, when trying to use S3FileSystem it fails with
{code:java}
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "pyarrow/_fs.pyx", line 597, in pyarrow._fs.FileSystem.open_input_stream
File "pyarrow/error.pxi", line 122, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
OSError: When reading information for key 'xxxxx' in bucket 'xxxxx': AWS Error [code 99]: curlCode: 77, Problem with the SSL CA cert (path? access rights?) with address{code}
It seems like it can't find the SSL CA cert directory that is installed in the runtime machine (tested on Ubuntu 16.04 and Ubuntu 18.04).
Reproduce with:
{code:java}
virtualenv -p python3.8 arrowenv
source arrowenv/bin/activate
pip install --extra-index-url https://repo.fury.io/arrow-nightlies/ --pre pyarrow
python -c "from pyarrow.fs import S3FileSystem; fs = S3FileSystem(); fs.open_input_stream('mybucket/myfile')"{code}
> [Python][Packaging] S3FileSystem curl errors in manylinux wheels
> ----------------------------------------------------------------
>
> Key: ARROW-9261
> URL: https://issues.apache.org/jira/browse/ARROW-9261
> Project: Apache Arrow
> Issue Type: Bug
> Components: Packaging, Python
> Reporter: Roee Shlomo
> Priority: Major
>
> https://issues.apache.org/jira/browse/ARROW-9109 introduced S3 support in manylinux wheels. However, when trying to use S3FileSystem it fails with
>
> {code:java}
> Traceback (most recent call last):
> File "<string>", line 1, in <module>
> File "pyarrow/_fs.pyx", line 597, in pyarrow._fs.FileSystem.open_input_stream
> File "pyarrow/error.pxi", line 122, in pyarrow.lib.pyarrow_internal_check_status
> File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
> OSError: When reading information for key 'xxxxx' in bucket 'xxxxx': AWS Error [code 99]: curlCode: 77, Problem with the SSL CA cert (path? access rights?) with address{code}
> It seems like it can't find the SSL CA cert directory that is installed in the runtime machine (tested on Ubuntu 16.04 and Ubuntu 18.04). It always searches in /etc/pki/tls/certs/ca-bundle.crt probably because the wheels are built on centos, whereas in Ubuntu the path is /etc/ssl/certs/ca-certificates.crt and is different on other distributions.
> Reproduce with:
> {code:java}
> virtualenv -p python3.8 arrowenv
> source arrowenv/bin/activate
> pip install --extra-index-url https://repo.fury.io/arrow-nightlies/ --pre pyarrow
> python -c "from pyarrow.fs import S3FileSystem; fs = S3FileSystem(); fs.open_input_stream('mybucket/myfile')"{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)