You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Jason Withrow <JW...@Acadian-Asset.com.INVALID> on 2021/09/13 01:06:59 UTC
arrow_iterator.cpython-38-darwin.so - the developer cannot be
verified
Greetings,
Apologies in advance if this is the wrong forum to raise this issue. I would be happy to file a bug in Jira If more appropriate. I am experiencing issues accessing files over NFS from Big Sure with pyarrow 5.0.0. I am running an arm chip, in case that matters. Pyarrow 4.0.1 does work, but when I update to 5.0.0 I immediately receive an alert from the crowdstrike falcon sensor noting that the arrow_iterator.cypthon-38-darwin.so file cannot be opened because the developer cannot be verified. This issue does not manifest when run on a host that is not enrolled in CrowdStrike, nor with pyarrow 4.0.1.
[Graphical user interface Description automatically generated]
Perhaps unrelated, but executing pyarrow.parquet.read_table against an NFS source in pyarrow 5.0.0 yields the following exception: (pyarrow 4.0.1 succeeds with the same sample)
~/gitlab/cache-access/pyenv/lib/python3.8/site-packages/pyarrow/parquet.py in read(self, columns, use_threads, use_pandas_metadata)
1742 use_threads = False
1743
-> 1744 table = self._dataset.to_table(
1745 columns=columns, filter=self._filter_expression,
1746 use_threads=use_threads
~/gitlab/cache-access/pyenv/lib/python3.8/site-packages/pyarrow/_dataset.pyx in pyarrow._dataset.Dataset.to_table()
~/gitlab/cache-access/pyenv/lib/python3.8/site-packages/pyarrow/_dataset.pyx in pyarrow._dataset.Scanner.to_table()
~/gitlab/cache-access/pyenv/lib/python3.8/site-packages/pyarrow/error.pxi in pyarrow.lib.pyarrow_internal_check_status()
~/gitlab/cache-access/pyenv/lib/python3.8/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status()
OSError: [Errno 25] fcntl(fd, F_RDADVISE, ...) failed. Detail: [errno 25] Inappropriate ioctl for device
A sample call is probably not important; but just in case, a minimalist example is below. Removing filters makes no difference in my experience.
import pandas as pd
import pyarrow.parquet as pq
from datetime import date
cols = 'RefIssuerID DateActive DateInactive Name CountryOfRisk PrimaryRefIdentifierID PricingSourceID'.split()
df = pq.read_table('/mnt/dataCacheA/phoenix/v2/reference/issuer/',
filters=[('CountryOfRisk', 'in', ['RU', 'CN']),
('RefIssuerID', 'not in', ['2477', '2771', '116214', '55955']),
('DateActive', '>=', date(2021, 2, 1)),
('DateInactive', '==', date(9999, 12, 31)) ],
columns=cols)
df = df.to_pandas()
Thanks,
Jay
________________________________
This e-Mail and any attachments may contain privileged and confidential information. If you are not the intended recipient or have received this e-mail in error, please notify the sender immediately and destroy/delete this e-mail. You are hereby notified that any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly prohibited. Confidentiality and privilege are not lost by this transmission having been sent or passed on to you in error. Acadian is not liable for any damage that may be caused by viruses or transmission errors.
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any security or an offer to establish an account. All information contained in this communication is not warranted as to completeness or accuracy and is subject to change without notice.
Acadian Asset Management LLC is registered as an investment adviser with the U.S. Securities and Exchange Commission. Registered Office: 260 Franklin Street, Boston, Massachusetts 02110. Our Data Protection Notice can be found at www.acadian-asset.com. Should you no longer wish to receive marketing-related materials from Acadian please email OptOut@acadian-asset.com.
________________________________
Re: [Errno 25] fcntl(fd, F_RDADVISE, ...) failed
Posted by Antoine Pitrou <an...@python.org>.
Le 13/09/2021 à 15:20, Antoine Pitrou a écrit :
>
>> Perhaps unrelated, but executing pyarrow.parquet.read_table against an
>> NFS source in pyarrow 5.0.0 yields the following exception: (pyarrow
>> 4.0.1 succeeds with the same sample)
>>
> [...]
>>
>> OSError: [Errno 25] fcntl(fd, F_RDADVISE, ...) failed. Detail: [errno
>> 25] Inappropriate ioctl for device
>
> Thanks for the report. I'll try to look into it.
I created https://issues.apache.org/jira/browse/ARROW-13983 and
submitted a PR for it which should ignore the (innocuous) error.
Regards
Antoine.
Re: arrow_iterator.cpython-38-darwin.so - the developer cannot be
verified
Posted by Jason Withrow <JW...@Acadian-Asset.com.INVALID>.
Oh, I bet it is snowflake-connector! Apologies for the misunderstanding.
On 9/13/21, 9:21 AM, "Antoine Pitrou" <an...@python.org> wrote:
Hello Jason,
Le 13/09/2021 à 03:06, Jason Withrow a écrit :
> Apologies in advance if this is the wrong forum to raise this issue. I
> would be happy to file a bug in Jira If more appropriate. I am
> experiencing issues accessing files over NFS from Big Sure with pyarrow
> 5.0.0. I am running an arm chip, in case that matters. Pyarrow 4.0.1
> does work, but when I update to 5.0.0 I immediately receive an alert
> from the crowdstrike falcon sensor noting that the
> arrow_iterator.cypthon-38-darwin.so file cannot be opened because the
> developer cannot be verified. This issue does not manifest when run on a
> host that is not enrolled in CrowdStrike, nor with pyarrow 4.0.1.
We do not provide a file named "arrow_iterator.<something>". It's
probably coming from some other package or library, but which one?
> Perhaps unrelated, but executing pyarrow.parquet.read_table against an
> NFS source in pyarrow 5.0.0 yields the following exception: (pyarrow
> 4.0.1 succeeds with the same sample)
>
[...]
>
> OSError: [Errno 25] fcntl(fd, F_RDADVISE, ...) failed. Detail: [errno
> 25] Inappropriate ioctl for device
Thanks for the report. I'll try to look into it.
Regards
Antoine.
________________________________
This e-Mail and any attachments may contain privileged and confidential information. If you are not the intended recipient or have received this e-mail in error, please notify the sender immediately and destroy/delete this e-mail. You are hereby notified that any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly prohibited. Confidentiality and privilege are not lost by this transmission having been sent or passed on to you in error. Acadian is not liable for any damage that may be caused by viruses or transmission errors.
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any security or an offer to establish an account. All information contained in this communication is not warranted as to completeness or accuracy and is subject to change without notice.
Acadian Asset Management LLC is registered as an investment adviser with the U.S. Securities and Exchange Commission. Registered Office: 260 Franklin Street, Boston, Massachusetts 02110. Our Data Protection Notice can be found at www.acadian-asset.com. Should you no longer wish to receive marketing-related materials from Acadian please email OptOut@acadian-asset.com.
________________________________
Re: arrow_iterator.cpython-38-darwin.so - the developer cannot be
verified
Posted by Antoine Pitrou <an...@python.org>.
Hello Jason,
Le 13/09/2021 à 03:06, Jason Withrow a écrit :
> Apologies in advance if this is the wrong forum to raise this issue. I
> would be happy to file a bug in Jira If more appropriate. I am
> experiencing issues accessing files over NFS from Big Sure with pyarrow
> 5.0.0. I am running an arm chip, in case that matters. Pyarrow 4.0.1
> does work, but when I update to 5.0.0 I immediately receive an alert
> from the crowdstrike falcon sensor noting that the
> arrow_iterator.cypthon-38-darwin.so file cannot be opened because the
> developer cannot be verified. This issue does not manifest when run on a
> host that is not enrolled in CrowdStrike, nor with pyarrow 4.0.1.
We do not provide a file named "arrow_iterator.<something>". It's
probably coming from some other package or library, but which one?
> Perhaps unrelated, but executing pyarrow.parquet.read_table against an
> NFS source in pyarrow 5.0.0 yields the following exception: (pyarrow
> 4.0.1 succeeds with the same sample)
>
[...]
>
> OSError: [Errno 25] fcntl(fd, F_RDADVISE, ...) failed. Detail: [errno
> 25] Inappropriate ioctl for device
Thanks for the report. I'll try to look into it.
Regards
Antoine.