[jira] [Created] (ARROW-8435) A TypeError is raised while token expires while writing to S3

["Shawn Li (Jira)" <ji...@apache.org>]

Shawn Li created ARROW-8435:
-------------------------------

             Summary: A TypeError is raised while token expires while writing to S3
                 Key: ARROW-8435
                 URL: https://issues.apache.org/jira/browse/ARROW-8435
             Project: Apache Arrow
          Issue Type: Bug
          Components: Python
    Affects Versions: 0.15.1
            Reporter: Shawn Li


This issue only occurs when a STS token expires *in the middle of* writing to S3. An OSError: Write failed: TypeError("'NoneType' object is not subscriptable",) is raised instead of a PermissionError.

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pyarrow/parquet.py", line 1450, in
write_to_dataset write_table(subtable, f, **kwargs)
File "/usr/local/lib/python3.6/site-packages/pyarrow/parquet.py", line 1344, in
write_table writer.write_table(table, row_group_size=row_group_size)
File "/usr/local/lib/python3.6/site-packages/pyarrow/parquet.py", line 474, in
write_table self.writer.write_table(table, row_group_size=row_group_size)
File "pyarrow/_parquet.pyx", line 1375, in pyarrow._parquet.ParquetWriter.write_table File "pyarrow/error.pxi", line 80, in
pyarrow.lib.check_statuspyarrow.lib.ArrowIOError: Arrow error: IOError: The provided token has expired.. Detail: Python exception: PermissionError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/s3fs/core.py", line 1096, in _upload_chunk PartNumber=part, UploadId=self.mpu['UploadId'],TypeError: 'NoneType' object is not subscriptable

environment is:
s3fs==0.4.0
boto3==1.10.27
botocore==1.13.27
pyarrow==0.15.1



--
This message was sent by Atlassian Jira
(v8.3.4#803005)