You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "John (Jira)" <ji...@apache.org> on 2022/10/27 20:46:00 UTC

[jira] [Created] (ARROW-18182) Error: AWS Error [code 99]: curlCode: 28, Timeout was reached

John created ARROW-18182:
----------------------------

             Summary: Error: AWS Error [code 99]: curlCode: 28, Timeout was reached
                 Key: ARROW-18182
                 URL: https://issues.apache.org/jira/browse/ARROW-18182
             Project: Apache Arrow
          Issue Type: Bug
          Components: Parquet
    Affects Versions: 9.0.0
            Reporter: John


We have AWS Lambdas that were updated to run with PyArrow 9.  We were calling the pyarrow.parquet.read_table function with the columns parameter to specify which columns to read.  With PyArrow version prior to 9 this worked fine with just 500MB of memory for the Lambda.  With PyArrow 9 we started to get these exceptions:

AWS Error [code 99]: curlCode: 28, Timeout was reached

We did some experimenting and found two options to get around the issue.

Option 1:  Increase the memory to 1.7 MB so the Lamba gets a full CPU for the container
Option 2:  Continue to allocate just 500 MB of memory, but stop using the columns parameter.

 

Looking for some insight as to what the issue might be.  Thanks.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)