[jira] [Commented] (ARROW-1293) Module initialization error when using pyarrow with AWS Lambda

["Martin Durant (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109813#comment-16109813 ] 

Martin Durant commented on ARROW-1293:
--------------------------------------

This very likely comes from snappy for the specific case of Lambda: https://github.com/andrix/python-snappy/issues/52

> Module initialization error when using pyarrow with AWS Lambda
> --------------------------------------------------------------
>
>                 Key: ARROW-1293
>                 URL: https://issues.apache.org/jira/browse/ARROW-1293
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 0.5.0
>         Environment: AWS Lambda
>            Reporter: Tanja Miličić
>            Assignee: Wes McKinney
>
> When using pyarrow in AWS Lambda function like this:
> {code}
> import pyarrow as pa
> import pyarrow.parquet as pq
> import pandas as pd
> def lambda_handler(event, context):
>     df = pd.DataFrame([data]) #data is dictionary
>     table = pa.Table.from_pandas(df)
>     pq.write_table(table, 'tmp/test.parquet', compression='snappy')
>     table = pq.read_table('tmp/test.parquet')
>     table.to_pandas()
>     print(table)
>     return "Success"
> {code}
> Module initialization error occurs: 
> {code}
> module initialization error: [Errno 2] No such file or directory: '/var/task/__pycache__/_cffi__x762f05ffx6bf5342b.c'
> {code}
> Deployment package was prepared by running following commands:
> {code}
> virtualenv nameofenv
> source nameofenv/bin/active
> pip install pyarrow
> sudo apt-get install libsnappy-dev
> pip install python-snappy
> pip install pandas
> {code}
> files from site-packages directory are than zipped together with lambda function.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)