[jira] [Assigned] (AIRFLOW-1583) PostgreSQL error when pushing a value bigger than 1GB

["Apache Spark (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned AIRFLOW-1583:
-------------------------------------

    Assignee: Holden Karau's magical unicorn  (was: Rémy Frenoy)

> PostgreSQL error when pushing a value bigger than 1GB
> -----------------------------------------------------
>
>                 Key: AIRFLOW-1583
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1583
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: xcom
>    Affects Versions: Airflow 1.8, Airflow 1.7.0
>            Reporter: Rémy Frenoy
>            Assignee: Holden Karau's magical unicorn
>            Priority: Minor
>   Original Estimate: 0.25h
>  Remaining Estimate: 0.25h
>
> When a task Operator or a PythonOperator returns a value, this value is automatically pushed to XCom. This results in a postgresql error when the value is bigger than 1GB :
> ERROR:  invalid memory alloc request size 1073741824
> A size limit could be added on the value size in the xcom-push() function in model.py. When value is higher than the limit size, a default value (maybe a string explaining that the initial value was too high to be saved) could be pushed to XCom instead of the real one, avoiding the postgresql error.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)