[jira] [Created] (AIRFLOW-447) Python 3 map object cannot be json-serialized - need lists instead

["Georg Walther (JIRA)" <ji...@apache.org>]

Georg Walther created AIRFLOW-447:
-------------------------------------

             Summary: Python 3 map object cannot be json-serialized - need lists instead
                 Key: AIRFLOW-447
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-447
             Project: Apache Airflow
          Issue Type: Bug
          Components: contrib
    Affects Versions: Airflow 1.7.1.3
         Environment: Python 3
            Reporter: Georg Walther
            Priority: Blocker


On this line, `source_uris` are generated with `map`:

https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/gcs_to_bq.py#L124

In Python 2 `map` would return a list, however in Python 3 `map` returns a `map` object / generator.
The Python 3 `map` object cannot be json-serialized downstream when storing corresponding job objects in the database:

https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/hooks/bigquery_hook.py#L441

Constructing `source_uris` with a list comprehension instead of `map` resolves the issue for Python 3 and retains compatibility with Python 2.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)