You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Joy Gao (JIRA)" <ji...@apache.org> on 2017/09/14 20:16:00 UTC

[jira] [Created] (AIRFLOW-1613) Make MySqlToGoogleCloudStorageOperator compaitible with python3

Joy Gao created AIRFLOW-1613:
--------------------------------

             Summary: Make MySqlToGoogleCloudStorageOperator compaitible with python3
                 Key: AIRFLOW-1613
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1613
             Project: Apache Airflow
          Issue Type: Bug
          Components: contrib
            Reporter: Joy Gao


In Python 3, map(...) returns an iterator, which can only be iterated over once. 
Therefore the current implementation will return an empty list after the first iteration of schema:


{code:python}
schema = map(lambda schema_tuple: schema_tuple[0], cursor.description)
        file_no = 0
        tmp_file_handle = NamedTemporaryFile(delete=True)
        tmp_file_handles = {self.filename.format(file_no): tmp_file_handle}

        for row in cursor:
            # Convert datetime objects to utc seconds, and decimals to floats
            row = map(self.convert_types, row)
            row_dict = dict(zip(schema, row))
{code}

Moving it inside the loop for re-use.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)