You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Bernardo Najlis <b5...@gmail.com> on 2018/03/24 17:46:22 UTC

AIRFLOW-2224: Add support for CSV files in mysql_to_gcs operator

Hey,

I submitted a PR associated with AIRFLOW-2224
<https://github.com/apache/incubator-airflow/pull/3139> last week that
updates the mysql_to_gcs contrib operator, adding support for CSV files
(currently only supports JSON).

One of the Travis CI tests failed and I'm having some challenges
understanding what the problem is: went through the test output log a
couple of times and it seems the issue is with a test completely unrelated
to my changes.

Is this something that anybody has seen before? Currently working to create
a test for my added functionality (which was missing from the original PR)
but I'm getting concerned that this unrelated test that failed before may
fail once again...

Here's a link to the failed test job
<https://travis-ci.org/apache/incubator-airflow/jobs/355158204> and a
summary of (what I think) the error is:


[2018-03-18 23:56:15,972] {models.py:4526} ERROR -
(_mysql_exceptions.IntegrityError) (1062, "Duplicate entry
'example_bash_operator-2018-03-18 23:56:15.000000' for key 'dag_id'") [SQL:
'INSERT INTO dag_run (dag_id, execution_date, start_date, end_date, state,
run_id, external_trigger, conf) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)']
[parameters: ('example_bash_operator', datetime.datetime(2018, 3, 18, 23,
56, 15, tzinfo=datetime.timezone.utc), datetime.datetime(2018, 3, 18, 23,
56, 15, 971501, tzinfo=datetime.timezone.utc), None, 'running',
'my_run2018-03-18T23:56:15.605960', 1, None)]

Any kind of help is greatly appreciated, really interested in getting this
PR into master, and also adding the same functionality (CSV file support)
to multiple other operators.

Thanks!

Bernardo Najlis