[jira] [Commented] (AIRFLOW-5664) postgres_to_gcs operator drops milliseconds from timestamps

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060047#comment-17060047 ] 

ASF GitHub Bot commented on AIRFLOW-5664:
-----------------------------------------

nuclearpinguin commented on pull request #6354: [AIRFLOW-5664] Store timestamps with microseconds precision in GCSToPSQL
URL: https://github.com/apache/airflow/pull/6354
 
 
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> postgres_to_gcs operator drops milliseconds from timestamps
> -----------------------------------------------------------
>
>                 Key: AIRFLOW-5664
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5664
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: gcp, operators
>    Affects Versions: 1.10.5
>            Reporter: Joseph
>            Assignee: Oluwafemi Sule
>            Priority: Blocker
>
> Postgres stores timestamps with microsecond resolution. When using the postgres_to_gcs operator, timestamps are converted to epoch/unix time using the datetime.timetuple() method. This method drops the microseconds and so you'll end up with a storage object that looks like this:
> {code:java}
> {"id": 1, "last_modified": 1571038537.0}
> {"id": 2, "last_modified": 1571038537.0}
> {"id": 3, "last_modified": 1571038537.0}
> {code}
> When it should look like this:
> {code:java}
> {"id": 1, "last_modified": 1571038537.123}
> {"id": 2, "last_modified": 1571038537.400}
> {"id": 3, "last_modified": 1571038537.455}
> {code}
> It would be useful to keep the timestamps' full resolution.
> I believe the same issue may occur with airflow.operators.mysql_to_gcs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)