[jira] [Commented] (SQOOP-423) Sqoop import of timestamps to Avro from Postgres - Timezone Issue

["Marcio Silva (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SQOOP-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504660#comment-13504660 ] 

Marcio Silva commented on SQOOP-423:
------------------------------------

A workaround you can try is passing the user.timezone property as a mapreduce child JVM option. We were seeing a similar error with Oracle Date mappings in Avro and we "resolved" it by specifying the system timezone that was being used to create the java.sql.TimeStamp's.

The HADOOP_OPTS setting only makes the change in the job submission process, but as the actual code is being run on the cluster, you need to modify the settings there.

{noformat}  
  -D mapred.child.java.opts=" -Duser.timezone=GMT"
{noformat}

                
> Sqoop import of timestamps to Avro from Postgres - Timezone Issue
> -----------------------------------------------------------------
>
>                 Key: SQOOP-423
>                 URL: https://issues.apache.org/jira/browse/SQOOP-423
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.3.0
>            Reporter: Lynn Goh
>
> I am running sqoop-1.3.0-cdh3u2 on a Mac and when I sqoop import from a postgres table with columns of type 'timestamp without time zone', they are converted to longs in the time zone of my local operating system, even after I have started Hadoop up with TZ=GMT or passed in HADOOP_OPTS="-Duser.timezone=GMT".  My ultimate goal is to sqoop import into long representations that are in GMT timezone rather than my operating system's timezone.
> Postgres example:
> {code}
> acamp_id |     start_time      |      end_time       
> ----------+---------------------+---------------------
>        1 | 2008-01-01 00:00:00 | 2011-12-16 00:00:00
> {code}
> After import, you can see the values are 8 hours ahead, even with TZ=GMT and user.timezone set properly (this is the json representation of the parsed imported avro file):
> {code}
> {"acamp_id": 1, "end_time": 1324022400000, "start_time": 1199174400000}
> {code}
> date utility invocation:
> {code}
> lynngoh@unknown:~$ date -u -r 1199174400
> Tue Jan  1 08:00:00 UTC 2008
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira