You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Adil Kaleem (JIRA)" <ji...@apache.org> on 2016/05/15 08:04:12 UTC
[jira] [Commented] (SQOOP-2402) Teradata CURRENT_TIMESTAMP handling
for Incremental Import
[ https://issues.apache.org/jira/browse/SQOOP-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15283807#comment-15283807 ]
Adil Kaleem commented on SQOOP-2402:
------------------------------------
Sanjiv, did you get a workaround to this yet?
> Teradata CURRENT_TIMESTAMP handling for Incremental Import
> ------------------------------------------------------------
>
> Key: SQOOP-2402
> URL: https://issues.apache.org/jira/browse/SQOOP-2402
> Project: Sqoop
> Issue Type: Bug
> Components: connectors
> Affects Versions: 1.4.5
> Reporter: sanjiv singh
>
> Command for incremental import :
> sqoop import --connect jdbc:teradata://XXXXXXXX/Database=XXXXX --driver com.teradata.jdbc.TeraDriver --username XXXXX --password XXXXXX --table Employee --target-dir /user/hive/incremental_emp_table -m 1 --check-column LastUpdated --incremental lastmodified --last-value "2001-12-17 07:36:01.280000"
> Error I am getting :
> Warning: /usr/share/hadoop_echosystem/sqoop-1.4.5//../accumulo does not exist! Accumulo imports will fail.
> Please set $ACCUMULO_HOME to the root of your Accumulo installation.
> Warning: /usr/share/hadoop_echosystem/sqoop-1.4.5//../zookeeper does not exist! Accumulo imports will fail.
> Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
> Note: /tmp/sqoop-cloud/compile/917cdf768aea5267d838a949502ed0d0/Employee.java uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/usr/share/hadoop_echosystem/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/share/hadoop_echosystem/hbase-0.96.1-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/share/hadoop_echosystem/apache-hive-1.0.0-bin/lib/hive-jdbc-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 15/06/16 14:20:16 ERROR manager.SqlManager: SQL exception accessing current timestamp: com.teradata.jdbc.jdbc_4.util.JDBCException: [Teradata Database] [TeraJDBC 14.10.00.26] [Error 3706] [SQLState 42000] Syntax error: expected something between '(' and ')'.
> com.teradata.jdbc.jdbc_4.util.JDBCException: [Teradata Database] [TeraJDBC 14.10.00.26] [Error 3706] [SQLState 42000] Syntax error: expected something between '(' and ')'.
> at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDatabaseSQLException(ErrorFactory.java:307)
> at com.teradata.jdbc.jdbc_4.statemachine.ReceiveInitSubState.action(ReceiveInitSubState.java:109)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.subStateMachine(StatementReceiveState.java:314)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.action(StatementReceiveState.java:202)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementController.runBody(StatementController.java:123)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementController.run(StatementController.java:114)
> at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:384)
> at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:326)
> at com.teradata.jdbc.jdbc_4.TDStatement.doNonPrepExecuteQuery(TDStatement.java:314)
> at com.teradata.jdbc.jdbc_4.TDStatement.executeQuery(TDStatement.java:1091)
> at org.apache.sqoop.manager.SqlManager.getCurrentDbTimestamp(SqlManager.java:960)
> at org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.java:328)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> 15/06/16 14:20:16 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Could not get current time from database
> at org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.java:330)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> I have walked through implementation of method org.apache.sqoop.manager.SqlManager.getCurrentDbTimestamp()
> protected String getCurTimestampQuery() {
> return "SELECT CURRENT_TIMESTAMP()";
> }
> SqlManager uses "SELECT CURRENT_TIMESTAMP();" to get current timestamp which is syntactically incorrect.
> For teradata, It should be "SELECT CURRENT_TIMESTAMP;"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [jira] [Commented] (SQOOP-2402) Teradata CURRENT_TIMESTAMP
handling for Incremental Import
Posted by Shashank Tandon <st...@expedia.com>.
Hi Adil,
Ther is a patch available for this.You can find the link
here.Please let me know if you find any issues in using them.
https://issues.apache.org/jira/browse/SQOOP-2620
Thanks,
Shashank Tandon
On 5/15/16, 1:34 PM, "Adil Kaleem (JIRA)" <ji...@apache.org> wrote:
>
> [
>https://issues.apache.org/jira/browse/SQOOP-2402?page=com.atlassian.jira.p
>lugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15283807#com
>ment-15283807 ]
>
>Adil Kaleem commented on SQOOP-2402:
>------------------------------------
>
>Sanjiv, did you get a workaround to this yet?
>
>> Teradata CURRENT_TIMESTAMP handling for Incremental Import
>> ------------------------------------------------------------
>>
>> Key: SQOOP-2402
>> URL: https://issues.apache.org/jira/browse/SQOOP-2402
>> Project: Sqoop
>> Issue Type: Bug
>> Components: connectors
>> Affects Versions: 1.4.5
>> Reporter: sanjiv singh
>>
>> Command for incremental import :
>> sqoop import --connect jdbc:teradata://XXXXXXXX/Database=XXXXX
>>--driver com.teradata.jdbc.TeraDriver --username XXXXX --password XXXXXX
>> --table Employee --target-dir /user/hive/incremental_emp_table -m 1
>>--check-column LastUpdated --incremental lastmodified --last-value
>>"2001-12-17 07:36:01.280000"
>> Error I am getting :
>> Warning: /usr/share/hadoop_echosystem/sqoop-1.4.5//../accumulo does not
>>exist! Accumulo imports will fail.
>> Please set $ACCUMULO_HOME to the root of your Accumulo installation.
>> Warning: /usr/share/hadoop_echosystem/sqoop-1.4.5//../zookeeper does
>>not exist! Accumulo imports will fail.
>> Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
>> Note:
>>/tmp/sqoop-cloud/compile/917cdf768aea5267d838a949502ed0d0/Employee.java
>>uses or overrides a deprecated API.
>> Note: Recompile with -Xlint:deprecation for details.
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in
>>[jar:file:/usr/share/hadoop_echosystem/hadoop-2.6.0/share/hadoop/common/l
>>ib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>>[jar:file:/usr/share/hadoop_echosystem/hbase-0.96.1-hadoop2/lib/slf4j-log
>>4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>>[jar:file:/usr/share/hadoop_echosystem/apache-hive-1.0.0-bin/lib/hive-jdb
>>c-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> 15/06/16 14:20:16 ERROR manager.SqlManager: SQL exception accessing
>>current timestamp: com.teradata.jdbc.jdbc_4.util.JDBCException:
>>[Teradata Database] [TeraJDBC 14.10.00.26] [Error 3706] [SQLState 42000]
>>Syntax error: expected something between '(' and ')'.
>> com.teradata.jdbc.jdbc_4.util.JDBCException: [Teradata Database]
>>[TeraJDBC 14.10.00.26] [Error 3706] [SQLState 42000] Syntax error:
>>expected something between '(' and ')'.
>> at
>>com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDatabaseSQLException(Error
>>Factory.java:307)
>> at
>>com.teradata.jdbc.jdbc_4.statemachine.ReceiveInitSubState.action(ReceiveI
>>nitSubState.java:109)
>> at
>>com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.subStateMachi
>>ne(StatementReceiveState.java:314)
>> at
>>com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.action(Statem
>>entReceiveState.java:202)
>> at
>>com.teradata.jdbc.jdbc_4.statemachine.StatementController.runBody(Stateme
>>ntController.java:123)
>> at
>>com.teradata.jdbc.jdbc_4.statemachine.StatementController.run(StatementCo
>>ntroller.java:114)
>> at
>>com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:38
>>4)
>> at
>>com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:32
>>6)
>> at
>>com.teradata.jdbc.jdbc_4.TDStatement.doNonPrepExecuteQuery(TDStatement.ja
>>va:314)
>> at
>>com.teradata.jdbc.jdbc_4.TDStatement.executeQuery(TDStatement.java:1091)
>> at
>>org.apache.sqoop.manager.SqlManager.getCurrentDbTimestamp(SqlManager.java
>>:960)
>> at
>>org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.ja
>>va:328)
>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>> 15/06/16 14:20:16 ERROR tool.ImportTool: Encountered IOException
>>running import job: java.io.IOException: Could not get current time from
>>database
>> at
>>org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.ja
>>va:330)
>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>> I have walked through implementation of method
>>org.apache.sqoop.manager.SqlManager.getCurrentDbTimestamp()
>> protected String getCurTimestampQuery() {
>> return "SELECT CURRENT_TIMESTAMP()";
>> }
>> SqlManager uses "SELECT CURRENT_TIMESTAMP();" to get current timestamp
>>which is syntactically incorrect.
>> For teradata, It should be "SELECT CURRENT_TIMESTAMP;"
>
>
>
>--
>This message was sent by Atlassian JIRA
>(v6.3.4#6332)