You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Gautham V <ga...@gmail.com> on 2013/12/12 04:30:56 UTC

Incremental import from Oracle - last-value not getting updated in metastore

Need help on an error encountered during incremental import. The sqoop job
completes and it does a full import rather than doing it incremental
resulting in duplicate records. From what I can deduce, the metastore is
not getting updated with the latest last-value.

Probably, this has been answered before, but I couldn't find it anywhere.
The error stack is below:

 INFO tool.ImportTool: Saving incremental import state to the metastore
 ERROR tool.ImportTool: Encountered IOException running import job:
java.io.IOException: Error communicating with database
        at
org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:426)
        at
org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.update(HsqldbJobStorage.java:445)
        at
org.apache.sqoop.tool.ImportTool.saveIncrementalState(ImportTool.java:131)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:428)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
        at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:228)
        at org.apache.sqoop.tool.JobTool.run(JobTool.java:283)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
Caused by: java.sql.SQLException: Violation of unique constraint
SQOOP_SESSIONS_UNQ: duplicate value(s) for column(s)
JOB_NAME,PROPNAME,PROPCLASS in statement [INSERT INTO SQOOP_SESSIONS
(propval, job_name, propclass, propname) VALUES (?, ?, ?, ?)]
        at org.hsqldb.jdbc.Util.throwError(Unknown Source)
        at org.hsqldb.jdbc.jdbcPreparedStatement.executeUpdate(Unknown
Source)
        at
org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.setV0Property(HsqldbJobStorage.java:707)
        at
org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:415)
        ... 12 more

Sqoop version:
Sqoop 1.4.4-mapr
git commit id 16d0124c5b5f7bc68b8f67fbe77f0c91d46d64c1

Appreciate if someone can throw some light on this.

Thanks.
Gautham

Re: Incremental import from Oracle - last-value not getting updated in metastore

Posted by Abraham Elmahrek <ab...@cloudera.com>.
Gautum,

Would you mind sharing your sqoop command that you are executing as well as
the sqoop logs generated with the --verbose option?

-Abe


On Wed, Dec 11, 2013 at 7:30 PM, Gautham V <ga...@gmail.com> wrote:

> Need help on an error encountered during incremental import. The sqoop job
> completes and it does a full import rather than doing it incremental
> resulting in duplicate records. From what I can deduce, the metastore is
> not getting updated with the latest last-value.
>
> Probably, this has been answered before, but I couldn't find it anywhere.
> The error stack is below:
>
>  INFO tool.ImportTool: Saving incremental import state to the metastore
>  ERROR tool.ImportTool: Encountered IOException running import job:
> java.io.IOException: Error communicating with database
>         at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:426)
>         at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.update(HsqldbJobStorage.java:445)
>          at
> org.apache.sqoop.tool.ImportTool.saveIncrementalState(ImportTool.java:131)
>         at
> org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:428)
>         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
>         at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:228)
>         at org.apache.sqoop.tool.JobTool.run(JobTool.java:283)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> Caused by: java.sql.SQLException: Violation of unique constraint
> SQOOP_SESSIONS_UNQ: duplicate value(s) for column(s)
> JOB_NAME,PROPNAME,PROPCLASS in statement [INSERT INTO SQOOP_SESSIONS
> (propval, job_name, propclass, propname) VALUES (?, ?, ?, ?)]
>         at org.hsqldb.jdbc.Util.throwError(Unknown Source)
>         at org.hsqldb.jdbc.jdbcPreparedStatement.executeUpdate(Unknown
> Source)
>         at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.setV0Property(HsqldbJobStorage.java:707)
>         at
> org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.createInternal(HsqldbJobStorage.java:415)
>         ... 12 more
>
> Sqoop version:
> Sqoop 1.4.4-mapr
> git commit id 16d0124c5b5f7bc68b8f67fbe77f0c91d46d64c1
>
> Appreciate if someone can throw some light on this.
>
> Thanks.
> Gautham
>