You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Yandex Mail <Sk...@ya.ru> on 2013/10/14 10:44:34 UTC
sqopp import issue
Hi all,
I have some issue with sqoop import can you help me with it?
For this command:
sqoop import --connect jdbc:oracle:thin:@//devdb:1521/DEV --username SYSTEM
--table DEV.TEST -P --split-by id --target-dir /user/oracle/sqoopImport/
I got exception:
INFO mapred.JobClient: Task Id : attempt_201309251246_0160_m_000000_2,
Status : FAILED
java.io.IOException: SQLException in nextKeyValue
at
org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.jav
a:266)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTas
k.java:483)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.
java:76)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(Wrapp
edMapper.java:85)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
at
org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64
)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
va:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.sql
Table have 272 348 767 rows
DDL for table:
CREATE TABLE "DEV"."TEST "
( "ID" NUMBER(19,0) NOT NULL ENABLE,
"TIME" TIMESTAMP (6) NOT NULL ENABLE,
"COL1" VARCHAR2(1000) NOT NULL ENABLE,
"COL2" VARCHAR2(1000) NOT NULL ENABLE,
"COL3" VARCHAR2(1000),
"COL4" VARCHAR2(1000),
"COL5" VARCHAR2(1000),
"COL6" VARCHAR2(1000),
"COL7" VARCHAR2(1000),
"COL8" VARCHAR2(1000),
"COL9" VARCHAR2(1000),
"COL10" VARCHAR2(1000),
"COL11" VARCHAR2(1000),
"COL12" VARCHAR2(1000),
"COL13" VARCHAR2(1000),
"COL14" VARCHAR2(1000),
"COL15" VARCHAR2(1000),
"REQUEST " NUMBER(19,0)
)
Re: sqopp import issue
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Ivan,
the ORA-01555 is suggesting that the UNDO log is too small. Are you by any chance also running some queries (SELECT?) against this table? What is the log configured size?
Jarcec
On Thu, Oct 17, 2013 at 12:03:56AM +0700, Skvortsov Ivan wrote:
> Hi Jarcec,
>
> My name is Ivan)
>
> I'm sorry about incomplete log. I found more detailed exception in logs:
>
> ava.sql.SQLException: ORA-01555: snapshot too old: rollback segment number 363 with name "_SYSSMU363_3377139733$" too small
>
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:879)
> at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
> at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
> at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
> at oracle.jdbc.driver.T4CPreparedStatement.fetch(T4CPreparedStatement.java:1119)
> at oracle.jdbc.driver.OracleResultSetImpl.close_or_fetch_from_next(OracleResultSetImpl.java:373)
> at oracle.jdbc.driver.OracleResultSetImpl.next(OracleResultSetImpl.java:277)
> at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:239)
> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:483)
> at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:76)
> at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:85)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
> at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.mapred.Child.main(Child.java:262)
>
> I think it's related with undo log in oracle database. Do you have any recomendation on this case?
>
> 16.10.2013, 22:11, "Jarek Jarcec Cecho" <ja...@apache.org>:
> > Hi Yandex,
> > what Sqoop version are you using?
> >
> > Would you mind sharing with us failed Map task log? It usually contains also the chained exceptions with more information about the failure.
> >
> > Jarcec
> >
> > On Mon, Oct 14, 2013 at 03:44:34PM +0700, Yandex Mail wrote:
> >
> >> Hi all,
> >>
> >> I have some issue with sqoop import can you help me with it?
> >>
> >> For this command:
> >>
> >> sqoop import --connect jdbc:oracle:thin:@//devdb:1521/DEV --username SYSTEM
> >> --table DEV.TEST -P --split-by id --target-dir /user/oracle/sqoopImport/
> >>
> >> I got exception:
> >>
> >> INFO mapred.JobClient: Task Id : attempt_201309251246_0160_m_000000_2,
> >> Status : FAILED
> >>
> >> java.io.IOException: SQLException in nextKeyValue
> >>
> >> at
> >> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.jav
> >> a:266)
> >>
> >> at
> >> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTas
> >> k.java:483)
> >>
> >> at
> >> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.
> >> java:76)
> >>
> >> at
> >> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(Wrapp
> >> edMapper.java:85)
> >>
> >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
> >>
> >> at
> >> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64
> >> )
> >>
> >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
> >>
> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
> >>
> >> at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
> >>
> >> at java.security.AccessController.doPrivileged(Native Method)
> >>
> >> at javax.security.auth.Subject.doAs(Subject.java:396)
> >>
> >> at
> >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
> >> va:1408)
> >>
> >> at org.apache.hadoop.mapred.Child.main(Child.java:262)
> >>
> >> Caused by: java.sql
> >>
> >> Table have 272 348 767 rows
> >>
> >> DDL for table:
> >>
> >> CREATE TABLE "DEV"."TEST "
> >>
> >> ( "ID" NUMBER(19,0) NOT NULL ENABLE,
> >>
> >> "TIME" TIMESTAMP (6) NOT NULL ENABLE,
> >>
> >> "COL1" VARCHAR2(1000) NOT NULL ENABLE,
> >>
> >> "COL2" VARCHAR2(1000) NOT NULL ENABLE,
> >>
> >> "COL3" VARCHAR2(1000),
> >>
> >> "COL4" VARCHAR2(1000),
> >>
> >> "COL5" VARCHAR2(1000),
> >>
> >> "COL6" VARCHAR2(1000),
> >>
> >> "COL7" VARCHAR2(1000),
> >>
> >> "COL8" VARCHAR2(1000),
> >>
> >> "COL9" VARCHAR2(1000),
> >>
> >> "COL10" VARCHAR2(1000),
> >>
> >> "COL11" VARCHAR2(1000),
> >>
> >> "COL12" VARCHAR2(1000),
> >>
> >> "COL13" VARCHAR2(1000),
> >>
> >> "COL14" VARCHAR2(1000),
> >>
> >> "COL15" VARCHAR2(1000),
> >>
> >> "REQUEST " NUMBER(19,0)
> >>
> >> )
Re: sqopp import issue
Posted by Skvortsov Ivan <sk...@yandex.ru>.
Hi Jarcec,
My name is Ivan)
I'm sorry about incomplete log. I found more detailed exception in logs:
ava.sql.SQLException: ORA-01555: snapshot too old: rollback segment number 363 with name "_SYSSMU363_3377139733$" too small
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:879)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.fetch(T4CPreparedStatement.java:1119)
at oracle.jdbc.driver.OracleResultSetImpl.close_or_fetch_from_next(OracleResultSetImpl.java:373)
at oracle.jdbc.driver.OracleResultSetImpl.next(OracleResultSetImpl.java:277)
at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:239)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:483)
at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:76)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:85)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
I think it's related with undo log in oracle database. Do you have any recomendation on this case?
16.10.2013, 22:11, "Jarek Jarcec Cecho" <ja...@apache.org>:
> Hi Yandex,
> what Sqoop version are you using?
>
> Would you mind sharing with us failed Map task log? It usually contains also the chained exceptions with more information about the failure.
>
> Jarcec
>
> On Mon, Oct 14, 2013 at 03:44:34PM +0700, Yandex Mail wrote:
>
>> Hi all,
>>
>> I have some issue with sqoop import can you help me with it?
>>
>> For this command:
>>
>> sqoop import --connect jdbc:oracle:thin:@//devdb:1521/DEV --username SYSTEM
>> --table DEV.TEST -P --split-by id --target-dir /user/oracle/sqoopImport/
>>
>> I got exception:
>>
>> INFO mapred.JobClient: Task Id : attempt_201309251246_0160_m_000000_2,
>> Status : FAILED
>>
>> java.io.IOException: SQLException in nextKeyValue
>>
>> at
>> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.jav
>> a:266)
>>
>> at
>> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTas
>> k.java:483)
>>
>> at
>> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.
>> java:76)
>>
>> at
>> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(Wrapp
>> edMapper.java:85)
>>
>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
>>
>> at
>> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64
>> )
>>
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
>>
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>>
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>>
>> at java.security.AccessController.doPrivileged(Native Method)
>>
>> at javax.security.auth.Subject.doAs(Subject.java:396)
>>
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
>> va:1408)
>>
>> at org.apache.hadoop.mapred.Child.main(Child.java:262)
>>
>> Caused by: java.sql
>>
>> Table have 272 348 767 rows
>>
>> DDL for table:
>>
>> CREATE TABLE "DEV"."TEST "
>>
>> ( "ID" NUMBER(19,0) NOT NULL ENABLE,
>>
>> "TIME" TIMESTAMP (6) NOT NULL ENABLE,
>>
>> "COL1" VARCHAR2(1000) NOT NULL ENABLE,
>>
>> "COL2" VARCHAR2(1000) NOT NULL ENABLE,
>>
>> "COL3" VARCHAR2(1000),
>>
>> "COL4" VARCHAR2(1000),
>>
>> "COL5" VARCHAR2(1000),
>>
>> "COL6" VARCHAR2(1000),
>>
>> "COL7" VARCHAR2(1000),
>>
>> "COL8" VARCHAR2(1000),
>>
>> "COL9" VARCHAR2(1000),
>>
>> "COL10" VARCHAR2(1000),
>>
>> "COL11" VARCHAR2(1000),
>>
>> "COL12" VARCHAR2(1000),
>>
>> "COL13" VARCHAR2(1000),
>>
>> "COL14" VARCHAR2(1000),
>>
>> "COL15" VARCHAR2(1000),
>>
>> "REQUEST " NUMBER(19,0)
>>
>> )
Re: sqopp import issue
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Yandex,
what Sqoop version are you using?
Would you mind sharing with us failed Map task log? It usually contains also the chained exceptions with more information about the failure.
Jarcec
On Mon, Oct 14, 2013 at 03:44:34PM +0700, Yandex Mail wrote:
> Hi all,
>
>
>
> I have some issue with sqoop import can you help me with it?
>
>
>
> For this command:
>
> sqoop import --connect jdbc:oracle:thin:@//devdb:1521/DEV --username SYSTEM
> --table DEV.TEST -P --split-by id --target-dir /user/oracle/sqoopImport/
>
>
>
> I got exception:
>
>
>
> INFO mapred.JobClient: Task Id : attempt_201309251246_0160_m_000000_2,
> Status : FAILED
>
> java.io.IOException: SQLException in nextKeyValue
>
> at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.jav
> a:266)
>
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTas
> k.java:483)
>
> at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.
> java:76)
>
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(Wrapp
> edMapper.java:85)
>
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
>
> at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64
> )
>
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>
> at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:396)
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
> va:1408)
>
> at org.apache.hadoop.mapred.Child.main(Child.java:262)
>
> Caused by: java.sql
>
>
>
> Table have 272 348 767 rows
>
> DDL for table:
>
> CREATE TABLE "DEV"."TEST "
>
> ( "ID" NUMBER(19,0) NOT NULL ENABLE,
>
> "TIME" TIMESTAMP (6) NOT NULL ENABLE,
>
> "COL1" VARCHAR2(1000) NOT NULL ENABLE,
>
> "COL2" VARCHAR2(1000) NOT NULL ENABLE,
>
> "COL3" VARCHAR2(1000),
>
> "COL4" VARCHAR2(1000),
>
> "COL5" VARCHAR2(1000),
>
> "COL6" VARCHAR2(1000),
>
> "COL7" VARCHAR2(1000),
>
> "COL8" VARCHAR2(1000),
>
> "COL9" VARCHAR2(1000),
>
> "COL10" VARCHAR2(1000),
>
> "COL11" VARCHAR2(1000),
>
> "COL12" VARCHAR2(1000),
>
> "COL13" VARCHAR2(1000),
>
> "COL14" VARCHAR2(1000),
>
> "COL15" VARCHAR2(1000),
>
> "REQUEST " NUMBER(19,0)
>
> )
>
>
>
>
>