[jira] [Commented] (SQOOP-3073) SQOOP Export fails randomly

["Raghunath (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SQOOP-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15728908#comment-15728908 ] 

Raghunath commented on SQOOP-3073:
----------------------------------

When we tried SQOOP EXPORT to Oracle. 

All Rows ---1689105
Failed in between as value larger than specified precision allowed for this column. 
So added row_id and found 172200 exported successfully.

Again when export rows 172200 & 172201 it went successfully. 

So splitted HIVE table upto 172200 and above 172200.

When exporting 172200 rows it failed 172177. 
When exporting above 172200 rows it failed in 452729.

When inspecting rows individually also not able to locate any error with data.


SQOOP Statements used.
sqoop export --connect jdbc:oracle:thin:@odevbi-XXXXXXX:1521/XXXXX \
-username YYYY -password YYYY123 \
--table TEST_TABLE \
--input-fields-terminated-by '\001' \
--lines-terminated-by '\n' \
-m 1 \
--input-null-non-string "\\\N" \
--input-null-string "\\\N" \
--direct \
--export-dir /user/hive/warehouse/yyyyy.db/xxxxx

> SQOOP Export fails randomly
> ---------------------------
>
>                 Key: SQOOP-3073
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3073
>             Project: Sqoop
>          Issue Type: Bug
>            Reporter: Raghunath
>
> While running SQOOP Export, failing as 
> ORA-01438: value larger than specified precision allowed for this column
> We tried splitting data and SQOOP, and could able to see rows which is failing. Again if we try to export only failed rows its success.
> We iterated up to failing row, again its failing in different row.
> 2016-12-07 06:53:14,691 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: 
> 2016-12-07 06:53:14,691 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: Exception raised during data export
> 2016-12-07 06:53:14,691 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: 
> 2016-12-07 06:53:14,691 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: Exception: 
> java.io.IOException: java.sql.SQLDataException: ORA-01438: value larger than specified precision allowed for this column
> 	at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:233)
> 	at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46)
> 	at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:658)
> 	at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> 	at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> 	at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:84)
> 	at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> 	at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1707)
> 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLDataException: ORA-01438: value larger than specified precision allowed for this column
> 	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450)
> 	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399)
> 	at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1017)
> 	at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:655)
> 	at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:249)
> 	at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:566)
> 	at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:215)
> 	at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:58)
> 	at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:943)
> 	at oracle.jdbc.driver.OraclePreparedStatement.executeForRowsWithTimeout(OraclePreparedStatement.java:10932)
> 	at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:11043)
> 	at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:244)
> 	at org.apache.sqoop.mapreduce.AsyncSqlOutputFormat$AsyncSqlExecThread.run(AsyncSqlOutputFormat.java:231)
> 2016-12-07 06:53:14,692 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: On input: 172177
JACOB IZBICKI & CO. LTD
101726825708
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
10.0
\N
\N
No Premium
No Premium
\N
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0234375
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
\N
\N
0.0
0.0
0.0
0.0
0.0
0.0
0.0
1905.0
0.0
No Data
2005-05-25 00:00:00.0
2007-05-26 00:00:00.0
\N
2.0
2.0
Low
0.0
Low
1.32
Low
0.0
Low
NONE
Increase price to drive to profitability
NULL
NULL
NULL
NULL
NULL
NULL
NULL
NULL
NULL
NULL
NULL
NULL
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
\N
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
> 2016-12-07 06:53:14,692 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: On input file: hdfs://dnvdevbigdata2.corp.nai.org:8020/user/hive/warehouse/mcafee_masterdatabase.db/wc_output_clv_detail_mdmparent1/000000_0
> 2016-12-07 06:53:14,692 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: At position 82909184
> 2016-12-07 06:53:14,692 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: 
> 2016-12-07 06:53:14,692 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: Currently processing split:
> 2016-12-07 06:53:14,692 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: Paths:/user/hive/warehouse/XXXXXX.db/XXXXXXX/000000_0:0+67108864,/user/hive/warehouse/mcafee_masterdatabase.db/wc_output_clv_detail_mdmparent1/000000_0:67108864+15820814,/user/hive/warehouse/XXXXXX.db/XXXXXX/000002_0:0+67108736
> 2016-12-07 06:53:14,692 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: 
> 2016-12-07 06:53:14,692 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: This issue might not necessarily be caused by current input
> 2016-12-07 06:53:14,692 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: due to the batching nature of export.
> 2016-12-07 06:53:14,692 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: 
> 2016-12-07 06:53:14,692 INFO [Thread-11] org.apache.sqoop.mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)