You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Joseph Lin (JIRA)" <ji...@apache.org> on 2012/04/27 01:36:47 UTC
[jira] [Created] (SQOOP-478) Sqoop export tool unexpectedly
re-order Oracle table columns
Joseph Lin created SQOOP-478:
--------------------------------
Summary: Sqoop export tool unexpectedly re-order Oracle table columns
Key: SQOOP-478
URL: https://issues.apache.org/jira/browse/SQOOP-478
Project: Sqoop
Issue Type: Bug
Components: connectors/oracle
Affects Versions: 1.3.0
Environment: CDH3 Update 1 (cdh3u1)
Reporter: Joseph Lin
Sqoop export tool (v1.3.0-cdh3u1) when exports a set of files from HDFS back to an Oracle database unexpectedly re-order Oracle table columns (number of columns > 62). As a result, input data fields mismatched table columns.
Step to reproduce:
1) create an Oracle table with 100 columns. For instance, create table lingo (col1 varchar(8), col2 varchar(8), ...., col100 varchar(8));
2) create a text file contains the following line and copy it over to HDFS (/user/lingo/data/lingo.csv)
1,2,3,...,100
3) run sqoop export tool similar to the following -
sqoop export --connect jdbc:oracle:thin:@//dbhost:1521/dbSID
--username scott --password tiger --table lingo --input-fields-terminated-by , --verbose --export-dir /user/lingo/data
One would expect sqoop export tool correctly insert a record, similar to the following -
col1 col2 col3 ... col100
1 2 3 ... 100
Unfortunately, here is the record inserted to Oracle table -
col1 col2 col3 ... col39 col40 col41 ... col99 col100
62 63 64 ... 100 1 2 ... 60 61
Looking at the Sqoop export console, it says -
...
12/04/25 17:27:43 DEBUG orm.ClassWriter: selected columns:
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL62
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL63
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL64
...
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL97
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL98
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL99
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL100
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL1
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL2
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL3
...
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL59
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL60
12/04/25 17:27:43 DEBUG orm.ClassWriter: COL61
Apparently, somewhat Sqoop export tool re-ordered an Oracle table's columns and thus input data fields mismatched the columns.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-478) Sqoop export tool unexpectedly
re-order Oracle table columns
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263352#comment-13263352 ]
Cheolsoo Park commented on SQOOP-478:
-------------------------------------
Hello Joseph,
I believe that this issue is already fixed at SQOOP-414 where the column names are now ordered by COLUMN_ID.
Unfortunately, SQOOP-414 is not ported to CDH3 but is only included in CDH4.
> Sqoop export tool unexpectedly re-order Oracle table columns
> -------------------------------------------------------------
>
> Key: SQOOP-478
> URL: https://issues.apache.org/jira/browse/SQOOP-478
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/oracle
> Affects Versions: 1.3.0
> Environment: CDH3 Update 1 (cdh3u1)
> Reporter: Joseph Lin
> Labels: columns, export, incorrect, oracle, order
>
> Sqoop export tool (v1.3.0-cdh3u1) when exports a set of files from HDFS back to an Oracle database unexpectedly re-order Oracle table columns (number of columns > 62). As a result, input data fields mismatched table columns.
> Step to reproduce:
> 1) create an Oracle table with 100 columns. For instance, create table lingo (col1 varchar(8), col2 varchar(8), ...., col100 varchar(8));
> 2) create a text file contains the following line and copy it over to HDFS (/user/lingo/data/lingo.csv)
> 1,2,3,...,100
> 3) run sqoop export tool similar to the following -
> sqoop export --connect jdbc:oracle:thin:@//dbhost:1521/dbSID
> --username scott --password tiger --table lingo --input-fields-terminated-by , --verbose --export-dir /user/lingo/data
> One would expect sqoop export tool correctly insert a record, similar to the following -
> col1 col2 col3 ... col100
> 1 2 3 ... 100
> Unfortunately, here is the record inserted to Oracle table -
> col1 col2 col3 ... col39 col40 col41 ... col99 col100
> 62 63 64 ... 100 1 2 ... 60 61
> Looking at the Sqoop export console, it says -
> ...
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: selected columns:
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL62
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL63
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL64
> ...
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL97
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL98
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL99
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL100
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL1
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL2
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL3
> ...
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL59
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL60
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL61
> Apparently, somewhat Sqoop export tool re-ordered an Oracle table's columns and thus input data fields mismatched the columns.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (SQOOP-478) Sqoop export tool unexpectedly
re-order Oracle table columns
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park resolved SQOOP-478.
---------------------------------
Resolution: Duplicate
Fix Version/s: 1.4.1-incubating
Assignee: Cheolsoo Park
> Sqoop export tool unexpectedly re-order Oracle table columns
> -------------------------------------------------------------
>
> Key: SQOOP-478
> URL: https://issues.apache.org/jira/browse/SQOOP-478
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/oracle
> Affects Versions: 1.3.0
> Environment: CDH3 Update 1 (cdh3u1)
> Reporter: Joseph Lin
> Assignee: Cheolsoo Park
> Labels: columns, export, incorrect, oracle, order
> Fix For: 1.4.1-incubating
>
>
> Sqoop export tool (v1.3.0-cdh3u1) when exports a set of files from HDFS back to an Oracle database unexpectedly re-order Oracle table columns (number of columns > 62). As a result, input data fields mismatched table columns.
> Step to reproduce:
> 1) create an Oracle table with 100 columns. For instance, create table lingo (col1 varchar(8), col2 varchar(8), ...., col100 varchar(8));
> 2) create a text file contains the following line and copy it over to HDFS (/user/lingo/data/lingo.csv)
> 1,2,3,...,100
> 3) run sqoop export tool similar to the following -
> sqoop export --connect jdbc:oracle:thin:@//dbhost:1521/dbSID
> --username scott --password tiger --table lingo --input-fields-terminated-by , --verbose --export-dir /user/lingo/data
> One would expect sqoop export tool correctly insert a record, similar to the following -
> col1 col2 col3 ... col100
> 1 2 3 ... 100
> Unfortunately, here is the record inserted to Oracle table -
> col1 col2 col3 ... col39 col40 col41 ... col99 col100
> 62 63 64 ... 100 1 2 ... 60 61
> Looking at the Sqoop export console, it says -
> ...
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: selected columns:
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL62
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL63
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL64
> ...
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL97
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL98
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL99
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL100
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL1
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL2
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL3
> ...
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL59
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL60
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL61
> Apparently, somewhat Sqoop export tool re-ordered an Oracle table's columns and thus input data fields mismatched the columns.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (SQOOP-478) Sqoop export tool
unexpectedly re-order Oracle table columns
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263352#comment-13263352 ]
Cheolsoo Park edited comment on SQOOP-478 at 4/27/12 5:40 PM:
--------------------------------------------------------------
Hello Joseph,
This issue is already fixed at SQOOP-414 where the column names are now ordered by COLUMN_ID.
In fact, I verified that the fix is ported to cdh3u3 so you should no longer run into this problem if you upgrade to cdh3u3 or higher.
was (Author: cheolsoo):
Hello Joseph,
I believe that this issue is already fixed at SQOOP-414 where the column names are now ordered by COLUMN_ID.
Unfortunately, SQOOP-414 is not ported to CDH3 but is only included in CDH4.
> Sqoop export tool unexpectedly re-order Oracle table columns
> -------------------------------------------------------------
>
> Key: SQOOP-478
> URL: https://issues.apache.org/jira/browse/SQOOP-478
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/oracle
> Affects Versions: 1.3.0
> Environment: CDH3 Update 1 (cdh3u1)
> Reporter: Joseph Lin
> Labels: columns, export, incorrect, oracle, order
>
> Sqoop export tool (v1.3.0-cdh3u1) when exports a set of files from HDFS back to an Oracle database unexpectedly re-order Oracle table columns (number of columns > 62). As a result, input data fields mismatched table columns.
> Step to reproduce:
> 1) create an Oracle table with 100 columns. For instance, create table lingo (col1 varchar(8), col2 varchar(8), ...., col100 varchar(8));
> 2) create a text file contains the following line and copy it over to HDFS (/user/lingo/data/lingo.csv)
> 1,2,3,...,100
> 3) run sqoop export tool similar to the following -
> sqoop export --connect jdbc:oracle:thin:@//dbhost:1521/dbSID
> --username scott --password tiger --table lingo --input-fields-terminated-by , --verbose --export-dir /user/lingo/data
> One would expect sqoop export tool correctly insert a record, similar to the following -
> col1 col2 col3 ... col100
> 1 2 3 ... 100
> Unfortunately, here is the record inserted to Oracle table -
> col1 col2 col3 ... col39 col40 col41 ... col99 col100
> 62 63 64 ... 100 1 2 ... 60 61
> Looking at the Sqoop export console, it says -
> ...
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: selected columns:
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL62
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL63
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL64
> ...
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL97
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL98
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL99
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL100
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL1
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL2
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL3
> ...
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL59
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL60
> 12/04/25 17:27:43 DEBUG orm.ClassWriter: COL61
> Apparently, somewhat Sqoop export tool re-ordered an Oracle table's columns and thus input data fields mismatched the columns.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira