You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Cheolsoo Park <ch...@cloudera.com> on 2012/04/07 02:30:40 UTC
Review Request: SQOOP-476 If table name is a qualified name,
Sqoop import fails in DB2
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4674/
-----------------------------------------------------------
Review request for Sqoop.
Summary
-------
During the import job, Sqoop uses the following select query:
SELECT <column names> FROM <table name> AS <table name>
This causes a SQL syntax error for DB2 if table name has a dot "." in it.
The changes include:
1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method.
2) Add a unit test that imports data from a table that has a qualified name.
This addresses bug SQOOP-476.
https://issues.apache.org/jira/browse/SQOOP-476
Diffs
-----
/src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1310644
/src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1310644
/src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION
/src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION
/src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION
/src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1310644
/src/java/org/apache/sqoop/manager/Db2Manager.java 1310644
Diff: https://reviews.apache.org/r/4674/diff
Testing
-------
Thanks,
Cheolsoo
Re: Review Request: SQOOP-476 If table name is a qualified name,
Sqoop import fails in DB2
Posted by Bilung Lee <bl...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4674/#review8293
-----------------------------------------------------------
- Bilung
On 2012-05-27 19:15:12, Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4674/
> -----------------------------------------------------------
>
> (Updated 2012-05-27 19:15:12)
>
>
> Review request for Sqoop.
>
>
> Summary
> -------
>
> During the import job, Sqoop uses the following select query:
>
> SELECT <column names> FROM <table name> AS <table name>
>
> This causes a SQL syntax error for DB2 if table name has a dot "." in it.
>
>
> The changes include:
>
> 1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method.
>
> 2) Add a unit test that imports data from a table that has a qualified name.
>
>
> This addresses bug SQOOP-476.
> https://issues.apache.org/jira/browse/SQOOP-476
>
>
> Diffs
> -----
>
> /src/java/org/apache/sqoop/manager/Db2Manager.java 1343070
> /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1343070
> /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1343070
> /src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION
> /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION
> /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION
> /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1343070
>
> Diff: https://reviews.apache.org/r/4674/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Cheolsoo
>
>
Re: Review Request: SQOOP-476 If table name is a qualified name,
Sqoop import fails in DB2
Posted by Cheolsoo Park <ch...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4674/
-----------------------------------------------------------
(Updated 2012-05-27 19:15:12.870398)
Review request for Sqoop.
Changes
-------
Extended the test cases to verify that both GenericJdbcManager and Db2Manager work.
Summary
-------
During the import job, Sqoop uses the following select query:
SELECT <column names> FROM <table name> AS <table name>
This causes a SQL syntax error for DB2 if table name has a dot "." in it.
The changes include:
1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method.
2) Add a unit test that imports data from a table that has a qualified name.
This addresses bug SQOOP-476.
https://issues.apache.org/jira/browse/SQOOP-476
Diffs (updated)
-----
/src/java/org/apache/sqoop/manager/Db2Manager.java 1343070
/src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1343070
/src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1343070
/src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION
/src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION
/src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION
/src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1343070
Diff: https://reviews.apache.org/r/4674/diff
Testing
-------
Thanks,
Cheolsoo
Re: Review Request: SQOOP-476 If table name is a qualified name,
Sqoop import fails in DB2
Posted by Cheolsoo Park <ch...@cloudera.com>.
> On 2012-05-15 22:28:27, Bilung Lee wrote:
> > Thanks for the patch! It looks good overall. One comment though. It seems that two DB2 record readers are
> > provided, but only one of them is actually used by the Db2Manager and is tested by the test case. Unless the
> > generic JDBC manager also works, there is probably no need for the other record reader.
Hello Bilung, thank you very much for pointing that out.
In fact, I've verified that GenericJdbcManager also works with DB2, so I decided to extend the unit tests to test GenericJdbManager as well as Db2Manager. Now I am testing:
1) Non-qualified table name
a) GenericJdbcManager
b) Db2Manager
2) Qualified table name
a) GenericJdbcManager
b) Db2Manager
- Cheolsoo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4674/#review7917
-----------------------------------------------------------
On 2012-05-27 19:15:12, Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4674/
> -----------------------------------------------------------
>
> (Updated 2012-05-27 19:15:12)
>
>
> Review request for Sqoop.
>
>
> Summary
> -------
>
> During the import job, Sqoop uses the following select query:
>
> SELECT <column names> FROM <table name> AS <table name>
>
> This causes a SQL syntax error for DB2 if table name has a dot "." in it.
>
>
> The changes include:
>
> 1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method.
>
> 2) Add a unit test that imports data from a table that has a qualified name.
>
>
> This addresses bug SQOOP-476.
> https://issues.apache.org/jira/browse/SQOOP-476
>
>
> Diffs
> -----
>
> /src/java/org/apache/sqoop/manager/Db2Manager.java 1343070
> /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1343070
> /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1343070
> /src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION
> /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION
> /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION
> /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1343070
>
> Diff: https://reviews.apache.org/r/4674/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Cheolsoo
>
>
Re: Review Request: SQOOP-476 If table name is a qualified name,
Sqoop import fails in DB2
Posted by Bilung Lee <bl...@gmail.com>.
> On 2012-05-15 22:28:27, Bilung Lee wrote:
> > Thanks for the patch! It looks good overall. One comment though. It seems that two DB2 record readers are
> > provided, but only one of them is actually used by the Db2Manager and is tested by the test case. Unless the
> > generic JDBC manager also works, there is probably no need for the other record reader.
>
> Cheolsoo Park wrote:
> Hello Bilung, thank you very much for pointing that out.
>
> In fact, I've verified that GenericJdbcManager also works with DB2, so I decided to extend the unit tests to test GenericJdbManager as well as Db2Manager. Now I am testing:
>
> 1) Non-qualified table name
> a) GenericJdbcManager
> b) Db2Manager
>
> 2) Qualified table name
> a) GenericJdbcManager
> b) Db2Manager
Thanks for verifying this and extending the test cases.
- Bilung
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4674/#review7917
-----------------------------------------------------------
On 2012-05-27 19:15:12, Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4674/
> -----------------------------------------------------------
>
> (Updated 2012-05-27 19:15:12)
>
>
> Review request for Sqoop.
>
>
> Summary
> -------
>
> During the import job, Sqoop uses the following select query:
>
> SELECT <column names> FROM <table name> AS <table name>
>
> This causes a SQL syntax error for DB2 if table name has a dot "." in it.
>
>
> The changes include:
>
> 1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method.
>
> 2) Add a unit test that imports data from a table that has a qualified name.
>
>
> This addresses bug SQOOP-476.
> https://issues.apache.org/jira/browse/SQOOP-476
>
>
> Diffs
> -----
>
> /src/java/org/apache/sqoop/manager/Db2Manager.java 1343070
> /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1343070
> /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1343070
> /src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION
> /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION
> /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION
> /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1343070
>
> Diff: https://reviews.apache.org/r/4674/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Cheolsoo
>
>
Re: Review Request: SQOOP-476 If table name is a qualified name,
Sqoop import fails in DB2
Posted by Bilung Lee <bl...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4674/#review7917
-----------------------------------------------------------
Thanks for the patch! It looks good overall. One comment though. It seems that two DB2 record readers are
provided, but only one of them is actually used by the Db2Manager and is tested by the test case. Unless the
generic JDBC manager also works, there is probably no need for the other record reader.
- Bilung
On 2012-04-07 00:30:40, Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4674/
> -----------------------------------------------------------
>
> (Updated 2012-04-07 00:30:40)
>
>
> Review request for Sqoop.
>
>
> Summary
> -------
>
> During the import job, Sqoop uses the following select query:
>
> SELECT <column names> FROM <table name> AS <table name>
>
> This causes a SQL syntax error for DB2 if table name has a dot "." in it.
>
>
> The changes include:
>
> 1) Create DB2-specific DBInputFormat/DBRecordReader + DataDrivenDBInputFormat. Note that DataDrivenDBRecordReader is not subclassed since DB2-specific logic is implemented directly in its getSelectQuery() method.
>
> 2) Add a unit test that imports data from a table that has a qualified name.
>
>
> This addresses bug SQOOP-476.
> https://issues.apache.org/jira/browse/SQOOP-476
>
>
> Diffs
> -----
>
> /src/java/org/apache/sqoop/mapreduce/db/DBInputFormat.java 1310644
> /src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java 1310644
> /src/java/org/apache/sqoop/mapreduce/db/Db2DBRecordReader.java PRE-CREATION
> /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBInputFormat.java PRE-CREATION
> /src/java/org/apache/sqoop/mapreduce/db/Db2DataDrivenDBRecordReader.java PRE-CREATION
> /src/test/com/cloudera/sqoop/manager/DB2ManagerImportManualTest.java 1310644
> /src/java/org/apache/sqoop/manager/Db2Manager.java 1310644
>
> Diff: https://reviews.apache.org/r/4674/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Cheolsoo
>
>