You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by David Robson <da...@quest.com> on 2012/11/26 05:57:14 UTC
Review Request: Connection parameters should be used on the mapper
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8221/
-----------------------------------------------------------
Review request for Sqoop.
Description
-------
This patch makes DBConfiguration add the parameters to the Configuration object under the parameter mapreduce.jdbc.params.
The mappers can then get the parameters from this Configuration parameter.
It looks like there needs to be a few things re-factored but I left the code as-is because this will probably be reworked in Sqoop 2.
I left the old methods there for backwards compatibility - but it means there is a lot of overloaded versions of configureDB at the moment.
This addresses bug SQOOP-382.
https://issues.apache.org/jira/browse/SQOOP-382
Diffs
-----
src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java 9e5f102
src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java bd52f00
src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java 21cb128
src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java c17b4bb
src/java/org/apache/sqoop/mapreduce/MySQLDumpImportJob.java 634bd34
src/java/org/apache/sqoop/mapreduce/MySQLExportJob.java 0523901
src/java/org/apache/sqoop/mapreduce/PGBulkloadExportJob.java f3f094b
src/java/org/apache/sqoop/mapreduce/db/DBConfiguration.java 22993df
src/test/com/cloudera/sqoop/manager/PGBulkloadManagerManualTest.java fff35dc
src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 6b4214c
src/test/org/apache/sqoop/mapreduce/db/TestDBConfiguration.java PRE-CREATION
Diff: https://reviews.apache.org/r/8221/diff/
Testing
-------
Added a unit test for the methods to put/get the properties into the Configuration object.
Did manual testing against Oracle database - the connection parameters are now used for all connections and not just the initial one.
Thanks,
David Robson
Re: Review Request: Connection parameters should be used on the mapper
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8221/#review13733
-----------------------------------------------------------
Ship it!
Seems pretty good to me. Please attach the patch to the JIRA and I'll commit it.
Jarcec
- Jarek Cecho
On Nov. 26, 2012, 4:57 a.m., David Robson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8221/
> -----------------------------------------------------------
>
> (Updated Nov. 26, 2012, 4:57 a.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> This patch makes DBConfiguration add the parameters to the Configuration object under the parameter mapreduce.jdbc.params.
> The mappers can then get the parameters from this Configuration parameter.
> It looks like there needs to be a few things re-factored but I left the code as-is because this will probably be reworked in Sqoop 2.
> I left the old methods there for backwards compatibility - but it means there is a lot of overloaded versions of configureDB at the moment.
>
>
> This addresses bug SQOOP-382.
> https://issues.apache.org/jira/browse/SQOOP-382
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java 9e5f102
> src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java bd52f00
> src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java 21cb128
> src/java/org/apache/sqoop/mapreduce/JdbcUpsertExportJob.java c17b4bb
> src/java/org/apache/sqoop/mapreduce/MySQLDumpImportJob.java 634bd34
> src/java/org/apache/sqoop/mapreduce/MySQLExportJob.java 0523901
> src/java/org/apache/sqoop/mapreduce/PGBulkloadExportJob.java f3f094b
> src/java/org/apache/sqoop/mapreduce/db/DBConfiguration.java 22993df
> src/test/com/cloudera/sqoop/manager/PGBulkloadManagerManualTest.java fff35dc
> src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 6b4214c
> src/test/org/apache/sqoop/mapreduce/db/TestDBConfiguration.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/8221/diff/
>
>
> Testing
> -------
>
> Added a unit test for the methods to put/get the properties into the Configuration object.
> Did manual testing against Oracle database - the connection parameters are now used for all connections and not just the initial one.
>
>
> Thanks,
>
> David Robson
>
>