You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Bilung Lee <bl...@gmail.com> on 2012/08/28 22:25:38 UTC
Review Request: SQOOP-588 Provide MapReduce classes for executing ETL classes
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6811/
-----------------------------------------------------------
Review request for Sqoop.
Description
-------
This patch provides necessary MapReduce classes (split, input format, mapper, and output format) for executing ETL classes (partition, partitioner, extractor, and loader).
Basically
- partition is used by split
- partitioner is invoked by input format
- extractor is invoked by mapper
- loader is invoked by output format
This addresses bug SQOOP-588.
https://issues.apache.org/jira/browse/SQOOP-588
Diffs
-----
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportDestroyer.java 1373167
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportInitializer.java 1373167
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java 1373167
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportDestroyer.java 1373167
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java 1373167
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportInitializer.java 1373167
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartition.java PRE-CREATION
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java 1373167
branches/sqoop2/core/pom.xml 1373167
branches/sqoop2/core/src/main/java/org/apache/sqoop/core/CoreError.java 1373167
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/JobConstants.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlContext.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlMutableContext.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/Data.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/FieldTypes.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopFileOutputFormat.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopNullOutputFormat.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopSplit.java PRE-CREATION
branches/sqoop2/core/src/test/java/org/apache/sqoop/job/TestMapReduce.java PRE-CREATION
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Context.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Destroyer.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Initializer.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Loader.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Partition.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Partitioner.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataReader.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataWriter.java 1373167
Diff: https://reviews.apache.org/r/6811/diff/
Testing
-------
Create a new TestMapReduce test case
Thanks,
Bilung Lee
Re: Review Request: SQOOP-588 Provide MapReduce classes for executing ETL
classes
Posted by Bilung Lee <bl...@gmail.com>.
> On Aug. 31, 2012, 5:48 a.m., Jarek Cecho wrote:
> > Hi Bilung,
> > thank you very much for your effort here. Please attach most recent version of this patch to JIRA.
> >
> > Jarcec
Thanks for your review! Patch is attached to the JIRA.
- Bilung
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6811/#review10930
-----------------------------------------------------------
On Aug. 28, 2012, 10:08 p.m., Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6811/
> -----------------------------------------------------------
>
> (Updated Aug. 28, 2012, 10:08 p.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> This patch provides necessary MapReduce classes (split, input format, mapper, and output format) for executing ETL classes (partition, partitioner, extractor, and loader).
>
> Basically
> - partition is used by split
> - partitioner is invoked by input format
> - extractor is invoked by mapper
> - loader is invoked by output format
>
>
> This addresses bug SQOOP-588.
> https://issues.apache.org/jira/browse/SQOOP-588
>
>
> Diffs
> -----
>
> branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java 1373167
> branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartition.java PRE-CREATION
> branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java 1373167
> branches/sqoop2/core/pom.xml 1373167
> branches/sqoop2/core/src/main/java/org/apache/sqoop/core/CoreError.java 1373167
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/JobConstants.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlContext.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlMutableContext.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/Data.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/FieldTypes.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopFileOutputFormat.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopNullOutputFormat.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopSplit.java PRE-CREATION
> branches/sqoop2/core/src/test/java/org/apache/sqoop/job/TestMapReduce.java PRE-CREATION
> branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Context.java 1373167
> branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Partition.java 1373167
> branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataReader.java 1373167
> branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataWriter.java 1373167
>
> Diff: https://reviews.apache.org/r/6811/diff/
>
>
> Testing
> -------
>
> Create a new TestMapReduce test case
>
>
> Thanks,
>
> Bilung Lee
>
>
Re: Review Request: SQOOP-588 Provide MapReduce classes for executing ETL
classes
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6811/#review10930
-----------------------------------------------------------
Ship it!
Hi Bilung,
thank you very much for your effort here. Please attach most recent version of this patch to JIRA.
Jarcec
- Jarek Cecho
On Aug. 28, 2012, 10:08 p.m., Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6811/
> -----------------------------------------------------------
>
> (Updated Aug. 28, 2012, 10:08 p.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> This patch provides necessary MapReduce classes (split, input format, mapper, and output format) for executing ETL classes (partition, partitioner, extractor, and loader).
>
> Basically
> - partition is used by split
> - partitioner is invoked by input format
> - extractor is invoked by mapper
> - loader is invoked by output format
>
>
> This addresses bug SQOOP-588.
> https://issues.apache.org/jira/browse/SQOOP-588
>
>
> Diffs
> -----
>
> branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java 1373167
> branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartition.java PRE-CREATION
> branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java 1373167
> branches/sqoop2/core/pom.xml 1373167
> branches/sqoop2/core/src/main/java/org/apache/sqoop/core/CoreError.java 1373167
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/JobConstants.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlContext.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlMutableContext.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/Data.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/FieldTypes.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopFileOutputFormat.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopNullOutputFormat.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java PRE-CREATION
> branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopSplit.java PRE-CREATION
> branches/sqoop2/core/src/test/java/org/apache/sqoop/job/TestMapReduce.java PRE-CREATION
> branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Context.java 1373167
> branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Partition.java 1373167
> branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataReader.java 1373167
> branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataWriter.java 1373167
>
> Diff: https://reviews.apache.org/r/6811/diff/
>
>
> Testing
> -------
>
> Create a new TestMapReduce test case
>
>
> Thanks,
>
> Bilung Lee
>
>
Re: Review Request: SQOOP-588 Provide MapReduce classes for executing ETL
classes
Posted by Bilung Lee <bl...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6811/
-----------------------------------------------------------
(Updated Aug. 28, 2012, 10:08 p.m.)
Review request for Sqoop.
Description
-------
This patch provides necessary MapReduce classes (split, input format, mapper, and output format) for executing ETL classes (partition, partitioner, extractor, and loader).
Basically
- partition is used by split
- partitioner is invoked by input format
- extractor is invoked by mapper
- loader is invoked by output format
This addresses bug SQOOP-588.
https://issues.apache.org/jira/browse/SQOOP-588
Diffs (updated)
-----
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java 1373167
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartition.java PRE-CREATION
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java 1373167
branches/sqoop2/core/pom.xml 1373167
branches/sqoop2/core/src/main/java/org/apache/sqoop/core/CoreError.java 1373167
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/JobConstants.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlContext.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlMutableContext.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/Data.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/FieldTypes.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopFileOutputFormat.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopNullOutputFormat.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopSplit.java PRE-CREATION
branches/sqoop2/core/src/test/java/org/apache/sqoop/job/TestMapReduce.java PRE-CREATION
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Context.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Partition.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataReader.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataWriter.java 1373167
Diff: https://reviews.apache.org/r/6811/diff/
Testing
-------
Create a new TestMapReduce test case
Thanks,
Bilung Lee
Re: Review Request: SQOOP-588 Provide MapReduce classes for executing ETL
classes
Posted by Bilung Lee <bl...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6811/
-----------------------------------------------------------
(Updated Aug. 28, 2012, 9:56 p.m.)
Review request for Sqoop.
Description
-------
This patch provides necessary MapReduce classes (split, input format, mapper, and output format) for executing ETL classes (partition, partitioner, extractor, and loader).
Basically
- partition is used by split
- partitioner is invoked by input format
- extractor is invoked by mapper
- loader is invoked by output format
This addresses bug SQOOP-588.
https://issues.apache.org/jira/browse/SQOOP-588
Diffs (updated)
-----
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java 1373167
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartition.java PRE-CREATION
branches/sqoop2/connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java 1373167
branches/sqoop2/core/pom.xml 1373167
branches/sqoop2/core/src/main/java/org/apache/sqoop/core/CoreError.java 1373167
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/JobConstants.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlContext.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/etl/EtlMutableContext.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/Data.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/io/FieldTypes.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopFileOutputFormat.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopNullOutputFormat.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java PRE-CREATION
branches/sqoop2/core/src/main/java/org/apache/sqoop/job/mr/SqoopSplit.java PRE-CREATION
branches/sqoop2/core/src/test/java/org/apache/sqoop/job/TestMapReduce.java PRE-CREATION
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Context.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/etl/Partition.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataReader.java 1373167
branches/sqoop2/spi/src/main/java/org/apache/sqoop/job/io/DataWriter.java 1373167
Diff: https://reviews.apache.org/r/6811/diff/
Testing
-------
Create a new TestMapReduce test case
Thanks,
Bilung Lee