You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Raghav Gautam <ra...@gmail.com> on 2013/08/27 04:12:47 UTC
Review Request 13839: Sqoop-1182: Compression option for import
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13839/
-----------------------------------------------------------
Review request for Sqoop.
Bugs: sqoop-1182
https://issues.apache.org/jira/browse/sqoop-1182
Repository: sqoop-sqoop2
Description
-------
Exposing compression options for Sqoop2
Diffs
-----
core/src/main/java/org/apache/sqoop/framework/configuration/OutputCompression.java PRE-CREATION
core/src/main/java/org/apache/sqoop/framework/configuration/OutputForm.java 3cb9499
core/src/main/resources/framework-resources.properties cebc90e
execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java 767080c
execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/13839/diff/
Testing
-------
Added unit tests.
Manually tested.
Thanks,
Raghav Gautam
Re: Review Request 13839: Sqoop-1182: Compression option for import
Posted by Venkat Ranganathan <n....@live.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13839/#review25593
-----------------------------------------------------------
Looks good - thanks for updating with the reviews.
- Venkat Ranganathan
On Aug. 27, 2013, 2:12 a.m., Raghav Gautam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13839/
> -----------------------------------------------------------
>
> (Updated Aug. 27, 2013, 2:12 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: sqoop-1182
> https://issues.apache.org/jira/browse/sqoop-1182
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> Exposing compression options for Sqoop2
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/sqoop/framework/configuration/OutputCompression.java PRE-CREATION
> core/src/main/java/org/apache/sqoop/framework/configuration/OutputForm.java 3cb9499
> core/src/main/resources/framework-resources.properties cebc90e
> execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java 767080c
> execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13839/diff/
>
>
> Testing
> -------
>
> Added unit tests.
> Manually tested.
>
>
> Thanks,
>
> Raghav Gautam
>
>
Re: Review Request 13839: Sqoop-1182: Compression option for import
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13839/#review25821
-----------------------------------------------------------
core/src/main/java/org/apache/sqoop/framework/configuration/OutputCompression.java
<https://reviews.apache.org/r/13839/#comment50367>
The enum seems to be as a great fit for the choice of compression format, but it might be tricky in following two situations:
* Not all the predefined compression codecs might be available on the Hadoop cluster.
* Some compression codecs available on the Hadoop cluster might not be available in the predefined list.
I think that technically the ideal solution would be to include String input that would accept classname for the codec - this way user would have complete power of what he wants to use. On the other hand such approach is not much user friendly.
What about merging this and the original idea? We can also put inside the enum value "OTHER" and let user specify custom class? We can always verify if the custom class is available in the Validator.
- Jarek Cecho
On Aug. 27, 2013, 2:12 a.m., Raghav Gautam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13839/
> -----------------------------------------------------------
>
> (Updated Aug. 27, 2013, 2:12 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: sqoop-1182
> https://issues.apache.org/jira/browse/sqoop-1182
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> Exposing compression options for Sqoop2
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/sqoop/framework/configuration/OutputCompression.java PRE-CREATION
> core/src/main/java/org/apache/sqoop/framework/configuration/OutputForm.java 3cb9499
> core/src/main/resources/framework-resources.properties cebc90e
> execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java 767080c
> execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13839/diff/
>
>
> Testing
> -------
>
> Added unit tests.
> Manually tested.
>
>
> Thanks,
>
> Raghav Gautam
>
>
Re: Review Request 13839: Sqoop-1182: Compression option for import
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13839/#review25985
-----------------------------------------------------------
Ship it!
Ship It!
- Jarek Cecho
On Sept. 5, 2013, 11:56 p.m., Raghav Gautam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13839/
> -----------------------------------------------------------
>
> (Updated Sept. 5, 2013, 11:56 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: sqoop-1182
> https://issues.apache.org/jira/browse/sqoop-1182
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> Exposing compression options for Sqoop2
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/sqoop/framework/configuration/OutputCompression.java PRE-CREATION
> core/src/main/java/org/apache/sqoop/framework/configuration/OutputForm.java 3cb9499
> core/src/main/resources/framework-resources.properties cebc90e
> execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java 767080c
> execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13839/diff/
>
>
> Testing
> -------
>
> Added unit tests.
> Manually tested.
>
>
> Thanks,
>
> Raghav Gautam
>
>
Re: Review Request 13839: Sqoop-1182: Compression option for import
Posted by Raghav Gautam <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13839/
-----------------------------------------------------------
(Updated Sept. 5, 2013, 4:56 p.m.)
Review request for Sqoop.
Changes
-------
Removing dependency on the codec classes from the unit tests.
Bugs: sqoop-1182
https://issues.apache.org/jira/browse/sqoop-1182
Repository: sqoop-sqoop2
Description
-------
Exposing compression options for Sqoop2
Diffs (updated)
-----
core/src/main/java/org/apache/sqoop/framework/configuration/OutputCompression.java PRE-CREATION
core/src/main/java/org/apache/sqoop/framework/configuration/OutputForm.java 3cb9499
core/src/main/resources/framework-resources.properties cebc90e
execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java 767080c
execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/13839/diff/
Testing
-------
Added unit tests.
Manually tested.
Thanks,
Raghav Gautam
Re: Review Request 13839: Sqoop-1182: Compression option for import
Posted by Venkat Ranganathan <n....@live.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13839/#review25901
-----------------------------------------------------------
Looks good. Regarding the support for custom compression format, I think we can track it as part of a separate JIRA. Do you agree Jarcec?
Thanks
Venkat
- Venkat Ranganathan
On Sept. 3, 2013, 9:56 p.m., Raghav Gautam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13839/
> -----------------------------------------------------------
>
> (Updated Sept. 3, 2013, 9:56 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: sqoop-1182
> https://issues.apache.org/jira/browse/sqoop-1182
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> Exposing compression options for Sqoop2
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/sqoop/framework/configuration/OutputCompression.java PRE-CREATION
> core/src/main/java/org/apache/sqoop/framework/configuration/OutputForm.java 3cb9499
> core/src/main/resources/framework-resources.properties cebc90e
> execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java 767080c
> execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13839/diff/
>
>
> Testing
> -------
>
> Added unit tests.
> Manually tested.
>
>
> Thanks,
>
> Raghav Gautam
>
>
Re: Review Request 13839: Sqoop-1182: Compression option for import
Posted by Raghav Gautam <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13839/
-----------------------------------------------------------
(Updated Sept. 3, 2013, 2:56 p.m.)
Review request for Sqoop.
Changes
-------
Adding LzoCodec and DeflateCodec.
All the codecs are manually tested.
Bugs: sqoop-1182
https://issues.apache.org/jira/browse/sqoop-1182
Repository: sqoop-sqoop2
Description
-------
Exposing compression options for Sqoop2
Diffs (updated)
-----
core/src/main/java/org/apache/sqoop/framework/configuration/OutputCompression.java PRE-CREATION
core/src/main/java/org/apache/sqoop/framework/configuration/OutputForm.java 3cb9499
core/src/main/resources/framework-resources.properties cebc90e
execution/mapreduce/src/main/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngine.java 767080c
execution/mapreduce/src/test/java/org/apache/sqoop/execution/mapreduce/MapreduceExecutionEngineTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/13839/diff/
Testing
-------
Added unit tests.
Manually tested.
Thanks,
Raghav Gautam