You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Venkat Ranganathan <n....@live.com> on 2013/05/21 06:09:05 UTC
Re: Review Request: Fix for SQOOP-1053
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11288/
-----------------------------------------------------------
(Updated May 21, 2013, 4:09 a.m.)
Review request for Sqoop and Jarek Cecho.
Changes
-------
Fixed the summary title to properly reflect the bug being targeted
Summary (updated)
-----------------
Fix for SQOOP-1053
Description
-------
I updated the build and ivy settings to build Sqoop with Hadoop 2.0.4 alpha.
One of the compression tests (testBzip2TextCompression) fails with the following stack trace
Testcase: testBzip2TextCompression took 1.21 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.isNativeBzip2Loaded(Bzip2Factory.java:54)
at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.getBzip2Decompressor(Bzip2Factory.java:123)
at org.apache.hadoop.io.compress.BZip2Codec.createDecompressor(BZip2Codec.java:250)
at org.apache.hadoop.io.compress.BZip2Codec.createInputStream(BZip2Codec.java:156)
at com.cloudera.sqoop.TestCompression.runTextCompressionTest(TestCompression.java:170)
at com.cloudera.sqoop.TestCompression.testBzip2TextCompression(TestCompression.java:189)
The reason is in 2.0.4, with native BZip2 codec, the configuration is checked and we don't do setConf for BZip2 codec. Fixed the test to set the configuration irrespective of whether it was passed in or we created the codec.
Diffs
-----
build.xml 1c33fee
ivy.xml 1fa4dd1
src/test/com/cloudera/sqoop/TestCompression.java 048b87d
Diff: https://reviews.apache.org/r/11288/diff/
Testing
-------
Ran tests with the new settings and all tests pass
Thanks,
Venkat Ranganathan
Re: Review Request: Fix for SQOOP-1053
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11288/#review20884
-----------------------------------------------------------
Ship it!
The changes looks good to me, I'll run tests for all supported hadoop version prior committing. Please do attach your patch to the JIRA so that I can commit it.
- Jarek Cecho
On May 21, 2013, 6:57 p.m., Venkat Ranganathan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11288/
> -----------------------------------------------------------
>
> (Updated May 21, 2013, 6:57 p.m.)
>
>
> Review request for Sqoop and Jarek Cecho.
>
>
> Description
> -------
>
> I updated the build and ivy settings to build Sqoop with Hadoop 2.0.4 alpha.
> One of the compression tests (testBzip2TextCompression) fails with the following stack trace
> Testcase: testBzip2TextCompression took 1.21 sec
> Caused an ERROR
> null
> java.lang.NullPointerException
> at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.isNativeBzip2Loaded(Bzip2Factory.java:54)
> at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.getBzip2Decompressor(Bzip2Factory.java:123)
> at org.apache.hadoop.io.compress.BZip2Codec.createDecompressor(BZip2Codec.java:250)
> at org.apache.hadoop.io.compress.BZip2Codec.createInputStream(BZip2Codec.java:156)
> at com.cloudera.sqoop.TestCompression.runTextCompressionTest(TestCompression.java:170)
> at com.cloudera.sqoop.TestCompression.testBzip2TextCompression(TestCompression.java:189)
> The reason is in 2.0.4, with native BZip2 codec, the configuration is checked and we don't do setConf for BZip2 codec. Fixed the test to set the configuration irrespective of whether it was passed in or we created the codec.
>
>
> Diffs
> -----
>
> build.xml 1c33fee
> src/test/com/cloudera/sqoop/TestCompression.java 048b87d
>
> Diff: https://reviews.apache.org/r/11288/diff/
>
>
> Testing
> -------
>
> Ran tests with the new settings and all tests pass
>
>
> Thanks,
>
> Venkat Ranganathan
>
>
Re: Review Request: Fix for SQOOP-1053
Posted by Venkat Ranganathan <n....@live.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11288/
-----------------------------------------------------------
(Updated May 21, 2013, 6:57 p.m.)
Review request for Sqoop and Jarek Cecho.
Changes
-------
Updated to reflect comments
Description
-------
I updated the build and ivy settings to build Sqoop with Hadoop 2.0.4 alpha.
One of the compression tests (testBzip2TextCompression) fails with the following stack trace
Testcase: testBzip2TextCompression took 1.21 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.isNativeBzip2Loaded(Bzip2Factory.java:54)
at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.getBzip2Decompressor(Bzip2Factory.java:123)
at org.apache.hadoop.io.compress.BZip2Codec.createDecompressor(BZip2Codec.java:250)
at org.apache.hadoop.io.compress.BZip2Codec.createInputStream(BZip2Codec.java:156)
at com.cloudera.sqoop.TestCompression.runTextCompressionTest(TestCompression.java:170)
at com.cloudera.sqoop.TestCompression.testBzip2TextCompression(TestCompression.java:189)
The reason is in 2.0.4, with native BZip2 codec, the configuration is checked and we don't do setConf for BZip2 codec. Fixed the test to set the configuration irrespective of whether it was passed in or we created the codec.
Diffs (updated)
-----
build.xml 1c33fee
src/test/com/cloudera/sqoop/TestCompression.java 048b87d
Diff: https://reviews.apache.org/r/11288/diff/
Testing
-------
Ran tests with the new settings and all tests pass
Thanks,
Venkat Ranganathan
Re: Review Request: Fix for SQOOP-1053
Posted by Venkat Ranganathan <n....@live.com>.
> On May 21, 2013, 8:39 a.m., Jarek Cecho wrote:
> > Hi Venkat,
> > thank you very much for upgrading the dependency on Hadoop to 2.0.4. Considering that entire Hadoop 2.0.0 is still alpha state, do you think that it make sense to make the profile hadoop "200" depending on Hadoop 2.0.4 rather then introducing completely new profile?
> >
> > Jarcec
Makes sense I will update the patch as you suggested. Thanks Jarek
- Venkat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11288/#review20829
-----------------------------------------------------------
On May 21, 2013, 4:09 a.m., Venkat Ranganathan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11288/
> -----------------------------------------------------------
>
> (Updated May 21, 2013, 4:09 a.m.)
>
>
> Review request for Sqoop and Jarek Cecho.
>
>
> Description
> -------
>
> I updated the build and ivy settings to build Sqoop with Hadoop 2.0.4 alpha.
> One of the compression tests (testBzip2TextCompression) fails with the following stack trace
> Testcase: testBzip2TextCompression took 1.21 sec
> Caused an ERROR
> null
> java.lang.NullPointerException
> at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.isNativeBzip2Loaded(Bzip2Factory.java:54)
> at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.getBzip2Decompressor(Bzip2Factory.java:123)
> at org.apache.hadoop.io.compress.BZip2Codec.createDecompressor(BZip2Codec.java:250)
> at org.apache.hadoop.io.compress.BZip2Codec.createInputStream(BZip2Codec.java:156)
> at com.cloudera.sqoop.TestCompression.runTextCompressionTest(TestCompression.java:170)
> at com.cloudera.sqoop.TestCompression.testBzip2TextCompression(TestCompression.java:189)
> The reason is in 2.0.4, with native BZip2 codec, the configuration is checked and we don't do setConf for BZip2 codec. Fixed the test to set the configuration irrespective of whether it was passed in or we created the codec.
>
>
> Diffs
> -----
>
> build.xml 1c33fee
> ivy.xml 1fa4dd1
> src/test/com/cloudera/sqoop/TestCompression.java 048b87d
>
> Diff: https://reviews.apache.org/r/11288/diff/
>
>
> Testing
> -------
>
> Ran tests with the new settings and all tests pass
>
>
> Thanks,
>
> Venkat Ranganathan
>
>
Re: Review Request: Fix for SQOOP-1053
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11288/#review20829
-----------------------------------------------------------
Hi Venkat,
thank you very much for upgrading the dependency on Hadoop to 2.0.4. Considering that entire Hadoop 2.0.0 is still alpha state, do you think that it make sense to make the profile hadoop "200" depending on Hadoop 2.0.4 rather then introducing completely new profile?
Jarcec
- Jarek Cecho
On May 21, 2013, 4:09 a.m., Venkat Ranganathan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11288/
> -----------------------------------------------------------
>
> (Updated May 21, 2013, 4:09 a.m.)
>
>
> Review request for Sqoop and Jarek Cecho.
>
>
> Description
> -------
>
> I updated the build and ivy settings to build Sqoop with Hadoop 2.0.4 alpha.
> One of the compression tests (testBzip2TextCompression) fails with the following stack trace
> Testcase: testBzip2TextCompression took 1.21 sec
> Caused an ERROR
> null
> java.lang.NullPointerException
> at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.isNativeBzip2Loaded(Bzip2Factory.java:54)
> at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.getBzip2Decompressor(Bzip2Factory.java:123)
> at org.apache.hadoop.io.compress.BZip2Codec.createDecompressor(BZip2Codec.java:250)
> at org.apache.hadoop.io.compress.BZip2Codec.createInputStream(BZip2Codec.java:156)
> at com.cloudera.sqoop.TestCompression.runTextCompressionTest(TestCompression.java:170)
> at com.cloudera.sqoop.TestCompression.testBzip2TextCompression(TestCompression.java:189)
> The reason is in 2.0.4, with native BZip2 codec, the configuration is checked and we don't do setConf for BZip2 codec. Fixed the test to set the configuration irrespective of whether it was passed in or we created the codec.
>
>
> Diffs
> -----
>
> build.xml 1c33fee
> ivy.xml 1fa4dd1
> src/test/com/cloudera/sqoop/TestCompression.java 048b87d
>
> Diff: https://reviews.apache.org/r/11288/diff/
>
>
> Testing
> -------
>
> Ran tests with the new settings and all tests pass
>
>
> Thanks,
>
> Venkat Ranganathan
>
>