You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Bilung Lee <bl...@gmail.com> on 2011/12/01 19:59:34 UTC
Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/
-----------------------------------------------------------
Review request for Sqoop, Arvind Prabhakar and Tom White.
Summary
-------
Make Sqoop work with Hadoop 0.23
This addresses bug SQOOP-397.
https://issues.apache.org/jira/browse/SQOOP-397
Diffs
-----
./COMPILING.txt 1197706
./build.xml 1208854
./ivy.xml 1208716
./ivy/libraries.properties 1208716
./src/ivy/ivy-hdp20.xml PRE-CREATION
./src/ivy/ivy-hdp23.xml PRE-CREATION
./src/ivy/libraries-hdp20.properties PRE-CREATION
./src/ivy/libraries-hdp23.properties PRE-CREATION
./src/test/com/cloudera/sqoop/TestCompression.java 1208716
./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1208054
./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1208054
./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1208054
./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1208054
Diff: https://reviews.apache.org/r/2983/diff
Testing
-------
Thanks,
Bilung
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Arvind Prabhakar <ar...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3599
-----------------------------------------------------------
Thanks for the patch Bilung. I have a few general comments:
1. Since the new mechanism for build switch between 20 and 23 overwrites the ivy.xml file which is inside source control, it can possibly cause confusion when a build is done. One way to fix this would be to remove top level ivy.xml from source control, put top level ivy.xml in svn:ignore property, and the always copy the appropriate ivy-hdpXX.xml file during build.
2. It would be great if we can use the hadoop version property to selectively exclude tests from running when the 23 version is used. Doing this will make sure that the tests continue to run when the build is done against 20.
- Arvind
On 2011-12-01 18:59:33, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-01 18:59:33)
>
>
> Review request for Sqoop, Arvind Prabhakar and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1208854
> ./ivy.xml 1208716
> ./ivy/libraries.properties 1208716
> ./src/ivy/ivy-hdp20.xml PRE-CREATION
> ./src/ivy/ivy-hdp23.xml PRE-CREATION
> ./src/ivy/libraries-hdp20.properties PRE-CREATION
> ./src/ivy/libraries-hdp23.properties PRE-CREATION
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1208054
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1208054
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1208054
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1208054
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Ahmed Radwan <a....@gmail.com>.
> On 2011-12-06 21:12:02, Ahmed Radwan wrote:
> > Thanks! Built a release tar ball with the new patch and did basic testing against a real 4-node hadoop 0.23 cluster. One issue I saw is the record count in the import use-case is showing 0, but the records are physically imported, and the job is successfull. Seems to be a Sqoop problem with mapreduce counters in 0.23. I think this can be addressed in a separate jira.
>
> Tom White wrote:
> I wonder if the counter issue is https://issues.apache.org/jira/browse/MAPREDUCE-3433 (fixed in 0.23.1).
>
> Ahmed Radwan wrote:
> Good point, I'll retry using the MAPREDUCE-3433 patch to confirm. Thanks!
Using a hadoop build having the MAPREDUCE-3433 patch resolves the counters problem described above. Thanks Tom for pointing that out. This patch is checked to trunk and 0.23.1 so this counters problem should only exist in 0.23.0, and not any of the following releases.
- Ahmed
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3673
-----------------------------------------------------------
On 2011-12-06 19:14:02, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-06 19:14:02)
>
>
> Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1210580
> ./ivy.xml 1210686
> ./ivy/libraries.properties 1210579
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1209661
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1209644
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Ahmed Radwan <a....@gmail.com>.
> On 2011-12-06 21:12:02, Ahmed Radwan wrote:
> > Thanks! Built a release tar ball with the new patch and did basic testing against a real 4-node hadoop 0.23 cluster. One issue I saw is the record count in the import use-case is showing 0, but the records are physically imported, and the job is successfull. Seems to be a Sqoop problem with mapreduce counters in 0.23. I think this can be addressed in a separate jira.
>
> Tom White wrote:
> I wonder if the counter issue is https://issues.apache.org/jira/browse/MAPREDUCE-3433 (fixed in 0.23.1).
Good point, I'll retry using the MAPREDUCE-3433 patch to confirm. Thanks!
- Ahmed
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3673
-----------------------------------------------------------
On 2011-12-06 19:14:02, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-06 19:14:02)
>
>
> Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1210580
> ./ivy.xml 1210686
> ./ivy/libraries.properties 1210579
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1209661
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1209644
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Tom White <to...@apache.org>.
> On 2011-12-06 21:12:02, Ahmed Radwan wrote:
> > Thanks! Built a release tar ball with the new patch and did basic testing against a real 4-node hadoop 0.23 cluster. One issue I saw is the record count in the import use-case is showing 0, but the records are physically imported, and the job is successfull. Seems to be a Sqoop problem with mapreduce counters in 0.23. I think this can be addressed in a separate jira.
I wonder if the counter issue is https://issues.apache.org/jira/browse/MAPREDUCE-3433 (fixed in 0.23.1).
- Tom
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3673
-----------------------------------------------------------
On 2011-12-06 19:14:02, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-06 19:14:02)
>
>
> Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1210580
> ./ivy.xml 1210686
> ./ivy/libraries.properties 1210579
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1209661
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1209644
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Ahmed Radwan <a....@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3673
-----------------------------------------------------------
Ship it!
Thanks! Built a release tar ball with the new patch and did basic testing against a real 4-node hadoop 0.23 cluster. One issue I saw is the record count in the import use-case is showing 0, but the records are physically imported, and the job is successfull. Seems to be a Sqoop problem with mapreduce counters in 0.23. I think this can be addressed in a separate jira.
- Ahmed
On 2011-12-06 19:14:02, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-06 19:14:02)
>
>
> Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1210580
> ./ivy.xml 1210686
> ./ivy/libraries.properties 1210579
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1209661
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1209644
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Tom White <to...@apache.org>.
> On 2011-12-06 21:04:57, Tom White wrote:
> > This looks good to me. Are you planning on doing the HBase and ZooKeeper version parameterization in another JIRA?
>
> Bilung Lee wrote:
> Thanks! Yes, it would be better to wait until it is clear about what versions to use.
Agreed. The HBase 0.92 RC (available now?) and ZooKeeper 3.4.0 (already out) are the ones to try with.
- Tom
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3672
-----------------------------------------------------------
On 2011-12-06 19:14:02, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-06 19:14:02)
>
>
> Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1210580
> ./ivy.xml 1210686
> ./ivy/libraries.properties 1210579
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1209661
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1209644
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Bilung Lee <bl...@gmail.com>.
> On 2011-12-06 21:04:57, Tom White wrote:
> > This looks good to me. Are you planning on doing the HBase and ZooKeeper version parameterization in another JIRA?
Thanks! Yes, it would be better to wait until it is clear about what versions to use.
- Bilung
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3672
-----------------------------------------------------------
On 2011-12-06 19:14:02, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-06 19:14:02)
>
>
> Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1210580
> ./ivy.xml 1210686
> ./ivy/libraries.properties 1210579
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1209661
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1209644
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Tom White <to...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3672
-----------------------------------------------------------
Ship it!
This looks good to me. Are you planning on doing the HBase and ZooKeeper version parameterization in another JIRA?
- Tom
On 2011-12-06 19:14:02, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-06 19:14:02)
>
>
> Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1210580
> ./ivy.xml 1210686
> ./ivy/libraries.properties 1210579
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1209661
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1209644
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Arvind Prabhakar <ar...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3676
-----------------------------------------------------------
Ship it!
+1
- Arvind
On 2011-12-06 19:14:02, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-06 19:14:02)
>
>
> Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1210580
> ./ivy.xml 1210686
> ./ivy/libraries.properties 1210579
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1209661
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1209661
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1209644
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Bilung Lee <bl...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/
-----------------------------------------------------------
(Updated 2011-12-06 19:14:02.584283)
Review request for Sqoop, Ahmed Radwan, Arvind Prabhakar, and Tom White.
Summary
-------
Make Sqoop work with Hadoop 0.23
This addresses bug SQOOP-397.
https://issues.apache.org/jira/browse/SQOOP-397
Diffs
-----
./COMPILING.txt 1197706
./build.xml 1210580
./ivy.xml 1210686
./ivy/libraries.properties 1210579
./src/test/com/cloudera/sqoop/TestCompression.java 1208716
./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1209661
./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1209661
./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1209661
./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1209644
Diff: https://reviews.apache.org/r/2983/diff
Testing
-------
Thanks,
Bilung
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Bilung Lee <bl...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/
-----------------------------------------------------------
(Updated 2011-12-06 19:12:36.179918)
Review request for Sqoop, Arvind Prabhakar and Tom White.
Changes
-------
Thanks for the feedback!
A new patch is uploaded:
1. To provide a property (hadoopversion) for the choice of Hadoop version.
2. To exclude those test cases conditionally based on Hadoop version.
Summary
-------
Make Sqoop work with Hadoop 0.23
This addresses bug SQOOP-397.
https://issues.apache.org/jira/browse/SQOOP-397
Diffs (updated)
-----
./COMPILING.txt 1197706
./build.xml 1210580
./ivy.xml 1210686
./ivy/libraries.properties 1210579
./src/test/com/cloudera/sqoop/TestCompression.java 1208716
./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1209661
./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1209661
./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1209661
./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1209644
Diff: https://reviews.apache.org/r/2983/diff
Testing
-------
Thanks,
Bilung
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Tom White <to...@apache.org>.
> On 2011-12-02 04:40:12, Tom White wrote:
> > ./COMPILING.txt, line 280
> > <https://reviews.apache.org/r/2983/diff/2/?file=61445#file61445line280>
> >
> > I think 0.20.x should continue to be the default until 0.23.x has stabilized.
I hadn't seen https://issues.apache.org/jira/browse/SQOOP-384 which makes this more complex than I had thought, since Sqoop depends on features not in Hadoop 0.20.20x. So making a dependency on 0.23.x in trunk is probably acceptable.
- Tom
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3600
-----------------------------------------------------------
On 2011-12-01 18:59:33, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-01 18:59:33)
>
>
> Review request for Sqoop, Arvind Prabhakar and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1208854
> ./ivy.xml 1208716
> ./ivy/libraries.properties 1208716
> ./src/ivy/ivy-hdp20.xml PRE-CREATION
> ./src/ivy/ivy-hdp23.xml PRE-CREATION
> ./src/ivy/libraries-hdp20.properties PRE-CREATION
> ./src/ivy/libraries-hdp23.properties PRE-CREATION
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1208054
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1208054
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1208054
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1208054
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>
Re: Review Request: SQOOP-397 Make Sqoop work with Hadoop 0.23
Posted by Tom White <to...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2983/#review3600
-----------------------------------------------------------
./COMPILING.txt
<https://reviews.apache.org/r/2983/#comment8025>
I think 0.20.x should continue to be the default until 0.23.x has stabilized.
./build.xml
<https://reviews.apache.org/r/2983/#comment8026>
Rather than using conditional logic in the build, it would be simpler to use the approach that Pig takes and drive the choice by a property.
Search for "hadoopversion" in http://svn.apache.org/repos/asf/pig/trunk/build.xml and http://svn.apache.org/repos/asf/pig/trunk/ivy.xml (note the way the configuration name is parameterized in the latter).
./ivy/libraries.properties
<https://reviews.apache.org/r/2983/#comment8027>
We should parameterize HBase at the same time so we can switch to 0.92.0 when it comes out, since it supports Hadoop 0.23.
./ivy/libraries.properties
<https://reviews.apache.org/r/2983/#comment8028>
ZooKeeper too?
./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java
<https://reviews.apache.org/r/2983/#comment8029>
Rather than commenting out, why not just add an @Ignore annotation? Also open a JIRA and link to it so it doesn't get forgotten.
- Tom
On 2011-12-01 18:59:33, Bilung Lee wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2983/
> -----------------------------------------------------------
>
> (Updated 2011-12-01 18:59:33)
>
>
> Review request for Sqoop, Arvind Prabhakar and Tom White.
>
>
> Summary
> -------
>
> Make Sqoop work with Hadoop 0.23
>
>
> This addresses bug SQOOP-397.
> https://issues.apache.org/jira/browse/SQOOP-397
>
>
> Diffs
> -----
>
> ./COMPILING.txt 1197706
> ./build.xml 1208854
> ./ivy.xml 1208716
> ./ivy/libraries.properties 1208716
> ./src/ivy/ivy-hdp20.xml PRE-CREATION
> ./src/ivy/ivy-hdp23.xml PRE-CREATION
> ./src/ivy/libraries-hdp20.properties PRE-CREATION
> ./src/ivy/libraries-hdp23.properties PRE-CREATION
> ./src/test/com/cloudera/sqoop/TestCompression.java 1208716
> ./src/test/com/cloudera/sqoop/hbase/HBaseTestCase.java 1208054
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseImport.java 1208054
> ./src/test/com/cloudera/sqoop/hbase/TestHBaseQueryImport.java 1208054
> ./src/test/com/cloudera/sqoop/mapreduce/db/TestDataDrivenDBInputFormat.java 1208054
>
> Diff: https://reviews.apache.org/r/2983/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bilung
>
>