You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Cheolsoo Park <ch...@cloudera.com> on 2012/10/22 08:50:03 UTC
Re: Review Request: TestJobSumission and TestHBaseStorage don't work with
HBase 0.94 and ZK 3.4.3
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7676/
-----------------------------------------------------------
(Updated Oct. 22, 2012, 6:50 a.m.)
Review request for pig and Santhosh Srinivasan.
Description
-------
The changes include:
1. Stop bundling hbase.jar and zookeoper.jar with pig.jar. So there should be no longer incompatibility issues when using pig.jar with different versions of hbase.jar. But to use HBaseStorage, HBASE_HOME and ZOOKEEPER_HOME must be set by the user. Note that I am adding protobuf-java.jar to pig.jar because otherwise it has to be explicitly added to PIG_CLASSPATH to use HBaseStorage, which is not very intuitive.
2. Bump hbase and zk to 0.94.1 and 3.4.3 respectively. Since we no longer bundle them in pig.jar, which versions we use doesn't matter. These jar files will be used for unit test only.
3. Make the unit test cases work with newer versions of hbase and zk.
4. Add hbase runtime dependencies to ivy.xml.
This addresses bug PIG-2885.
https://issues.apache.org/jira/browse/PIG-2885
Diffs
-----
build.xml 6b04f8a
ivy.xml 6e0a2e5
ivy/libraries.properties 55da6c6
test/org/apache/pig/test/TestHBaseStorage.java cc1efef
test/org/apache/pig/test/TestJobSubmission.java 021662f
Diff: https://reviews.apache.org/r/7676/diff/
Testing
-------
ant clean test-commit -Dhadoopversion=20
ant clean test -Dtestcase=TestHBaseStorage -Dhadoopversion=20
ant clean test -Dtestcase=TestJobSumission -Dhadoopversion=20
I also manually tested pig.jar with hbase 0.90 and 0.94. Once HBASE_HOME and ZOOKEEPER_HOME are set, HBaseStorage works fine with both versions.
Thanks,
Cheolsoo Park
Re: Review Request: TestJobSumission and TestHBaseStorage don't work with
HBase 0.94 and ZK 3.4.3
Posted by Cheolsoo Park <ch...@cloudera.com>.
> On Oct. 23, 2012, 6:55 p.m., Bill Graham wrote:
> > Do both HBASE_HOME and ZOOKEEPER_HOME need to be set, or would it suffice to just have their respective jars + HBASE_CONF_DIR in the classpath? Ideally, only the classpath would need to be set.
> >
> > Besides that, the code looks good.
Hi Bill,
Including hbase-*.jar and zookeeper-*.jar in PIG_CLASSPATH also works. All HBASE_HOME and ZOOKEEPER_HOME do is finding hbase-*.jar and zookeeper-*.jar in respective dirs and adding them to CLASSPATH.
So the following commands are basically equivalent:
PIG_CLASSPATH=~/workspace/hbase-0.94.1/hbase-0.94.1.jar:~/workspace/hbase-0.94.1/lib/zookeeper-3.4.3.jar ./bin/pig -x local
or
HBASE_HOME=~/workspace/hbase-0.94.1 ZOOKEEPER_HOME=~/workspace/hbase-0.94.1/lib ./bin/pig -x local
Of course, hbase-*.jar and zookeeper-*.jar are already present in your classpath, nothing has to be set.
I can definitely document both in the release note.
Thanks!
- Cheolsoo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7676/#review12689
-----------------------------------------------------------
On Oct. 23, 2012, 4:48 p.m., Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7676/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2012, 4:48 p.m.)
>
>
> Review request for pig and Santhosh Srinivasan.
>
>
> Description
> -------
>
> The changes include:
>
> 1. Stop bundling hbase.jar and zookeoper.jar with pig.jar. So there should be no longer incompatibility issues when using pig.jar with different versions of hbase.jar. But to use HBaseStorage, HBASE_HOME and ZOOKEEPER_HOME must be set by the user. Note that I am adding protobuf-java.jar to pig.jar because otherwise it has to be explicitly added to PIG_CLASSPATH to use HBaseStorage, which is not very intuitive.
>
> 2. Bump hbase and zk to 0.94.1 and 3.4.3 respectively. Since we no longer bundle them in pig.jar, which versions we use doesn't matter. These jar files will be used for unit test only.
>
> 3. Make the unit test cases work with newer versions of hbase and zk.
>
> 4. Add hbase runtime dependencies to ivy.xml.
>
>
> This addresses bug PIG-2885.
> https://issues.apache.org/jira/browse/PIG-2885
>
>
> Diffs
> -----
>
> build.xml 6b04f8a
> ivy.xml 6e0a2e5
> ivy/libraries.properties 55da6c6
> test/org/apache/pig/test/TestHBaseStorage.java cc1efef
> test/org/apache/pig/test/TestJobSubmission.java 021662f
>
> Diff: https://reviews.apache.org/r/7676/diff/
>
>
> Testing
> -------
>
> ant clean test-commit -Dhadoopversion=20
> ant clean test -Dtestcase=TestHBaseStorage -Dhadoopversion=20
> ant clean test -Dtestcase=TestJobSumission -Dhadoopversion=20
>
> I also manually tested pig.jar with hbase 0.90 and 0.94. Once HBASE_HOME and ZOOKEEPER_HOME are set, HBaseStorage works fine with both versions.
>
>
> Thanks,
>
> Cheolsoo Park
>
>
Re: Review Request: TestJobSumission and TestHBaseStorage don't work with
HBase 0.94 and ZK 3.4.3
Posted by Bill Graham <bi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7676/#review12689
-----------------------------------------------------------
Do both HBASE_HOME and ZOOKEEPER_HOME need to be set, or would it suffice to just have their respective jars + HBASE_CONF_DIR in the classpath? Ideally, only the classpath would need to be set.
Besides that, the code looks good.
- Bill Graham
On Oct. 23, 2012, 4:48 p.m., Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7676/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2012, 4:48 p.m.)
>
>
> Review request for pig and Santhosh Srinivasan.
>
>
> Description
> -------
>
> The changes include:
>
> 1. Stop bundling hbase.jar and zookeoper.jar with pig.jar. So there should be no longer incompatibility issues when using pig.jar with different versions of hbase.jar. But to use HBaseStorage, HBASE_HOME and ZOOKEEPER_HOME must be set by the user. Note that I am adding protobuf-java.jar to pig.jar because otherwise it has to be explicitly added to PIG_CLASSPATH to use HBaseStorage, which is not very intuitive.
>
> 2. Bump hbase and zk to 0.94.1 and 3.4.3 respectively. Since we no longer bundle them in pig.jar, which versions we use doesn't matter. These jar files will be used for unit test only.
>
> 3. Make the unit test cases work with newer versions of hbase and zk.
>
> 4. Add hbase runtime dependencies to ivy.xml.
>
>
> This addresses bug PIG-2885.
> https://issues.apache.org/jira/browse/PIG-2885
>
>
> Diffs
> -----
>
> build.xml 6b04f8a
> ivy.xml 6e0a2e5
> ivy/libraries.properties 55da6c6
> test/org/apache/pig/test/TestHBaseStorage.java cc1efef
> test/org/apache/pig/test/TestJobSubmission.java 021662f
>
> Diff: https://reviews.apache.org/r/7676/diff/
>
>
> Testing
> -------
>
> ant clean test-commit -Dhadoopversion=20
> ant clean test -Dtestcase=TestHBaseStorage -Dhadoopversion=20
> ant clean test -Dtestcase=TestJobSumission -Dhadoopversion=20
>
> I also manually tested pig.jar with hbase 0.90 and 0.94. Once HBASE_HOME and ZOOKEEPER_HOME are set, HBaseStorage works fine with both versions.
>
>
> Thanks,
>
> Cheolsoo Park
>
>
Re: Review Request: TestJobSumission and TestHBaseStorage don't work with
HBase 0.94 and ZK 3.4.3
Posted by Bill Graham <bi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7676/#review12694
-----------------------------------------------------------
Ship it!
Ship It!
- Bill Graham
On Oct. 23, 2012, 4:48 p.m., Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7676/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2012, 4:48 p.m.)
>
>
> Review request for pig and Santhosh Srinivasan.
>
>
> Description
> -------
>
> The changes include:
>
> 1. Stop bundling hbase.jar and zookeoper.jar with pig.jar. So there should be no longer incompatibility issues when using pig.jar with different versions of hbase.jar. But to use HBaseStorage, HBASE_HOME and ZOOKEEPER_HOME must be set by the user. Note that I am adding protobuf-java.jar to pig.jar because otherwise it has to be explicitly added to PIG_CLASSPATH to use HBaseStorage, which is not very intuitive.
>
> 2. Bump hbase and zk to 0.94.1 and 3.4.3 respectively. Since we no longer bundle them in pig.jar, which versions we use doesn't matter. These jar files will be used for unit test only.
>
> 3. Make the unit test cases work with newer versions of hbase and zk.
>
> 4. Add hbase runtime dependencies to ivy.xml.
>
>
> This addresses bug PIG-2885.
> https://issues.apache.org/jira/browse/PIG-2885
>
>
> Diffs
> -----
>
> build.xml 6b04f8a
> ivy.xml 6e0a2e5
> ivy/libraries.properties 55da6c6
> test/org/apache/pig/test/TestHBaseStorage.java cc1efef
> test/org/apache/pig/test/TestJobSubmission.java 021662f
>
> Diff: https://reviews.apache.org/r/7676/diff/
>
>
> Testing
> -------
>
> ant clean test-commit -Dhadoopversion=20
> ant clean test -Dtestcase=TestHBaseStorage -Dhadoopversion=20
> ant clean test -Dtestcase=TestJobSumission -Dhadoopversion=20
>
> I also manually tested pig.jar with hbase 0.90 and 0.94. Once HBASE_HOME and ZOOKEEPER_HOME are set, HBaseStorage works fine with both versions.
>
>
> Thanks,
>
> Cheolsoo Park
>
>
Re: Review Request: TestJobSumission and TestHBaseStorage don't work with
HBase 0.94 and ZK 3.4.3
Posted by Cheolsoo Park <ch...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7676/
-----------------------------------------------------------
(Updated Oct. 23, 2012, 4:48 p.m.)
Review request for pig and Santhosh Srinivasan.
Changes
-------
Updating the version ZK to 3.4.4 as per Santhosh's suggestion.
Description
-------
The changes include:
1. Stop bundling hbase.jar and zookeoper.jar with pig.jar. So there should be no longer incompatibility issues when using pig.jar with different versions of hbase.jar. But to use HBaseStorage, HBASE_HOME and ZOOKEEPER_HOME must be set by the user. Note that I am adding protobuf-java.jar to pig.jar because otherwise it has to be explicitly added to PIG_CLASSPATH to use HBaseStorage, which is not very intuitive.
2. Bump hbase and zk to 0.94.1 and 3.4.3 respectively. Since we no longer bundle them in pig.jar, which versions we use doesn't matter. These jar files will be used for unit test only.
3. Make the unit test cases work with newer versions of hbase and zk.
4. Add hbase runtime dependencies to ivy.xml.
This addresses bug PIG-2885.
https://issues.apache.org/jira/browse/PIG-2885
Diffs (updated)
-----
build.xml 6b04f8a
ivy.xml 6e0a2e5
ivy/libraries.properties 55da6c6
test/org/apache/pig/test/TestHBaseStorage.java cc1efef
test/org/apache/pig/test/TestJobSubmission.java 021662f
Diff: https://reviews.apache.org/r/7676/diff/
Testing
-------
ant clean test-commit -Dhadoopversion=20
ant clean test -Dtestcase=TestHBaseStorage -Dhadoopversion=20
ant clean test -Dtestcase=TestJobSumission -Dhadoopversion=20
I also manually tested pig.jar with hbase 0.90 and 0.94. Once HBASE_HOME and ZOOKEEPER_HOME are set, HBaseStorage works fine with both versions.
Thanks,
Cheolsoo Park
Re: Review Request: TestJobSumission and TestHBaseStorage don't work with
HBase 0.94 and ZK 3.4.3
Posted by Santhosh Srinivasan <sm...@apache.org>.
> On Oct. 23, 2012, 5:07 a.m., Santhosh Srinivasan wrote:
> > A few comments. Its probably a good idea to have someone with more knowledge of HBaseStorage to take a second look.
>
> Cheolsoo Park wrote:
> Thank you very much for your feedback! I added answers below. Please let me know if you disagree with me.
Bill Graham would be a good choice to take a second look.
- Santhosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7676/#review12676
-----------------------------------------------------------
On Oct. 22, 2012, 6:50 a.m., Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7676/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2012, 6:50 a.m.)
>
>
> Review request for pig and Santhosh Srinivasan.
>
>
> Description
> -------
>
> The changes include:
>
> 1. Stop bundling hbase.jar and zookeoper.jar with pig.jar. So there should be no longer incompatibility issues when using pig.jar with different versions of hbase.jar. But to use HBaseStorage, HBASE_HOME and ZOOKEEPER_HOME must be set by the user. Note that I am adding protobuf-java.jar to pig.jar because otherwise it has to be explicitly added to PIG_CLASSPATH to use HBaseStorage, which is not very intuitive.
>
> 2. Bump hbase and zk to 0.94.1 and 3.4.3 respectively. Since we no longer bundle them in pig.jar, which versions we use doesn't matter. These jar files will be used for unit test only.
>
> 3. Make the unit test cases work with newer versions of hbase and zk.
>
> 4. Add hbase runtime dependencies to ivy.xml.
>
>
> This addresses bug PIG-2885.
> https://issues.apache.org/jira/browse/PIG-2885
>
>
> Diffs
> -----
>
> build.xml 6b04f8a
> ivy.xml 6e0a2e5
> ivy/libraries.properties 55da6c6
> test/org/apache/pig/test/TestHBaseStorage.java cc1efef
> test/org/apache/pig/test/TestJobSubmission.java 021662f
>
> Diff: https://reviews.apache.org/r/7676/diff/
>
>
> Testing
> -------
>
> ant clean test-commit -Dhadoopversion=20
> ant clean test -Dtestcase=TestHBaseStorage -Dhadoopversion=20
> ant clean test -Dtestcase=TestJobSumission -Dhadoopversion=20
>
> I also manually tested pig.jar with hbase 0.90 and 0.94. Once HBASE_HOME and ZOOKEEPER_HOME are set, HBaseStorage works fine with both versions.
>
>
> Thanks,
>
> Cheolsoo Park
>
>
Re: Review Request: TestJobSumission and TestHBaseStorage don't work with
HBase 0.94 and ZK 3.4.3
Posted by Cheolsoo Park <ch...@cloudera.com>.
> On Oct. 23, 2012, 5:07 a.m., Santhosh Srinivasan wrote:
> > A few comments. Its probably a good idea to have someone with more knowledge of HBaseStorage to take a second look.
Thank you very much for your feedback! I added answers below. Please let me know if you disagree with me.
> On Oct. 23, 2012, 5:07 a.m., Santhosh Srinivasan wrote:
> > ivy/libraries.properties, line 74
> > <https://reviews.apache.org/r/7676/diff/2/?file=178458#file178458line74>
> >
> > Zookeeper-3.4.4 has been out but has a known issue with SASL and Java 1.7. Is 3.3.3 required for Hbase 0.94.1 ?
You're asking whether ZK 3.4.3 (not 3.3.3) is required by hbase 0.94.1, right?
The answer is yes. In particular, HBaseTestingUtility depends on the following ZK class, which doesn't seem to exist in ZK 3.3.3:
java.lang.NoClassDefFoundError: org/apache/zookeeper/server/NIOServerCnxnFactory
In fact, I don't think that we should worry about those ZK known issues because the versions of HBase and ZK that I am updating only matter to unit test. As far as I can tell, HBaseStorage itself is fully compatible with all of HBase 0.90, 0.92, and 0.94 and won't be effected by this change at all.
> On Oct. 23, 2012, 5:07 a.m., Santhosh Srinivasan wrote:
> > test/org/apache/pig/test/TestJobSubmission.java, line 431
> > <https://reviews.apache.org/r/7676/diff/2/?file=178460#file178460line431>
> >
> > Can the commented out code be removed?
To be honest, I do not know why we keep that block of code. Nevertheless, I am hesitating to remove it since someone might have commented it out only temporarily.
- Cheolsoo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7676/#review12676
-----------------------------------------------------------
On Oct. 22, 2012, 6:50 a.m., Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7676/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2012, 6:50 a.m.)
>
>
> Review request for pig and Santhosh Srinivasan.
>
>
> Description
> -------
>
> The changes include:
>
> 1. Stop bundling hbase.jar and zookeoper.jar with pig.jar. So there should be no longer incompatibility issues when using pig.jar with different versions of hbase.jar. But to use HBaseStorage, HBASE_HOME and ZOOKEEPER_HOME must be set by the user. Note that I am adding protobuf-java.jar to pig.jar because otherwise it has to be explicitly added to PIG_CLASSPATH to use HBaseStorage, which is not very intuitive.
>
> 2. Bump hbase and zk to 0.94.1 and 3.4.3 respectively. Since we no longer bundle them in pig.jar, which versions we use doesn't matter. These jar files will be used for unit test only.
>
> 3. Make the unit test cases work with newer versions of hbase and zk.
>
> 4. Add hbase runtime dependencies to ivy.xml.
>
>
> This addresses bug PIG-2885.
> https://issues.apache.org/jira/browse/PIG-2885
>
>
> Diffs
> -----
>
> build.xml 6b04f8a
> ivy.xml 6e0a2e5
> ivy/libraries.properties 55da6c6
> test/org/apache/pig/test/TestHBaseStorage.java cc1efef
> test/org/apache/pig/test/TestJobSubmission.java 021662f
>
> Diff: https://reviews.apache.org/r/7676/diff/
>
>
> Testing
> -------
>
> ant clean test-commit -Dhadoopversion=20
> ant clean test -Dtestcase=TestHBaseStorage -Dhadoopversion=20
> ant clean test -Dtestcase=TestJobSumission -Dhadoopversion=20
>
> I also manually tested pig.jar with hbase 0.90 and 0.94. Once HBASE_HOME and ZOOKEEPER_HOME are set, HBaseStorage works fine with both versions.
>
>
> Thanks,
>
> Cheolsoo Park
>
>
Re: Review Request: TestJobSumission and TestHBaseStorage don't work with
HBase 0.94 and ZK 3.4.3
Posted by Santhosh Srinivasan <sm...@apache.org>.
> On Oct. 23, 2012, 5:07 a.m., Santhosh Srinivasan wrote:
> > ivy/libraries.properties, line 74
> > <https://reviews.apache.org/r/7676/diff/2/?file=178458#file178458line74>
> >
> > Zookeeper-3.4.4 has been out but has a known issue with SASL and Java 1.7. Is 3.3.3 required for Hbase 0.94.1 ?
>
> Cheolsoo Park wrote:
> You're asking whether ZK 3.4.3 (not 3.3.3) is required by hbase 0.94.1, right?
>
> The answer is yes. In particular, HBaseTestingUtility depends on the following ZK class, which doesn't seem to exist in ZK 3.3.3:
>
> java.lang.NoClassDefFoundError: org/apache/zookeeper/server/NIOServerCnxnFactory
>
> In fact, I don't think that we should worry about those ZK known issues because the versions of HBase and ZK that I am updating only matter to unit test. As far as I can tell, HBaseStorage itself is fully compatible with all of HBase 0.90, 0.92, and 0.94 and won't be effected by this change at all.
Actually, I was asking if we can pick up ZK-3.4.4 instead of ZK-3.4.3. Sorry, if I was not clear in my previous comment.
> On Oct. 23, 2012, 5:07 a.m., Santhosh Srinivasan wrote:
> > test/org/apache/pig/test/TestJobSubmission.java, line 431
> > <https://reviews.apache.org/r/7676/diff/2/?file=178460#file178460line431>
> >
> > Can the commented out code be removed?
>
> Cheolsoo Park wrote:
> To be honest, I do not know why we keep that block of code. Nevertheless, I am hesitating to remove it since someone might have commented it out only temporarily.
That should be fine. Can you create a separate JIRA to remove the commented out code. We are using version control for a reason. If the author is interested then (s)he can retrieve it from an earlier revision.
- Santhosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7676/#review12676
-----------------------------------------------------------
On Oct. 22, 2012, 6:50 a.m., Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7676/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2012, 6:50 a.m.)
>
>
> Review request for pig and Santhosh Srinivasan.
>
>
> Description
> -------
>
> The changes include:
>
> 1. Stop bundling hbase.jar and zookeoper.jar with pig.jar. So there should be no longer incompatibility issues when using pig.jar with different versions of hbase.jar. But to use HBaseStorage, HBASE_HOME and ZOOKEEPER_HOME must be set by the user. Note that I am adding protobuf-java.jar to pig.jar because otherwise it has to be explicitly added to PIG_CLASSPATH to use HBaseStorage, which is not very intuitive.
>
> 2. Bump hbase and zk to 0.94.1 and 3.4.3 respectively. Since we no longer bundle them in pig.jar, which versions we use doesn't matter. These jar files will be used for unit test only.
>
> 3. Make the unit test cases work with newer versions of hbase and zk.
>
> 4. Add hbase runtime dependencies to ivy.xml.
>
>
> This addresses bug PIG-2885.
> https://issues.apache.org/jira/browse/PIG-2885
>
>
> Diffs
> -----
>
> build.xml 6b04f8a
> ivy.xml 6e0a2e5
> ivy/libraries.properties 55da6c6
> test/org/apache/pig/test/TestHBaseStorage.java cc1efef
> test/org/apache/pig/test/TestJobSubmission.java 021662f
>
> Diff: https://reviews.apache.org/r/7676/diff/
>
>
> Testing
> -------
>
> ant clean test-commit -Dhadoopversion=20
> ant clean test -Dtestcase=TestHBaseStorage -Dhadoopversion=20
> ant clean test -Dtestcase=TestJobSumission -Dhadoopversion=20
>
> I also manually tested pig.jar with hbase 0.90 and 0.94. Once HBASE_HOME and ZOOKEEPER_HOME are set, HBaseStorage works fine with both versions.
>
>
> Thanks,
>
> Cheolsoo Park
>
>
Re: Review Request: TestJobSumission and TestHBaseStorage don't work with
HBase 0.94 and ZK 3.4.3
Posted by Santhosh Srinivasan <sm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7676/#review12676
-----------------------------------------------------------
A few comments. Its probably a good idea to have someone with more knowledge of HBaseStorage to take a second look.
ivy/libraries.properties
<https://reviews.apache.org/r/7676/#comment27031>
Zookeeper-3.4.4 has been out but has a known issue with SASL and Java 1.7. Is 3.3.3 required for Hbase 0.94.1 ?
test/org/apache/pig/test/TestJobSubmission.java
<https://reviews.apache.org/r/7676/#comment27032>
Can the commented out code be removed?
- Santhosh Srinivasan
On Oct. 22, 2012, 6:50 a.m., Cheolsoo Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7676/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2012, 6:50 a.m.)
>
>
> Review request for pig and Santhosh Srinivasan.
>
>
> Description
> -------
>
> The changes include:
>
> 1. Stop bundling hbase.jar and zookeoper.jar with pig.jar. So there should be no longer incompatibility issues when using pig.jar with different versions of hbase.jar. But to use HBaseStorage, HBASE_HOME and ZOOKEEPER_HOME must be set by the user. Note that I am adding protobuf-java.jar to pig.jar because otherwise it has to be explicitly added to PIG_CLASSPATH to use HBaseStorage, which is not very intuitive.
>
> 2. Bump hbase and zk to 0.94.1 and 3.4.3 respectively. Since we no longer bundle them in pig.jar, which versions we use doesn't matter. These jar files will be used for unit test only.
>
> 3. Make the unit test cases work with newer versions of hbase and zk.
>
> 4. Add hbase runtime dependencies to ivy.xml.
>
>
> This addresses bug PIG-2885.
> https://issues.apache.org/jira/browse/PIG-2885
>
>
> Diffs
> -----
>
> build.xml 6b04f8a
> ivy.xml 6e0a2e5
> ivy/libraries.properties 55da6c6
> test/org/apache/pig/test/TestHBaseStorage.java cc1efef
> test/org/apache/pig/test/TestJobSubmission.java 021662f
>
> Diff: https://reviews.apache.org/r/7676/diff/
>
>
> Testing
> -------
>
> ant clean test-commit -Dhadoopversion=20
> ant clean test -Dtestcase=TestHBaseStorage -Dhadoopversion=20
> ant clean test -Dtestcase=TestJobSumission -Dhadoopversion=20
>
> I also manually tested pig.jar with hbase 0.90 and 0.94. Once HBASE_HOME and ZOOKEEPER_HOME are set, HBaseStorage works fine with both versions.
>
>
> Thanks,
>
> Cheolsoo Park
>
>