You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/08/28 18:50:55 UTC
[GitHub] [hbase-filesystem] wchevreuil opened a new pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
wchevreuil opened a new pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] wchevreuil commented on pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15#issuecomment-684754172
"Strangerthings here":
1)
>I see this when running locally mvn clean verify -Dhadoop.profile=3 -Dhadoop3.version=3.3.0
Tried it locally. First run got same error, but subsequent runs succeeded.
2)
> When I compile a custom stack based on Hadoop 3.3.0, HBase 2.3.1, and ZooKeeper 3.5.8, such that all of Hadoop, HBase, and ZK versions are harmonized across those components (meaning: Hadoop is built with ZK 3.5.8, HBase is built with Hadoop 3.3.0 hadoop.profile=3.0 and ZK 3.5.8), I still see the ZK related runtime link errors. (NoClassDefFoundError: org/apache/zookeeper/common/X509Exception$SSLContextException)
Got same result. Among the test errors, saw this ` The value of property fs.hboss.sync.zk.connectionString must not be null` message, which makes me suspect the hbase profile specified is missing some properties definition. Looking further into this, will reply later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] wchevreuil commented on pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15#issuecomment-685894894
Right, so here some findings:
1) Setting hbase version to 2.3.1 requires updating ZK version in hboss to 3.5.7, otherwise, we get the NCDFE already mentioned above.
2) Additional adjustment is required in EmbeddedZK class to use loopback address, instead of localhost, per the changes in MiniZooKeeperCluster by HBASE-23993.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] apurtell commented on pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
apurtell commented on pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15#issuecomment-683115709
When I compile a custom stack based on Hadoop 3.3.0, HBase 2.3.1, and ZooKeeper 3.5.8, such that all of Hadoop, HBase, and ZK versions are harmonized across those components (meaning: Hadoop is built with ZK 3.5.8, HBase is built with Hadoop 3.3.0 hadoop.profile=3.0 and ZK 3.5.8), I still see the ZK related runtime link errors. (NoClassDefFoundError: org/apache/zookeeper/common/X509Exception$SSLContextException)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] apurtell edited a comment on pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
apurtell edited a comment on pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15#issuecomment-683113896
I tried mvn clean install -Dhadoop.profile=3 -Dhadoop3.version=3.3.0 -Dhbase.profile=2 -Dhbase2.version=2.3.1
[INFO] Running org.apache.hadoop.hbase.oss.contract.TestHBOSSContractSetTimes
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.363 s <<< FAILURE! - in org.apache.hadoop.hbase.oss.contract.TestHBOSSContractSetTimes
[ERROR] testSetTimesNonexistentFile(org.apache.hadoop.hbase.oss.contract.TestHBOSSContractSetTimes) Time elapsed: 0.214 s <<< ERROR!
java.io.IOException: Failed to get FS
Caused by: java.lang.reflect.InvocationTargetException
Caused by: java.lang.NoClassDefFoundError: org/apache/zookeeper/common/X509Exception$SSLContextException
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.common.X509Exception$SSLContextException
Not related to the changes proposed by this PR, but every unit test fails for me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] wchevreuil merged pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
wchevreuil merged pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] apurtell commented on pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
apurtell commented on pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15#issuecomment-683113896
I tried mvn clean install -Dhadoop.profile=3 -Dhadoop3.version=3.3.0 -Dhbase.profile=2 -Dhbase2.version=2.3.1
[INFO] Running org.apache.hadoop.hbase.oss.contract.TestHBOSSContractSetTimes
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.363 s <<< FAILURE! - in org.apache.hadoop.hbase.oss.contract.TestHBOSSContractSetTimes
[ERROR] testSetTimesNonexistentFile(org.apache.hadoop.hbase.oss.contract.TestHBOSSContractSetTimes) Time elapsed: 0.214 s <<< ERROR!
java.io.IOException: Failed to get FS
Caused by: java.lang.reflect.InvocationTargetException
Caused by: java.lang.NoClassDefFoundError: org/apache/zookeeper/common/X509Exception$SSLContextException
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.common.X509Exception$SSLContextException
Not related to this PR, but every unit test fails for me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] joshelser commented on pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
joshelser commented on pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15#issuecomment-689670441
`mvn clean install -Dhadoop.profile=3 -Dhadoop3.version=3.3.0 -Dhbase.profile=2 -Dhbase2.version=2.3.1 -Dzookeeper.version=3.5.7` worked just fine for me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] joshelser commented on pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
joshelser commented on pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15#issuecomment-683089142
```
[INFO] Running org.apache.hadoop.hbase.oss.contract.TestHBOSSContractRenameS3A
[ERROR] Tests run: 10, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 9.049 s <<< FAILURE! - in org.apache.hadoop.hbase.oss.contract.TestHBOSSContractRenameS3A
[ERROR] testRenamePopulatesFileAncestors(org.apache.hadoop.hbase.oss.contract.TestHBOSSContractRenameS3A) Time elapsed: 0.144 s <<< ERROR!
java.io.FileNotFoundException: Expected to find a directory: not found s3a://embedded/hboss-junit-fork-0001/contract-tests/testRenamePopulatesFileAncestorsNew in s3a://embedded/hboss-junit-fork-0001/contract-tests
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://embedded/hboss-junit-fork-0001/contract-tests/testRenamePopulatesFileAncestorsNew
[ERROR] testRenameNewFileSameDir(org.apache.hadoop.hbase.oss.contract.TestHBOSSContractRenameS3A) Time elapsed: 0.095 s <<< FAILURE!
java.lang.AssertionError: rename(s3a://embedded/hboss-junit-fork-0001/contract-tests/rename_src, s3a://embedded/hboss-junit-fork-0001/contract-tests/rename_dest) returned false
```
I see this when running locally `mvn clean verify -Dhadoop.profile=3 -Dhadoop3.version=3.3.0`. Am I doing something wrong?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] joshelser commented on pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
joshelser commented on pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15#issuecomment-689660267
> I was thinking maybe worth raise these dependency versions on another jira?
Yeah, don't need to hold up this one fix for a full harmonization of all branches. We can do them in pieces (hopefully with testing to catch each one and keep them stable as we're moving forward).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] wchevreuil commented on pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15#issuecomment-685047520
>When I compile a custom stack based on Hadoop 3.3.0, HBase 2.3.1, and ZooKeeper 3.5.8, such that all of Hadoop, HBase, and ZK versions are harmonized across those components (meaning: Hadoop is built with ZK 3.5.8, HBase is built with Hadoop 3.3.0 hadoop.profile=3.0 and ZK 3.5.8), I still see the ZK related runtime link errors. (NoClassDefFoundError: org/apache/zookeeper/common/X509Exception$SSLContextException)
Looked further into this, so the real issue is the NCDFE mentioned by @apurtell earlier. Yeah, hboss is defining a ZK dependency on version `3.4.10`, whilst HBase 2.3.1 is on `3.5.7`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] wchevreuil commented on pull request #15: HBASE-24963 [HBOSS] UTs failing when running against hadoop 3.3.
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on pull request #15:
URL: https://github.com/apache/hbase-filesystem/pull/15#issuecomment-685954468
Had pushed a new commit, with a change to EmbeddedZK to make sure it works with hbase versions including HBASE-23993 (like 2.3.1), for example. As mentioned on my previous comments, testing against hbase 2.3.1 works as long as ZK version specified is 3.5.7:
`mvn clean install -Dhadoop.profile=3 -Dhadoop3.version=3.3.0 -Dhbase.profile=2 -Dhbase2.version=2.3.1 -Dzookeeper.version=3.5.7`
@apurtell @joshelser , mind give it another try? I was thinking maybe worth raise these dependency versions on another jira?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org