You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/08/03 21:02:42 UTC
[GitHub] [hbase-filesystem] joshelser opened a new pull request #25: HBASE-25900 Hadoop 3.2 and 3.3 support
joshelser opened a new pull request #25:
URL: https://github.com/apache/hbase-filesystem/pull/25
This builds on @steveloughran's changes in #23 , pretty much doing everything he said needed to be done.
* Creates some new maven modules to separate the S3ClientFactory which inevitably won't compile (I don' think we can have the 3.2 and 3.3 versions of the factory existing on the classpath at the same time)
* Lots of indirection to determine what hadoop version is "active"
* disabling of the new tests codes on the older versions of hadoop where the rename() changes don't exist
I have _not_ gotten into testing the hadoop 2/hbase 1 compatibility at this point.
This change needs some more polish, but thought it could benefit from some eyes on it now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] joshelser merged pull request #25: HBASE-25900 Hadoop 3.2 and 3.3 support
Posted by GitBox <gi...@apache.org>.
joshelser merged pull request #25:
URL: https://github.com/apache/hbase-filesystem/pull/25
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] joshelser commented on pull request #25: HBASE-25900 Hadoop 3.2 and 3.3 support
Posted by GitBox <gi...@apache.org>.
joshelser commented on pull request #25:
URL: https://github.com/apache/hbase-filesystem/pull/25#issuecomment-934585280
```
mvn clean package
mvn clean package -Dhadoop.profile=3.2
mvn clean package -Dhadoop.profile=3.3
```
all worked for me locally. Thanks for fixing the last little bit, Peter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] petersomogyi commented on a change in pull request #25: HBASE-25900 Hadoop 3.2 and 3.3 support
Posted by GitBox <gi...@apache.org>.
petersomogyi commented on a change in pull request #25:
URL: https://github.com/apache/hbase-filesystem/pull/25#discussion_r719640566
##########
File path: pom.xml
##########
@@ -45,15 +45,16 @@
<!-- XXX If you change these versions please ensure the bundled LICENSE/NOTICE in
hbase-oss is correct
-->
+ <assertj.version>3.12.2</assertj.version>
<audience-annotations.version>0.5.0</audience-annotations.version>
<aws-java-sdk.version>1.11.1026</aws-java-sdk.version>
<commons-io.version>2.5</commons-io.version>
<commons-lang3.version>3.6</commons-lang3.version>
<curator.version>4.2.0</curator.version>
<enforcer.version>3.0.0-M3</enforcer.version>
<extra.enforcer.version>1.2</extra.enforcer.version>
- <hadoop2.version>2.9.2</hadoop2.version>
- <hadoop3.version>3.2.2</hadoop3.version>
+ <hadoop32.version>3.2.2</hadoop32.version>
+ <hadoop33.version>3.3.1</hadoop33.version>
<hbase1.version>1.4.10</hbase1.version>
<hbase2.version>2.3.6</hbase2.version>
Review comment:
Should we remove the hbase1 profile with this change as hadoop2 support is being dropped?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] petersomogyi commented on a change in pull request #25: HBASE-25900 Hadoop 3.2 and 3.3 support
Posted by GitBox <gi...@apache.org>.
petersomogyi commented on a change in pull request #25:
URL: https://github.com/apache/hbase-filesystem/pull/25#discussion_r719652902
##########
File path: pom.xml
##########
@@ -45,15 +45,16 @@
<!-- XXX If you change these versions please ensure the bundled LICENSE/NOTICE in
hbase-oss is correct
-->
+ <assertj.version>3.12.2</assertj.version>
<audience-annotations.version>0.5.0</audience-annotations.version>
<aws-java-sdk.version>1.11.1026</aws-java-sdk.version>
<commons-io.version>2.5</commons-io.version>
<commons-lang3.version>3.6</commons-lang3.version>
<curator.version>4.2.0</curator.version>
<enforcer.version>3.0.0-M3</enforcer.version>
<extra.enforcer.version>1.2</extra.enforcer.version>
- <hadoop2.version>2.9.2</hadoop2.version>
- <hadoop3.version>3.2.2</hadoop3.version>
+ <hadoop32.version>3.2.2</hadoop32.version>
+ <hadoop33.version>3.3.1</hadoop33.version>
<hbase1.version>1.4.10</hbase1.version>
<hbase2.version>2.3.6</hbase2.version>
Review comment:
Yes, HBase 1 does not support Hadoop3 based on the Ref.Guide. https://hbase.apache.org/book.html#hadoop
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] steveloughran commented on pull request #25: HBASE-25900 Hadoop 3.2 and 3.3 support
Posted by GitBox <gi...@apache.org>.
steveloughran commented on pull request #25:
URL: https://github.com/apache/hbase-filesystem/pull/25#issuecomment-892561357
looks ok to me. Sorry for ruining your life but (a) the builder API lets us add params without breaking things in future and (b) javadocs for the interface in hadoop-aws warns that the hbase team use this in tests and will be unhappy if any more changes break their build again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] joshelser commented on pull request #25: HBASE-25900 Hadoop 3.2 and 3.3 support
Posted by GitBox <gi...@apache.org>.
joshelser commented on pull request #25:
URL: https://github.com/apache/hbase-filesystem/pull/25#issuecomment-905843587
I just stumbled back across this one, left lingering on my laptop trying to figure out hadoop 2.9 support.
At this point, I think we're closing to being able to rip out HBOSS in general (after Duo and Wellington's efforts on how HBase tracks files), to just invalidate the need for HBOSS completely. Probably not worth the pain for me to try to get 2.9 working. I noticed I was treating on Steve's hallowed-ground (where JUnit3 from hadoop2 was causing test ignoring to not work).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] joshelser commented on pull request #25: HBASE-25900 Hadoop 3.2 and 3.3 support
Posted by GitBox <gi...@apache.org>.
joshelser commented on pull request #25:
URL: https://github.com/apache/hbase-filesystem/pull/25#issuecomment-905965152
Dropped Hadoop 2 support as getting Hadoop 2.9 to just pass tests was getting dodgy. If there's appetite for HBOSS to work against Hadoop2, we can revisit that later.
Can you give this a pass when you can, @wchevreuil ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-filesystem] steveloughran commented on pull request #25: HBASE-25900 Hadoop 3.2 and 3.3 support
Posted by GitBox <gi...@apache.org>.
steveloughran commented on pull request #25:
URL: https://github.com/apache/hbase-filesystem/pull/25#issuecomment-892561357
looks ok to me. Sorry for ruining your life but (a) the builder API lets us add params without breaking things in future and (b) javadocs for the interface in hadoop-aws warns that the hbase team use this in tests and will be unhappy if any more changes break their build again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org