[jira] [Commented] (HBASE-25792) Filter out o.a.hadoop.thirdparty building shaded jars

["Bryan Beaudreault (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-25792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17454729#comment-17454729 ] 

Bryan Beaudreault commented on HBASE-25792:
-------------------------------------------

[~stack]  I just stumbled across this and I think it may not be the correct solution. I wonder if you can give more context around the decision. I think it may have been more appropriate to add these classes to the allow list. Here's my reasoning:

I tried creating a downstream project which depends on {{{}hbase-shaded-client{}}}. I'd expect this to Just Work, but I get the following error when constructing a Configuration object:

{{Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/thirdparty/com/google/common/base/Preconditions
}}{{  at org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:430)
}}{{  at org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:443)
}}{{  at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:525)}}
 
Looking at the {{hbase-shaded-client}} output fat jar, it does indeed include {{org/apache/hadoop/conf/Configuration}} as I'd expect. If you decompile that Configuration, you'll see that it imports {{{}org.apache.hadoop.thirdparty.com.google.common.base.Preconditions{}}}. But due to this JIRA, we do not include that Preconditions class in the fat jar.
 
One way to work around this on the client side is to explicitly add a dependency on hadoop-shaded-guava, but that is problematic because it requires the end user to keep track of the appropriate version to include over time. It also only solves for guava, when there is also protobuf and potentially other thirdparty artifacts that might cause issues over time.
 
Thoughts on reverting this and instead adding to the allow list? If you agree I can create a new Jira to track.

> Filter out o.a.hadoop.thirdparty building shaded jars
> -----------------------------------------------------
>
>                 Key: HBASE-25792
>                 URL: https://issues.apache.org/jira/browse/HBASE-25792
>             Project: HBase
>          Issue Type: Bug
>          Components: shading
>    Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.4.3
>            Reporter: Michael Stack
>            Assignee: Michael Stack
>            Priority: Major
>             Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3
>
>
> Hadoop 3.3.1 (unreleased currently) shades guava. The shaded guava then trips the check in our shading that tries to exclude hadoop bits from the fat jars we build.
> For the issue to trigger, need to build against tip of hadoop branch-3.3. You then get this complaint:
> {code}
> [INFO] --- exec-maven-plugin:1.6.0:exec (check-jar-contents) @ hbase-shaded-check-invariants ---
> [ERROR] Found artifact with unexpected contents: '/Users/stack/.m2/repository/org/apache/hbase/hbase-shaded-mapreduce/2.3.6-SNAPSHOT/hbase-shaded-mapreduce-2.3.6-SNAPSHOT.jar'
>     Please check the following and either correct the build or update
>     the allowed list with reasoning.
>     org/apache/hadoop/thirdparty/
>     org/apache/hadoop/thirdparty/com/
>     org/apache/hadoop/thirdparty/com/google/
>     org/apache/hadoop/thirdparty/com/google/common/
>     org/apache/hadoop/thirdparty/com/google/common/annotations/
>     org/apache/hadoop/thirdparty/com/google/common/annotations/Beta.class
>     org/apache/hadoop/thirdparty/com/google/common/annotations/GwtCompatible.class
>     org/apache/hadoop/thirdparty/com/google/common/annotations/GwtIncompatible.class
>     org/apache/hadoop/thirdparty/com/google/common/annotations/VisibleForTesting.class
>     org/apache/hadoop/thirdparty/com/google/common/base/
>     org/apache/hadoop/thirdparty/com/google/common/base/Absent.class
>     org/apache/hadoop/thirdparty/com/google/common/base/AbstractIterator$1.class
>     org/apache/hadoop/thirdparty/com/google/common/base/AbstractIterator$State.class
>     org/apache/hadoop/thirdparty/com/google/common/base/AbstractIterator.class
>     org/apache/hadoop/thirdparty/com/google/common/base/Ascii.class
>     org/apache/hadoop/thirdparty/com/google/common/base/CaseFormat$1.class
>     org/apache/hadoop/thirdparty/com/google/common/base/CaseFormat$2.class
>     org/apache/hadoop/thirdparty/com/google/common/base/CaseFormat$3.class
>     org/apache/hadoop/thirdparty/com/google/common/base/CaseFormat$4.class
> ....
> {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)