You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Bryan Beaudreault <bb...@hubspot.com.INVALID> on 2021/06/11 17:23:56 UTC

Building 3.2.2 -- hadoop-client-check-test-invariants failing on edu.umd.cs.findbugs

Hey all,

I'm trying to build hadoop from source. I've checked out the
rel/release-3.2.2 tag from github.com/apache/hadoop. The maven build gets
most of the way through, but fails on hadoop-client-check-test-invariants
with the following error:

[ERROR] Found artifact with unexpected contents:
'/hadoop/rpm/scratch/BUILD/hadoop-3.2.2-src/hadoop-client-modules/hadoop-client-minicluster/target/hadoop-client-minicluster-3.2.2.jar'
    Please check the following and either correct the build or update
    the allowed list with reasoning.

    edu/
    edu/umd/
    edu/umd/cs/
    edu/umd/cs/findbugs/
    edu/umd/cs/findbugs/annotations/
    edu/umd/cs/findbugs/annotations/NonNull.class
    ... snip (there's more classes in this package) ...

I know I could modify the shell script to allow this, or I could try to
modify the pom to exclude it. But it seems weird that I'm hitting this at
all. I have not made any changes to the source yet.

Has anyone else hit this or have any idea why it's happening? I can't find
any reference to "edu/umd" or "edu.umd" in the source. I'm guessing it's
coming from a transitive dependency, but I tried running mvn
dependency:tree and did not see any mention of this version of findbugs.

Thanks!

Re: Building 3.2.2 -- hadoop-client-check-test-invariants failing on edu.umd.cs.findbugs

Posted by Bryan Beaudreault <bb...@hubspot.com.INVALID>.
Actually now that you mention it, I see someone already created such an
issue in 2019. https://issues.apache.org/jira/browse/ZOOKEEPER-3367. I'm
guessing it's gotten no attention because 3.4 is EOL. Not sure if this
persists in later versions.

I actually would like to update the zookeeper version beyond 3.4, but
unfortunately there are compile time issues that came into play when i
tried that. That's probably a convo for a different thread though. For now
I think I'm ok compiling against 3.4.13.

Thanks!

On Fri, Jun 11, 2021 at 1:54 PM Sean Busbey <sb...@apple.com> wrote:

> If you want to update to that zookeeper version then you should update the
> build files to exclude those classes from getting included transitively
> from it.
>
> Would you mind filing a bug against zookeeper as well?
> spotbugs-annotations 3.1.9 is LGPL, so they should not be exposing it as a
> downstream transitive dependency.
>
>
>
> > On Jun 11, 2021, at 12:47 PM, Bryan Beaudreault
> <bb...@hubspot.com.INVALID> wrote:
> >
> > Ok sorry, false alarm. I figured this out by cleaning out my .m2
> > repository, re-building just hadoop-client-minicluster, and then
> iterating
> > all resulting jars in the .m2 using jar -tf with grep to find which jar
> was
> > providing these classes. Turns out it was
> > com.github.spotbugs:spotbugs-annotations:jar:3.1.9, which I could then
> see
> > in dependency:tree as coming from zookeeper 3.4.14. My build environment
> > was setting the zookeeper.version property to 3.4.14, and it seems like
> > this was a new dependency added between the hadoop-default version of
> > 3.4.13 and 14.
> >
> > The build works with 3.4.13, so now I can decide whether to use that
> > version or update the invariant check accordingly.
> >
> > On Fri, Jun 11, 2021 at 1:23 PM Bryan Beaudreault <
> bbeaudreault@hubspot.com>
> > wrote:
> >
> >> Hey all,
> >>
> >> I'm trying to build hadoop from source. I've checked out the
> >> rel/release-3.2.2 tag from github.com/apache/hadoop
> <http://github.com/apache/hadoop>.
> The maven build gets
> >> most of the way through, but fails on
> hadoop-client-check-test-invariants
> >> with the following error:
> >>
> >> [ERROR] Found artifact with unexpected contents:
> >>
> '/hadoop/rpm/scratch/BUILD/hadoop-3.2.2-src/hadoop-client-modules/hadoop-client-minicluster/target/hadoop-client-minicluster-3.2.2.jar'
> >> Please check the following and either correct the build or update
> >> the allowed list with reasoning.
> >>
> >> edu/
> >> edu/umd/
> >> edu/umd/cs/
> >> edu/umd/cs/findbugs/
> >> edu/umd/cs/findbugs/annotations/
> >> edu/umd/cs/findbugs/annotations/NonNull.class
> >> ... snip (there's more classes in this package) ...
> >>
> >> I know I could modify the shell script to allow this, or I could try to
> >> modify the pom to exclude it. But it seems weird that I'm hitting this
> at
> >> all. I have not made any changes to the source yet.
> >>
> >> Has anyone else hit this or have any idea why it's happening? I can't
> find
> >> any reference to "edu/umd" or "edu.umd" in the source. I'm guessing it's
> >> coming from a transitive dependency, but I tried running mvn
> >> dependency:tree and did not see any mention of this version of findbugs.
> >>
> >> Thanks!
> >>
>
>

Re: Building 3.2.2 -- hadoop-client-check-test-invariants failing on edu.umd.cs.findbugs

Posted by Sean Busbey <sb...@apple.com.INVALID>.
If you want to update to that zookeeper version then you should update the build files to exclude those classes from getting included transitively from it.

Would you mind filing a bug against zookeeper as well? spotbugs-annotations 3.1.9 is LGPL, so they should not be exposing it as a downstream transitive dependency.



> On Jun 11, 2021, at 12:47 PM, Bryan Beaudreault <bb...@hubspot.com.INVALID> wrote:
> 
> Ok sorry, false alarm. I figured this out by cleaning out my .m2
> repository, re-building just hadoop-client-minicluster, and then iterating
> all resulting jars in the .m2 using jar -tf with grep to find which jar was
> providing these classes. Turns out it was
> com.github.spotbugs:spotbugs-annotations:jar:3.1.9, which I could then see
> in dependency:tree as coming from zookeeper 3.4.14. My build environment
> was setting the zookeeper.version property to 3.4.14, and it seems like
> this was a new dependency added between the hadoop-default version of
> 3.4.13 and 14.
> 
> The build works with 3.4.13, so now I can decide whether to use that
> version or update the invariant check accordingly.
> 
> On Fri, Jun 11, 2021 at 1:23 PM Bryan Beaudreault <bb...@hubspot.com>
> wrote:
> 
>> Hey all,
>> 
>> I'm trying to build hadoop from source. I've checked out the
>> rel/release-3.2.2 tag from github.com/apache/hadoop. The maven build gets
>> most of the way through, but fails on hadoop-client-check-test-invariants
>> with the following error:
>> 
>> [ERROR] Found artifact with unexpected contents:
>> '/hadoop/rpm/scratch/BUILD/hadoop-3.2.2-src/hadoop-client-modules/hadoop-client-minicluster/target/hadoop-client-minicluster-3.2.2.jar'
>>   Please check the following and either correct the build or update
>>   the allowed list with reasoning.
>> 
>>   edu/
>>   edu/umd/
>>   edu/umd/cs/
>>   edu/umd/cs/findbugs/
>>   edu/umd/cs/findbugs/annotations/
>>   edu/umd/cs/findbugs/annotations/NonNull.class
>>   ... snip (there's more classes in this package) ...
>> 
>> I know I could modify the shell script to allow this, or I could try to
>> modify the pom to exclude it. But it seems weird that I'm hitting this at
>> all. I have not made any changes to the source yet.
>> 
>> Has anyone else hit this or have any idea why it's happening? I can't find
>> any reference to "edu/umd" or "edu.umd" in the source. I'm guessing it's
>> coming from a transitive dependency, but I tried running mvn
>> dependency:tree and did not see any mention of this version of findbugs.
>> 
>> Thanks!
>> 



---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: Building 3.2.2 -- hadoop-client-check-test-invariants failing on edu.umd.cs.findbugs

Posted by Bryan Beaudreault <bb...@hubspot.com.INVALID>.
Ok sorry, false alarm. I figured this out by cleaning out my .m2
repository, re-building just hadoop-client-minicluster, and then iterating
all resulting jars in the .m2 using jar -tf with grep to find which jar was
providing these classes. Turns out it was
com.github.spotbugs:spotbugs-annotations:jar:3.1.9, which I could then see
in dependency:tree as coming from zookeeper 3.4.14. My build environment
was setting the zookeeper.version property to 3.4.14, and it seems like
this was a new dependency added between the hadoop-default version of
3.4.13 and 14.

The build works with 3.4.13, so now I can decide whether to use that
version or update the invariant check accordingly.

On Fri, Jun 11, 2021 at 1:23 PM Bryan Beaudreault <bb...@hubspot.com>
wrote:

> Hey all,
>
> I'm trying to build hadoop from source. I've checked out the
> rel/release-3.2.2 tag from github.com/apache/hadoop. The maven build gets
> most of the way through, but fails on hadoop-client-check-test-invariants
> with the following error:
>
> [ERROR] Found artifact with unexpected contents:
> '/hadoop/rpm/scratch/BUILD/hadoop-3.2.2-src/hadoop-client-modules/hadoop-client-minicluster/target/hadoop-client-minicluster-3.2.2.jar'
>     Please check the following and either correct the build or update
>     the allowed list with reasoning.
>
>     edu/
>     edu/umd/
>     edu/umd/cs/
>     edu/umd/cs/findbugs/
>     edu/umd/cs/findbugs/annotations/
>     edu/umd/cs/findbugs/annotations/NonNull.class
>     ... snip (there's more classes in this package) ...
>
> I know I could modify the shell script to allow this, or I could try to
> modify the pom to exclude it. But it seems weird that I'm hitting this at
> all. I have not made any changes to the source yet.
>
> Has anyone else hit this or have any idea why it's happening? I can't find
> any reference to "edu/umd" or "edu.umd" in the source. I'm guessing it's
> coming from a transitive dependency, but I tried running mvn
> dependency:tree and did not see any mention of this version of findbugs.
>
> Thanks!
>