You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yetus.apache.org by Siyao Meng <sm...@cloudera.com.INVALID> on 2019/10/05 03:14:47 UTC
Bringing Java API compliance checker to Hadoop (YETUS-445)
Hi folks,
Recently our (Cloudera) internal API checker detects an API compat issue when backporting a Hadoop commit (HDFS-14564). Though later it turns out it isn't an issue (HDFS-14564 adds a new method in an interface, but that interface isn't in any releases yet, so it's safe to do so), it would be great to have this checker in Yetus.
Previously, we've also caught another API compatibility issue with our internal checker, which is later solved in HDFS-14595 <https://issues.apache.org/jira/browse/HDFS-14595>. If we could have this API checker in Yetus, such problem could be caught and solved much earlier IMO.
This is why I commented in YETUS-445 <https://issues.apache.org/jira/browse/YETUS-445>. Would there be a problem if we use GPL binaries in Yetus? If we are only using the binaries, not including the source code, it should be fine? What am I missing here, any suggestions?
Thanks!
Siyao Meng
Re: Bringing Java API compliance checker to Hadoop (YETUS-445)
Posted by Nick Dimiduk <nd...@gmail.com>.
Hi Wei-Chiu,
I read back through the comments on YETUS-445 but without looking back
through patches, I’m missing some context. Is there a specific question we
need to bring to legal in order to get progress? Specific examples around
usage or not if category-x licensed software? Shipping source in an Apache
Yetus release vs providing code that downloads binaries at runtime?
Something like that?
Thanks,
Nick
On Mon, Oct 7, 2019 at 09:19 Wei-Chiu Chuang <we...@apache.org> wrote:
> Big +1
> We've hit this similar problem at least 3 times this year along.
>
> On Fri, Oct 4, 2019 at 8:14 PM Siyao Meng <sm...@cloudera.com> wrote:
>
> > Hi folks,
> >
> > Recently our (Cloudera) internal API checker detects an API compat
> issue
> > when backporting a Hadoop commit (HDFS-14564). Though later it turns out
> it
> > isn't an issue (HDFS-14564 adds a new method in an interface, but that
> > interface isn't in any releases yet, so it's safe to do so), *it would be
> > great to have this checker in Yetus*.
> >
> > Previously, we've also caught another API compatibility issue with our
> > internal checker, which is later solved in HDFS-14595
> > <https://issues.apache.org/jira/browse/HDFS-14595>. If we could have
> this
> > API checker in Yetus, such problem could be caught and solved much
> earlier
> > IMO.
> >
> > This is why I commented in YETUS-445
> > <https://issues.apache.org/jira/browse/YETUS-445>. Would there be a
> > problem if we use GPL binaries in Yetus? If we are only using the
> binaries,
> > not including the source code, it should be fine? What am I missing here,
> > any suggestions?
> >
> >
> > Thanks!
> > Siyao Meng
> >
>
Re: Bringing Java API compliance checker to Hadoop (YETUS-445)
Posted by Wei-Chiu Chuang <we...@apache.org>.
Big +1
We've hit this similar problem at least 3 times this year along.
On Fri, Oct 4, 2019 at 8:14 PM Siyao Meng <sm...@cloudera.com> wrote:
> Hi folks,
>
> Recently our (Cloudera) internal API checker detects an API compat issue
> when backporting a Hadoop commit (HDFS-14564). Though later it turns out it
> isn't an issue (HDFS-14564 adds a new method in an interface, but that
> interface isn't in any releases yet, so it's safe to do so), *it would be
> great to have this checker in Yetus*.
>
> Previously, we've also caught another API compatibility issue with our
> internal checker, which is later solved in HDFS-14595
> <https://issues.apache.org/jira/browse/HDFS-14595>. If we could have this
> API checker in Yetus, such problem could be caught and solved much earlier
> IMO.
>
> This is why I commented in YETUS-445
> <https://issues.apache.org/jira/browse/YETUS-445>. Would there be a
> problem if we use GPL binaries in Yetus? If we are only using the binaries,
> not including the source code, it should be fine? What am I missing here,
> any suggestions?
>
>
> Thanks!
> Siyao Meng
>
Re: Bringing Java API compliance checker to Hadoop (YETUS-445)
Posted by Sean Busbey <bu...@apache.org>.
I would love to have an API compatibility check in Yetus.
IIRC, the key issue that we got stuck on in YETUS-445 was how we provide such a tool. All of the foundational libraries for JVM compatibility (jdiff, JACC) are under a license that the foundation has deemed "category-x". That means that no ASF project can produce an artifact with a hard dependency on them.
In YETUS-445, folks wanted to have a tool that release managers could use standalone. Sort of like how there's currently Apache Yetus Release Doc Maker. Since such a tool would be its own downstream facing thing, we can't have a hard dependency on a GPL library.
We are allowed to have _optional_ dependencies on GPL libraries. So it would be perfectly fine if we add a plugin to Apache Yetus Test Patch that provided integration with JACC, or jdiff, or whatever. Since the plugins to Test Patch are all optional they can have whatever runtime dependencies we like so long as we aren't redistributing any bits that are cat-x. (and we have a long pattern of releasing plugins for Test Patch without the runtime binary bits a user needs to install to make use of the integration.)
So if you can fit the API checker into the shape of a Test Patch plugin, I'm a big +1.
On 2019/10/05 03:14:47, Siyao Meng <sm...@cloudera.com.INVALID> wrote:
> Hi folks,
>
> Recently our (Cloudera) internal API checker detects an API compat issue when backporting a Hadoop commit (HDFS-14564). Though later it turns out it isn't an issue (HDFS-14564 adds a new method in an interface, but that interface isn't in any releases yet, so it's safe to do so), it would be great to have this checker in Yetus.
>
> Previously, we've also caught another API compatibility issue with our internal checker, which is later solved in HDFS-14595 <https://issues.apache.org/jira/browse/HDFS-14595>. If we could have this API checker in Yetus, such problem could be caught and solved much earlier IMO.
>
> This is why I commented in YETUS-445 <https://issues.apache.org/jira/browse/YETUS-445>. Would there be a problem if we use GPL binaries in Yetus? If we are only using the binaries, not including the source code, it should be fine? What am I missing here, any suggestions?
>
>
> Thanks!
> Siyao Meng