You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hbase.apache.org by Dan Huff <da...@dremio.com.INVALID> on 2024/01/02 19:39:45 UTC

Hbase, Hbase-thirdparty, and netty

Hello there Hbase Devs--

I have been investigating taking an update to Hbase 2.5.7 after the release
last week and have what I hope is a quick question about commit 7639345
<https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9>
in
branch-2.5.

Am I correct in believing that the direct inclusion of netty 4.1.100.Final
in Hbase's pom.xml will override the 4.1.97.Final version that is
specified in hbase-thirdparty
<https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137>? I
see 4.1.100.Final listed on
https://hbase.apache.org/dependency-management.html which to me suggests
that I am understanding this correctly that issues flagged against
4.1.97.Final can be ignored since Hbase will now just use 4.1.100.Final.

Thanks so much for your time,

Dan Huff

Re: Hbase, Hbase-thirdparty, and netty

Posted by "rajeshbabu@apache.org" <ch...@gmail.com>.

jetty version upgrade patch committed to hbase-thirdparty master.

Thanks,
Rajeshbabu.

On Fri, Jan 5, 2024 at 9:38 PM rajeshbabu@apache.org <
chrajeshbabu32@gmail.com> wrote:

> Raised PR for jetty version bump up.
> https://github.com/apache/hbase-thirdparty/pull/107
>
> Thanks,
> Rajeshbabu.
>
> On Fri, Jan 5, 2024 at 7:12 PM Bryan Beaudreault <bb...@apache.org>
> wrote:
>
>> The netty version bump is committed to hbase-thirdparty master. The jetty
>> one looks good to include as well. Do you want to handle that one Nihal?
>>
>> On Fri, Jan 5, 2024 at 2:25 AM Nihal Jain <ni...@gmail.com> wrote:
>>
>> > Since we are planning a new thirdparty release, IMO we also put
>> > https://issues.apache.org/jira/browse/HBASE-28279, which bumps to
>> latest
>> > jetty. Package jetty-http, that we bundle (
>> >
>> >
>> https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-http/9.4.52.v20230823
>> > ),
>> > has a direct CVE
>> > https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-36478
>> >
>> > Regards,
>> > Nihal
>> >
>> > On Fri, 5 Jan 2024, 02:43 Andrew Purtell, <ap...@apache.org> wrote:
>> >
>> > > Sounds good. I think we agree on everything.
>> > >
>> > > On Thu, Jan 4, 2024 at 12:37 PM Bryan Beaudreault <
>> > bbeaudreault@apache.org
>> > > >
>> > > wrote:
>> > >
>> > > > To clarify, I was referring to "It’s weird to have netty dependency
>> > > > versions diverging." Which I do agree with, but was explaining my
>> > > > understanding of the rationale.
>> > > >
>> > > > Otherwise, I agree with everything else. I have the PR ready for
>> > updating
>> > > > netty in hbase-thirdparty. I can also vote and do the bump of
>> > > > hbase-thirdparty.version in hbase once the release is made.
>> > > >
>> > > > On Thu, Jan 4, 2024 at 3:15 PM Andrew Purtell <ap...@apache.org>
>> > > wrote:
>> > > >
>> > > > > We should not ship a netty with known CVEs in hbase-thirdparty.
>> > > Likewise
>> > > > > for the other components shipped in hbase-thirdparty. As with all
>> > > things
>> > > > > this is a guideline, not a rule, because each situation is
>> different
>> > > and
>> > > > > people do not always share the same opinion.
>> > > > >
>> > > > > I am of the opinion that moderate to high scoring CVEs in a
>> > dependency
>> > > --
>> > > > > and it doesn't matter if direct, transitive, or shaded -- is
>> going to
>> > > be
>> > > > a
>> > > > > problem for many users or potential users simply if they exist in
>> our
>> > > > bill
>> > > > > of materials. At my employer we need to juggle the "cleanliness"
>> of
>> > our
>> > > > > software bill of materials among priorities and I do not think we
>> are
>> > > > > exceptional in any way. We do it in a fork but I have been
>> meaning to
>> > > do
>> > > > a
>> > > > > pass over the public open source project's dependency set, if this
>> > > would
>> > > > > have some value for the project (which I believe there is).
>> > > > >
>> > > > > I can do a thirdparty release now if everyone else is busy.
>> > > > >
>> > > > > On Thu, Jan 4, 2024 at 10:50 AM Bryan Beaudreault <
>> > > > bbeaudreault@apache.org
>> > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > I think we explicitly don't want our hbase-thirdparty netty
>> version
>> > > to
>> > > > be
>> > > > > > locked to the one for transitive dependencies. That's sort of
>> why
>> > we
>> > > > have
>> > > > > > it in thirdparty/shaded at all, right?
>> > > > > >
>> > > > > > I created https://issues.apache.org/jira/browse/HBASE-28291 to
>> > > update
>> > > > > > pom.xml in hbase-thirdparty. I will handle that. We will need
>> to do
>> > > an
>> > > > > > hbase-thirdparty release, which I'm not sure I'll have time for
>> > given
>> > > > I'm
>> > > > > > already behind on the 2.6.0 release
>> > > > > >
>> > > > > > On Thu, Jan 4, 2024 at 11:58 AM Andrew Purtell <
>> > > > andrew.purtell@gmail.com
>> > > > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > > > We should do that bump to hbase-thirdparty and spin another
>> > release
>> > > > to
>> > > > > > > keep our house in order. It isn’t urgent but would be good to
>> > > address
>> > > > > > this
>> > > > > > > in the normal release cadence. That has been about once per
>> > fiscal
>> > > > > > quarter
>> > > > > > > recently.  It’s weird to have netty dependency versions
>> > diverging.
>> > > > > > >
>> > > > > > > I will make a note as RM to look at hbase-thirdparty versions
>> > with
>> > > > > > respect
>> > > > > > > to the base POM and known security issues, using snyk
>> probably,
>> > and
>> > > > > > update
>> > > > > > > it ahead of 2.5.8. As well as direct dependencies in the base
>> > POM.
>> > > > > > > Unfortunately I can’t promise to do anything about transitive
>> > > issues
>> > > > > > > imported from something that impacts operational
>> compatibility.
>> > > Those
>> > > > > > must
>> > > > > > > be weighed case by case.
>> > > > > > >
>> > > > > > > > On Jan 4, 2024, at 8:31 AM, Bryan Beaudreault <
>> > > > > bbeaudreault@apache.org
>> > > > > > >
>> > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > It looks like that CVE only affects
>> > io.netty:netty-codec-http2.
>> > > > > Since
>> > > > > > > our
>> > > > > > > > hbase-shaded-netty depends on netty-all, that module is
>> > included.
>> > > > > > > However,
>> > > > > > > > I don't think we use anything from netty-codec-http2. So I
>> > don't
>> > > > > think
>> > > > > > > the
>> > > > > > > > CVE is a risk for this usage, unless you are building an app
>> > > using
>> > > > > the
>> > > > > > > > org.apache.hbase.thirdparty.io.netty classes. This would
>> not be
>> > > > > > advised.
>> > > > > > > >
>> > > > > > > > That said, we could try to bump hbase-thirdparty to 4.1.100+
>> > and
>> > > > > > include
>> > > > > > > > that in the upcoming 2.6.0 or 2.5.8 when that happens. If
>> the
>> > CVE
>> > > > > were
>> > > > > > > > critical we could rush out another minor release, but I
>> don't
>> > > think
>> > > > > > it's
>> > > > > > > > necessary here? I also wonder if we should update
>> > > > hbase-shaded-netty
>> > > > > to
>> > > > > > > > only pull in the netty modules we actually use.
>> > > > > > > >
>> > > > > > > >> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff
>> > > > > <dan.huff@dremio.com.invalid
>> > > > > > >
>> > > > > > > >> wrote:
>> > > > > > > >>
>> > > > > > > >> Thanks Bryan. That does help explain things. I have been
>> > looking
>> > > > at
>> > > > > > > >>
>> > > > > >
>> > >
>> https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p
>> > > > > > > and
>> > > > > > > >> have been trying to determine if hbase is vulnerable to
>> this
>> > > > attack
>> > > > > > > vector
>> > > > > > > >> or not. I got excited when I saw 4.1.100.Final in 2.5.7
>> but it
>> > > > > sounds
>> > > > > > > like
>> > > > > > > >> that excitement was misplaced :)
>> > > > > > > >>
>> > > > > > > >> Dan
>> > > > > > > >>
>> > > > > > > >> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <
>> > > > > > > bbeaudreault@apache.org
>> > > > > > > >>>
>> > > > > > > >> wrote:
>> > > > > > > >>
>> > > > > > > >>> Hello,
>> > > > > > > >>>
>> > > > > > > >>> As the comment above the netty version change says, this
>> only
>> > > > > affects
>> > > > > > > the
>> > > > > > > >>> transitive netty dependencies from thirdparty dependencies
>> > like
>> > > > > > > zookeeper
>> > > > > > > >>> and hadoop. HBase's internal netty usage (i.e. for HBase's
>> > RPC
>> > > > > > > protocol)
>> > > > > > > >>> uses the shaded netty provided by hbase-thirdparty.
>> > > > > > > >>>
>> > > > > > > >>> While you're generally correct that in maven you'd expect
>> a
>> > > > version
>> > > > > > > >> defined
>> > > > > > > >>> in dependencyManagement to affect all transitive
>> > dependencies,
>> > > > that
>> > > > > > is
>> > > > > > > >> not
>> > > > > > > >>> the case for hbase-thirdparty due to the shading we do
>> there.
>> > > At
>> > > > > the
>> > > > > > > time
>> > > > > > > >>> of building hbase-thirdparty, the defined netty version
>> there
>> > > is
>> > > > > > pulled
>> > > > > > > >> in
>> > > > > > > >>> and relocated to org.apache.hbase.thirdparty.io.netty and
>> > > > published
>> > > > > > as
>> > > > > > > a
>> > > > > > > >>> new maven module named hbase-shaded-netty. As such, the
>> > > > > > > >>> dependencyManagement has no effect on it.
>> > > > > > > >>>
>> > > > > > > >>> I hope this helps
>> > > > > > > >>>
>> > > > > > > >>> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff
>> > > > > <dan.huff@dremio.com.invalid
>> > > > > > >
>> > > > > > > >>> wrote:
>> > > > > > > >>>
>> > > > > > > >>>> Hello there Hbase Devs--
>> > > > > > > >>>>
>> > > > > > > >>>> I have been investigating taking an update to Hbase 2.5.7
>> > > after
>> > > > > the
>> > > > > > > >>> release
>> > > > > > > >>>> last week and have what I hope is a quick question about
>> > > commit
>> > > > > > > 7639345
>> > > > > > > >>>> <
>> > > > > > > >>>>
>> > > > > > > >>>
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
>> > > > > > > >>>>>
>> > > > > > > >>>> in
>> > > > > > > >>>> branch-2.5.
>> > > > > > > >>>>
>> > > > > > > >>>> Am I correct in believing that the direct inclusion of
>> netty
>> > > > > > > >>> 4.1.100.Final
>> > > > > > > >>>> in Hbase's pom.xml will override the 4.1.97.Final version
>> > that
>> > > > is
>> > > > > > > >>>> specified in hbase-thirdparty
>> > > > > > > >>>> <
>> > > > > > > >>
>> > > > > >
>> > >
>> https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
>> > > > > > > >>>> ?
>> > > > > > > >>>> I
>> > > > > > > >>>> see 4.1.100.Final listed on
>> > > > > > > >>>> https://hbase.apache.org/dependency-management.html
>> which
>> > to
>> > > me
>> > > > > > > >> suggests
>> > > > > > > >>>> that I am understanding this correctly that issues
>> flagged
>> > > > against
>> > > > > > > >>>> 4.1.97.Final can be ignored since Hbase will now just use
>> > > > > > > >> 4.1.100.Final.
>> > > > > > > >>>>
>> > > > > > > >>>> Thanks so much for your time,
>> > > > > > > >>>>
>> > > > > > > >>>> Dan Huff
>> > > > > > > >>>>
>> > > > > > > >>>
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Best regards,
>> > > > > Andrew
>> > > > >
>> > > > > Unrest, ignorance distilled, nihilistic imbeciles -
>> > > > >     It's what we’ve earned
>> > > > > Welcome, apocalypse, what’s taken you so long?
>> > > > > Bring us the fitting end that we’ve been counting on
>> > > > >    - A23, Welcome, Apocalypse
>> > > > >
>> > > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > > Andrew
>> > >
>> > > Unrest, ignorance distilled, nihilistic imbeciles -
>> > >     It's what we’ve earned
>> > > Welcome, apocalypse, what’s taken you so long?
>> > > Bring us the fitting end that we’ve been counting on
>> > >    - A23, Welcome, Apocalypse
>> > >
>> >
>>
>

Re: Hbase, Hbase-thirdparty, and netty

Posted by "rajeshbabu@apache.org" <ch...@gmail.com>.

Raised PR for jetty version bump up.
https://github.com/apache/hbase-thirdparty/pull/107

Thanks,
Rajeshbabu.

On Fri, Jan 5, 2024 at 7:12 PM Bryan Beaudreault <bb...@apache.org>
wrote:

> The netty version bump is committed to hbase-thirdparty master. The jetty
> one looks good to include as well. Do you want to handle that one Nihal?
>
> On Fri, Jan 5, 2024 at 2:25 AM Nihal Jain <ni...@gmail.com> wrote:
>
> > Since we are planning a new thirdparty release, IMO we also put
> > https://issues.apache.org/jira/browse/HBASE-28279, which bumps to latest
> > jetty. Package jetty-http, that we bundle (
> >
> >
> https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-http/9.4.52.v20230823
> > ),
> > has a direct CVE
> > https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-36478
> >
> > Regards,
> > Nihal
> >
> > On Fri, 5 Jan 2024, 02:43 Andrew Purtell, <ap...@apache.org> wrote:
> >
> > > Sounds good. I think we agree on everything.
> > >
> > > On Thu, Jan 4, 2024 at 12:37 PM Bryan Beaudreault <
> > bbeaudreault@apache.org
> > > >
> > > wrote:
> > >
> > > > To clarify, I was referring to "It’s weird to have netty dependency
> > > > versions diverging." Which I do agree with, but was explaining my
> > > > understanding of the rationale.
> > > >
> > > > Otherwise, I agree with everything else. I have the PR ready for
> > updating
> > > > netty in hbase-thirdparty. I can also vote and do the bump of
> > > > hbase-thirdparty.version in hbase once the release is made.
> > > >
> > > > On Thu, Jan 4, 2024 at 3:15 PM Andrew Purtell <ap...@apache.org>
> > > wrote:
> > > >
> > > > > We should not ship a netty with known CVEs in hbase-thirdparty.
> > > Likewise
> > > > > for the other components shipped in hbase-thirdparty. As with all
> > > things
> > > > > this is a guideline, not a rule, because each situation is
> different
> > > and
> > > > > people do not always share the same opinion.
> > > > >
> > > > > I am of the opinion that moderate to high scoring CVEs in a
> > dependency
> > > --
> > > > > and it doesn't matter if direct, transitive, or shaded -- is going
> to
> > > be
> > > > a
> > > > > problem for many users or potential users simply if they exist in
> our
> > > > bill
> > > > > of materials. At my employer we need to juggle the "cleanliness" of
> > our
> > > > > software bill of materials among priorities and I do not think we
> are
> > > > > exceptional in any way. We do it in a fork but I have been meaning
> to
> > > do
> > > > a
> > > > > pass over the public open source project's dependency set, if this
> > > would
> > > > > have some value for the project (which I believe there is).
> > > > >
> > > > > I can do a thirdparty release now if everyone else is busy.
> > > > >
> > > > > On Thu, Jan 4, 2024 at 10:50 AM Bryan Beaudreault <
> > > > bbeaudreault@apache.org
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > I think we explicitly don't want our hbase-thirdparty netty
> version
> > > to
> > > > be
> > > > > > locked to the one for transitive dependencies. That's sort of why
> > we
> > > > have
> > > > > > it in thirdparty/shaded at all, right?
> > > > > >
> > > > > > I created https://issues.apache.org/jira/browse/HBASE-28291 to
> > > update
> > > > > > pom.xml in hbase-thirdparty. I will handle that. We will need to
> do
> > > an
> > > > > > hbase-thirdparty release, which I'm not sure I'll have time for
> > given
> > > > I'm
> > > > > > already behind on the 2.6.0 release
> > > > > >
> > > > > > On Thu, Jan 4, 2024 at 11:58 AM Andrew Purtell <
> > > > andrew.purtell@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > We should do that bump to hbase-thirdparty and spin another
> > release
> > > > to
> > > > > > > keep our house in order. It isn’t urgent but would be good to
> > > address
> > > > > > this
> > > > > > > in the normal release cadence. That has been about once per
> > fiscal
> > > > > > quarter
> > > > > > > recently.  It’s weird to have netty dependency versions
> > diverging.
> > > > > > >
> > > > > > > I will make a note as RM to look at hbase-thirdparty versions
> > with
> > > > > > respect
> > > > > > > to the base POM and known security issues, using snyk probably,
> > and
> > > > > > update
> > > > > > > it ahead of 2.5.8. As well as direct dependencies in the base
> > POM.
> > > > > > > Unfortunately I can’t promise to do anything about transitive
> > > issues
> > > > > > > imported from something that impacts operational compatibility.
> > > Those
> > > > > > must
> > > > > > > be weighed case by case.
> > > > > > >
> > > > > > > > On Jan 4, 2024, at 8:31 AM, Bryan Beaudreault <
> > > > > bbeaudreault@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > It looks like that CVE only affects
> > io.netty:netty-codec-http2.
> > > > > Since
> > > > > > > our
> > > > > > > > hbase-shaded-netty depends on netty-all, that module is
> > included.
> > > > > > > However,
> > > > > > > > I don't think we use anything from netty-codec-http2. So I
> > don't
> > > > > think
> > > > > > > the
> > > > > > > > CVE is a risk for this usage, unless you are building an app
> > > using
> > > > > the
> > > > > > > > org.apache.hbase.thirdparty.io.netty classes. This would not
> be
> > > > > > advised.
> > > > > > > >
> > > > > > > > That said, we could try to bump hbase-thirdparty to 4.1.100+
> > and
> > > > > > include
> > > > > > > > that in the upcoming 2.6.0 or 2.5.8 when that happens. If the
> > CVE
> > > > > were
> > > > > > > > critical we could rush out another minor release, but I don't
> > > think
> > > > > > it's
> > > > > > > > necessary here? I also wonder if we should update
> > > > hbase-shaded-netty
> > > > > to
> > > > > > > > only pull in the netty modules we actually use.
> > > > > > > >
> > > > > > > >> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff
> > > > > <dan.huff@dremio.com.invalid
> > > > > > >
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >> Thanks Bryan. That does help explain things. I have been
> > looking
> > > > at
> > > > > > > >>
> > > > > >
> > > https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p
> > > > > > > and
> > > > > > > >> have been trying to determine if hbase is vulnerable to this
> > > > attack
> > > > > > > vector
> > > > > > > >> or not. I got excited when I saw 4.1.100.Final in 2.5.7 but
> it
> > > > > sounds
> > > > > > > like
> > > > > > > >> that excitement was misplaced :)
> > > > > > > >>
> > > > > > > >> Dan
> > > > > > > >>
> > > > > > > >> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <
> > > > > > > bbeaudreault@apache.org
> > > > > > > >>>
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >>> Hello,
> > > > > > > >>>
> > > > > > > >>> As the comment above the netty version change says, this
> only
> > > > > affects
> > > > > > > the
> > > > > > > >>> transitive netty dependencies from thirdparty dependencies
> > like
> > > > > > > zookeeper
> > > > > > > >>> and hadoop. HBase's internal netty usage (i.e. for HBase's
> > RPC
> > > > > > > protocol)
> > > > > > > >>> uses the shaded netty provided by hbase-thirdparty.
> > > > > > > >>>
> > > > > > > >>> While you're generally correct that in maven you'd expect a
> > > > version
> > > > > > > >> defined
> > > > > > > >>> in dependencyManagement to affect all transitive
> > dependencies,
> > > > that
> > > > > > is
> > > > > > > >> not
> > > > > > > >>> the case for hbase-thirdparty due to the shading we do
> there.
> > > At
> > > > > the
> > > > > > > time
> > > > > > > >>> of building hbase-thirdparty, the defined netty version
> there
> > > is
> > > > > > pulled
> > > > > > > >> in
> > > > > > > >>> and relocated to org.apache.hbase.thirdparty.io.netty and
> > > > published
> > > > > > as
> > > > > > > a
> > > > > > > >>> new maven module named hbase-shaded-netty. As such, the
> > > > > > > >>> dependencyManagement has no effect on it.
> > > > > > > >>>
> > > > > > > >>> I hope this helps
> > > > > > > >>>
> > > > > > > >>> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff
> > > > > <dan.huff@dremio.com.invalid
> > > > > > >
> > > > > > > >>> wrote:
> > > > > > > >>>
> > > > > > > >>>> Hello there Hbase Devs--
> > > > > > > >>>>
> > > > > > > >>>> I have been investigating taking an update to Hbase 2.5.7
> > > after
> > > > > the
> > > > > > > >>> release
> > > > > > > >>>> last week and have what I hope is a quick question about
> > > commit
> > > > > > > 7639345
> > > > > > > >>>> <
> > > > > > > >>>>
> > > > > > > >>>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> > > > > > > >>>>>
> > > > > > > >>>> in
> > > > > > > >>>> branch-2.5.
> > > > > > > >>>>
> > > > > > > >>>> Am I correct in believing that the direct inclusion of
> netty
> > > > > > > >>> 4.1.100.Final
> > > > > > > >>>> in Hbase's pom.xml will override the 4.1.97.Final version
> > that
> > > > is
> > > > > > > >>>> specified in hbase-thirdparty
> > > > > > > >>>> <
> > > > > > > >>
> > > > > >
> > > https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
> > > > > > > >>>> ?
> > > > > > > >>>> I
> > > > > > > >>>> see 4.1.100.Final listed on
> > > > > > > >>>> https://hbase.apache.org/dependency-management.html which
> > to
> > > me
> > > > > > > >> suggests
> > > > > > > >>>> that I am understanding this correctly that issues flagged
> > > > against
> > > > > > > >>>> 4.1.97.Final can be ignored since Hbase will now just use
> > > > > > > >> 4.1.100.Final.
> > > > > > > >>>>
> > > > > > > >>>> Thanks so much for your time,
> > > > > > > >>>>
> > > > > > > >>>> Dan Huff
> > > > > > > >>>>
> > > > > > > >>>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Andrew
> > > > >
> > > > > Unrest, ignorance distilled, nihilistic imbeciles -
> > > > >     It's what we’ve earned
> > > > > Welcome, apocalypse, what’s taken you so long?
> > > > > Bring us the fitting end that we’ve been counting on
> > > > >    - A23, Welcome, Apocalypse
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Unrest, ignorance distilled, nihilistic imbeciles -
> > >     It's what we’ve earned
> > > Welcome, apocalypse, what’s taken you so long?
> > > Bring us the fitting end that we’ve been counting on
> > >    - A23, Welcome, Apocalypse
> > >
> >
>

Re: Hbase, Hbase-thirdparty, and netty

Posted by Bryan Beaudreault <bb...@apache.org>.

The netty version bump is committed to hbase-thirdparty master. The jetty
one looks good to include as well. Do you want to handle that one Nihal?

On Fri, Jan 5, 2024 at 2:25 AM Nihal Jain <ni...@gmail.com> wrote:

> Since we are planning a new thirdparty release, IMO we also put
> https://issues.apache.org/jira/browse/HBASE-28279, which bumps to latest
> jetty. Package jetty-http, that we bundle (
>
> https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-http/9.4.52.v20230823
> ),
> has a direct CVE
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-36478
>
> Regards,
> Nihal
>
> On Fri, 5 Jan 2024, 02:43 Andrew Purtell, <ap...@apache.org> wrote:
>
> > Sounds good. I think we agree on everything.
> >
> > On Thu, Jan 4, 2024 at 12:37 PM Bryan Beaudreault <
> bbeaudreault@apache.org
> > >
> > wrote:
> >
> > > To clarify, I was referring to "It’s weird to have netty dependency
> > > versions diverging." Which I do agree with, but was explaining my
> > > understanding of the rationale.
> > >
> > > Otherwise, I agree with everything else. I have the PR ready for
> updating
> > > netty in hbase-thirdparty. I can also vote and do the bump of
> > > hbase-thirdparty.version in hbase once the release is made.
> > >
> > > On Thu, Jan 4, 2024 at 3:15 PM Andrew Purtell <ap...@apache.org>
> > wrote:
> > >
> > > > We should not ship a netty with known CVEs in hbase-thirdparty.
> > Likewise
> > > > for the other components shipped in hbase-thirdparty. As with all
> > things
> > > > this is a guideline, not a rule, because each situation is different
> > and
> > > > people do not always share the same opinion.
> > > >
> > > > I am of the opinion that moderate to high scoring CVEs in a
> dependency
> > --
> > > > and it doesn't matter if direct, transitive, or shaded -- is going to
> > be
> > > a
> > > > problem for many users or potential users simply if they exist in our
> > > bill
> > > > of materials. At my employer we need to juggle the "cleanliness" of
> our
> > > > software bill of materials among priorities and I do not think we are
> > > > exceptional in any way. We do it in a fork but I have been meaning to
> > do
> > > a
> > > > pass over the public open source project's dependency set, if this
> > would
> > > > have some value for the project (which I believe there is).
> > > >
> > > > I can do a thirdparty release now if everyone else is busy.
> > > >
> > > > On Thu, Jan 4, 2024 at 10:50 AM Bryan Beaudreault <
> > > bbeaudreault@apache.org
> > > > >
> > > > wrote:
> > > >
> > > > > I think we explicitly don't want our hbase-thirdparty netty version
> > to
> > > be
> > > > > locked to the one for transitive dependencies. That's sort of why
> we
> > > have
> > > > > it in thirdparty/shaded at all, right?
> > > > >
> > > > > I created https://issues.apache.org/jira/browse/HBASE-28291 to
> > update
> > > > > pom.xml in hbase-thirdparty. I will handle that. We will need to do
> > an
> > > > > hbase-thirdparty release, which I'm not sure I'll have time for
> given
> > > I'm
> > > > > already behind on the 2.6.0 release
> > > > >
> > > > > On Thu, Jan 4, 2024 at 11:58 AM Andrew Purtell <
> > > andrew.purtell@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > We should do that bump to hbase-thirdparty and spin another
> release
> > > to
> > > > > > keep our house in order. It isn’t urgent but would be good to
> > address
> > > > > this
> > > > > > in the normal release cadence. That has been about once per
> fiscal
> > > > > quarter
> > > > > > recently.  It’s weird to have netty dependency versions
> diverging.
> > > > > >
> > > > > > I will make a note as RM to look at hbase-thirdparty versions
> with
> > > > > respect
> > > > > > to the base POM and known security issues, using snyk probably,
> and
> > > > > update
> > > > > > it ahead of 2.5.8. As well as direct dependencies in the base
> POM.
> > > > > > Unfortunately I can’t promise to do anything about transitive
> > issues
> > > > > > imported from something that impacts operational compatibility.
> > Those
> > > > > must
> > > > > > be weighed case by case.
> > > > > >
> > > > > > > On Jan 4, 2024, at 8:31 AM, Bryan Beaudreault <
> > > > bbeaudreault@apache.org
> > > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > It looks like that CVE only affects
> io.netty:netty-codec-http2.
> > > > Since
> > > > > > our
> > > > > > > hbase-shaded-netty depends on netty-all, that module is
> included.
> > > > > > However,
> > > > > > > I don't think we use anything from netty-codec-http2. So I
> don't
> > > > think
> > > > > > the
> > > > > > > CVE is a risk for this usage, unless you are building an app
> > using
> > > > the
> > > > > > > org.apache.hbase.thirdparty.io.netty classes. This would not be
> > > > > advised.
> > > > > > >
> > > > > > > That said, we could try to bump hbase-thirdparty to 4.1.100+
> and
> > > > > include
> > > > > > > that in the upcoming 2.6.0 or 2.5.8 when that happens. If the
> CVE
> > > > were
> > > > > > > critical we could rush out another minor release, but I don't
> > think
> > > > > it's
> > > > > > > necessary here? I also wonder if we should update
> > > hbase-shaded-netty
> > > > to
> > > > > > > only pull in the netty modules we actually use.
> > > > > > >
> > > > > > >> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff
> > > > <dan.huff@dremio.com.invalid
> > > > > >
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> Thanks Bryan. That does help explain things. I have been
> looking
> > > at
> > > > > > >>
> > > > >
> > https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p
> > > > > > and
> > > > > > >> have been trying to determine if hbase is vulnerable to this
> > > attack
> > > > > > vector
> > > > > > >> or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it
> > > > sounds
> > > > > > like
> > > > > > >> that excitement was misplaced :)
> > > > > > >>
> > > > > > >> Dan
> > > > > > >>
> > > > > > >> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <
> > > > > > bbeaudreault@apache.org
> > > > > > >>>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >>> Hello,
> > > > > > >>>
> > > > > > >>> As the comment above the netty version change says, this only
> > > > affects
> > > > > > the
> > > > > > >>> transitive netty dependencies from thirdparty dependencies
> like
> > > > > > zookeeper
> > > > > > >>> and hadoop. HBase's internal netty usage (i.e. for HBase's
> RPC
> > > > > > protocol)
> > > > > > >>> uses the shaded netty provided by hbase-thirdparty.
> > > > > > >>>
> > > > > > >>> While you're generally correct that in maven you'd expect a
> > > version
> > > > > > >> defined
> > > > > > >>> in dependencyManagement to affect all transitive
> dependencies,
> > > that
> > > > > is
> > > > > > >> not
> > > > > > >>> the case for hbase-thirdparty due to the shading we do there.
> > At
> > > > the
> > > > > > time
> > > > > > >>> of building hbase-thirdparty, the defined netty version there
> > is
> > > > > pulled
> > > > > > >> in
> > > > > > >>> and relocated to org.apache.hbase.thirdparty.io.netty and
> > > published
> > > > > as
> > > > > > a
> > > > > > >>> new maven module named hbase-shaded-netty. As such, the
> > > > > > >>> dependencyManagement has no effect on it.
> > > > > > >>>
> > > > > > >>> I hope this helps
> > > > > > >>>
> > > > > > >>> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff
> > > > <dan.huff@dremio.com.invalid
> > > > > >
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > > >>>> Hello there Hbase Devs--
> > > > > > >>>>
> > > > > > >>>> I have been investigating taking an update to Hbase 2.5.7
> > after
> > > > the
> > > > > > >>> release
> > > > > > >>>> last week and have what I hope is a quick question about
> > commit
> > > > > > 7639345
> > > > > > >>>> <
> > > > > > >>>>
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> > > > > > >>>>>
> > > > > > >>>> in
> > > > > > >>>> branch-2.5.
> > > > > > >>>>
> > > > > > >>>> Am I correct in believing that the direct inclusion of netty
> > > > > > >>> 4.1.100.Final
> > > > > > >>>> in Hbase's pom.xml will override the 4.1.97.Final version
> that
> > > is
> > > > > > >>>> specified in hbase-thirdparty
> > > > > > >>>> <
> > > > > > >>
> > > > >
> > https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
> > > > > > >>>> ?
> > > > > > >>>> I
> > > > > > >>>> see 4.1.100.Final listed on
> > > > > > >>>> https://hbase.apache.org/dependency-management.html which
> to
> > me
> > > > > > >> suggests
> > > > > > >>>> that I am understanding this correctly that issues flagged
> > > against
> > > > > > >>>> 4.1.97.Final can be ignored since Hbase will now just use
> > > > > > >> 4.1.100.Final.
> > > > > > >>>>
> > > > > > >>>> Thanks so much for your time,
> > > > > > >>>>
> > > > > > >>>> Dan Huff
> > > > > > >>>>
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew
> > > >
> > > > Unrest, ignorance distilled, nihilistic imbeciles -
> > > >     It's what we’ve earned
> > > > Welcome, apocalypse, what’s taken you so long?
> > > > Bring us the fitting end that we’ve been counting on
> > > >    - A23, Welcome, Apocalypse
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Unrest, ignorance distilled, nihilistic imbeciles -
> >     It's what we’ve earned
> > Welcome, apocalypse, what’s taken you so long?
> > Bring us the fitting end that we’ve been counting on
> >    - A23, Welcome, Apocalypse
> >
>

Re: Hbase, Hbase-thirdparty, and netty

Posted by Nihal Jain <ni...@gmail.com>.

Since we are planning a new thirdparty release, IMO we also put
https://issues.apache.org/jira/browse/HBASE-28279, which bumps to latest
jetty. Package jetty-http, that we bundle (
https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-http/9.4.52.v20230823),
has a direct CVE
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-36478

Regards,
Nihal

On Fri, 5 Jan 2024, 02:43 Andrew Purtell, <ap...@apache.org> wrote:

> Sounds good. I think we agree on everything.
>
> On Thu, Jan 4, 2024 at 12:37 PM Bryan Beaudreault <bbeaudreault@apache.org
> >
> wrote:
>
> > To clarify, I was referring to "It’s weird to have netty dependency
> > versions diverging." Which I do agree with, but was explaining my
> > understanding of the rationale.
> >
> > Otherwise, I agree with everything else. I have the PR ready for updating
> > netty in hbase-thirdparty. I can also vote and do the bump of
> > hbase-thirdparty.version in hbase once the release is made.
> >
> > On Thu, Jan 4, 2024 at 3:15 PM Andrew Purtell <ap...@apache.org>
> wrote:
> >
> > > We should not ship a netty with known CVEs in hbase-thirdparty.
> Likewise
> > > for the other components shipped in hbase-thirdparty. As with all
> things
> > > this is a guideline, not a rule, because each situation is different
> and
> > > people do not always share the same opinion.
> > >
> > > I am of the opinion that moderate to high scoring CVEs in a dependency
> --
> > > and it doesn't matter if direct, transitive, or shaded -- is going to
> be
> > a
> > > problem for many users or potential users simply if they exist in our
> > bill
> > > of materials. At my employer we need to juggle the "cleanliness" of our
> > > software bill of materials among priorities and I do not think we are
> > > exceptional in any way. We do it in a fork but I have been meaning to
> do
> > a
> > > pass over the public open source project's dependency set, if this
> would
> > > have some value for the project (which I believe there is).
> > >
> > > I can do a thirdparty release now if everyone else is busy.
> > >
> > > On Thu, Jan 4, 2024 at 10:50 AM Bryan Beaudreault <
> > bbeaudreault@apache.org
> > > >
> > > wrote:
> > >
> > > > I think we explicitly don't want our hbase-thirdparty netty version
> to
> > be
> > > > locked to the one for transitive dependencies. That's sort of why we
> > have
> > > > it in thirdparty/shaded at all, right?
> > > >
> > > > I created https://issues.apache.org/jira/browse/HBASE-28291 to
> update
> > > > pom.xml in hbase-thirdparty. I will handle that. We will need to do
> an
> > > > hbase-thirdparty release, which I'm not sure I'll have time for given
> > I'm
> > > > already behind on the 2.6.0 release
> > > >
> > > > On Thu, Jan 4, 2024 at 11:58 AM Andrew Purtell <
> > andrew.purtell@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > We should do that bump to hbase-thirdparty and spin another release
> > to
> > > > > keep our house in order. It isn’t urgent but would be good to
> address
> > > > this
> > > > > in the normal release cadence. That has been about once per fiscal
> > > > quarter
> > > > > recently.  It’s weird to have netty dependency versions diverging.
> > > > >
> > > > > I will make a note as RM to look at hbase-thirdparty versions with
> > > > respect
> > > > > to the base POM and known security issues, using snyk probably, and
> > > > update
> > > > > it ahead of 2.5.8. As well as direct dependencies in the base POM.
> > > > > Unfortunately I can’t promise to do anything about transitive
> issues
> > > > > imported from something that impacts operational compatibility.
> Those
> > > > must
> > > > > be weighed case by case.
> > > > >
> > > > > > On Jan 4, 2024, at 8:31 AM, Bryan Beaudreault <
> > > bbeaudreault@apache.org
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > It looks like that CVE only affects io.netty:netty-codec-http2.
> > > Since
> > > > > our
> > > > > > hbase-shaded-netty depends on netty-all, that module is included.
> > > > > However,
> > > > > > I don't think we use anything from netty-codec-http2. So I don't
> > > think
> > > > > the
> > > > > > CVE is a risk for this usage, unless you are building an app
> using
> > > the
> > > > > > org.apache.hbase.thirdparty.io.netty classes. This would not be
> > > > advised.
> > > > > >
> > > > > > That said, we could try to bump hbase-thirdparty to 4.1.100+ and
> > > > include
> > > > > > that in the upcoming 2.6.0 or 2.5.8 when that happens. If the CVE
> > > were
> > > > > > critical we could rush out another minor release, but I don't
> think
> > > > it's
> > > > > > necessary here? I also wonder if we should update
> > hbase-shaded-netty
> > > to
> > > > > > only pull in the netty modules we actually use.
> > > > > >
> > > > > >> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff
> > > <dan.huff@dremio.com.invalid
> > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> Thanks Bryan. That does help explain things. I have been looking
> > at
> > > > > >>
> > > >
> https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p
> > > > > and
> > > > > >> have been trying to determine if hbase is vulnerable to this
> > attack
> > > > > vector
> > > > > >> or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it
> > > sounds
> > > > > like
> > > > > >> that excitement was misplaced :)
> > > > > >>
> > > > > >> Dan
> > > > > >>
> > > > > >> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <
> > > > > bbeaudreault@apache.org
> > > > > >>>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Hello,
> > > > > >>>
> > > > > >>> As the comment above the netty version change says, this only
> > > affects
> > > > > the
> > > > > >>> transitive netty dependencies from thirdparty dependencies like
> > > > > zookeeper
> > > > > >>> and hadoop. HBase's internal netty usage (i.e. for HBase's RPC
> > > > > protocol)
> > > > > >>> uses the shaded netty provided by hbase-thirdparty.
> > > > > >>>
> > > > > >>> While you're generally correct that in maven you'd expect a
> > version
> > > > > >> defined
> > > > > >>> in dependencyManagement to affect all transitive dependencies,
> > that
> > > > is
> > > > > >> not
> > > > > >>> the case for hbase-thirdparty due to the shading we do there.
> At
> > > the
> > > > > time
> > > > > >>> of building hbase-thirdparty, the defined netty version there
> is
> > > > pulled
> > > > > >> in
> > > > > >>> and relocated to org.apache.hbase.thirdparty.io.netty and
> > published
> > > > as
> > > > > a
> > > > > >>> new maven module named hbase-shaded-netty. As such, the
> > > > > >>> dependencyManagement has no effect on it.
> > > > > >>>
> > > > > >>> I hope this helps
> > > > > >>>
> > > > > >>> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff
> > > <dan.huff@dremio.com.invalid
> > > > >
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>> Hello there Hbase Devs--
> > > > > >>>>
> > > > > >>>> I have been investigating taking an update to Hbase 2.5.7
> after
> > > the
> > > > > >>> release
> > > > > >>>> last week and have what I hope is a quick question about
> commit
> > > > > 7639345
> > > > > >>>> <
> > > > > >>>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> > > > > >>>>>
> > > > > >>>> in
> > > > > >>>> branch-2.5.
> > > > > >>>>
> > > > > >>>> Am I correct in believing that the direct inclusion of netty
> > > > > >>> 4.1.100.Final
> > > > > >>>> in Hbase's pom.xml will override the 4.1.97.Final version that
> > is
> > > > > >>>> specified in hbase-thirdparty
> > > > > >>>> <
> > > > > >>
> > > >
> https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
> > > > > >>>> ?
> > > > > >>>> I
> > > > > >>>> see 4.1.100.Final listed on
> > > > > >>>> https://hbase.apache.org/dependency-management.html which to
> me
> > > > > >> suggests
> > > > > >>>> that I am understanding this correctly that issues flagged
> > against
> > > > > >>>> 4.1.97.Final can be ignored since Hbase will now just use
> > > > > >> 4.1.100.Final.
> > > > > >>>>
> > > > > >>>> Thanks so much for your time,
> > > > > >>>>
> > > > > >>>> Dan Huff
> > > > > >>>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Unrest, ignorance distilled, nihilistic imbeciles -
> > >     It's what we’ve earned
> > > Welcome, apocalypse, what’s taken you so long?
> > > Bring us the fitting end that we’ve been counting on
> > >    - A23, Welcome, Apocalypse
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Unrest, ignorance distilled, nihilistic imbeciles -
>     It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
>    - A23, Welcome, Apocalypse
>

Re: Hbase, Hbase-thirdparty, and netty

Posted by Andrew Purtell <ap...@apache.org>.

Sounds good. I think we agree on everything.

On Thu, Jan 4, 2024 at 12:37 PM Bryan Beaudreault <bb...@apache.org>
wrote:

> To clarify, I was referring to "It’s weird to have netty dependency
> versions diverging." Which I do agree with, but was explaining my
> understanding of the rationale.
>
> Otherwise, I agree with everything else. I have the PR ready for updating
> netty in hbase-thirdparty. I can also vote and do the bump of
> hbase-thirdparty.version in hbase once the release is made.
>
> On Thu, Jan 4, 2024 at 3:15 PM Andrew Purtell <ap...@apache.org> wrote:
>
> > We should not ship a netty with known CVEs in hbase-thirdparty. Likewise
> > for the other components shipped in hbase-thirdparty. As with all things
> > this is a guideline, not a rule, because each situation is different and
> > people do not always share the same opinion.
> >
> > I am of the opinion that moderate to high scoring CVEs in a dependency --
> > and it doesn't matter if direct, transitive, or shaded -- is going to be
> a
> > problem for many users or potential users simply if they exist in our
> bill
> > of materials. At my employer we need to juggle the "cleanliness" of our
> > software bill of materials among priorities and I do not think we are
> > exceptional in any way. We do it in a fork but I have been meaning to do
> a
> > pass over the public open source project's dependency set, if this would
> > have some value for the project (which I believe there is).
> >
> > I can do a thirdparty release now if everyone else is busy.
> >
> > On Thu, Jan 4, 2024 at 10:50 AM Bryan Beaudreault <
> bbeaudreault@apache.org
> > >
> > wrote:
> >
> > > I think we explicitly don't want our hbase-thirdparty netty version to
> be
> > > locked to the one for transitive dependencies. That's sort of why we
> have
> > > it in thirdparty/shaded at all, right?
> > >
> > > I created https://issues.apache.org/jira/browse/HBASE-28291 to update
> > > pom.xml in hbase-thirdparty. I will handle that. We will need to do an
> > > hbase-thirdparty release, which I'm not sure I'll have time for given
> I'm
> > > already behind on the 2.6.0 release
> > >
> > > On Thu, Jan 4, 2024 at 11:58 AM Andrew Purtell <
> andrew.purtell@gmail.com
> > >
> > > wrote:
> > >
> > > > We should do that bump to hbase-thirdparty and spin another release
> to
> > > > keep our house in order. It isn’t urgent but would be good to address
> > > this
> > > > in the normal release cadence. That has been about once per fiscal
> > > quarter
> > > > recently.  It’s weird to have netty dependency versions diverging.
> > > >
> > > > I will make a note as RM to look at hbase-thirdparty versions with
> > > respect
> > > > to the base POM and known security issues, using snyk probably, and
> > > update
> > > > it ahead of 2.5.8. As well as direct dependencies in the base POM.
> > > > Unfortunately I can’t promise to do anything about transitive issues
> > > > imported from something that impacts operational compatibility. Those
> > > must
> > > > be weighed case by case.
> > > >
> > > > > On Jan 4, 2024, at 8:31 AM, Bryan Beaudreault <
> > bbeaudreault@apache.org
> > > >
> > > > wrote:
> > > > >
> > > > > It looks like that CVE only affects io.netty:netty-codec-http2.
> > Since
> > > > our
> > > > > hbase-shaded-netty depends on netty-all, that module is included.
> > > > However,
> > > > > I don't think we use anything from netty-codec-http2. So I don't
> > think
> > > > the
> > > > > CVE is a risk for this usage, unless you are building an app using
> > the
> > > > > org.apache.hbase.thirdparty.io.netty classes. This would not be
> > > advised.
> > > > >
> > > > > That said, we could try to bump hbase-thirdparty to 4.1.100+ and
> > > include
> > > > > that in the upcoming 2.6.0 or 2.5.8 when that happens. If the CVE
> > were
> > > > > critical we could rush out another minor release, but I don't think
> > > it's
> > > > > necessary here? I also wonder if we should update
> hbase-shaded-netty
> > to
> > > > > only pull in the netty modules we actually use.
> > > > >
> > > > >> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff
> > <dan.huff@dremio.com.invalid
> > > >
> > > > >> wrote:
> > > > >>
> > > > >> Thanks Bryan. That does help explain things. I have been looking
> at
> > > > >>
> > > https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p
> > > > and
> > > > >> have been trying to determine if hbase is vulnerable to this
> attack
> > > > vector
> > > > >> or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it
> > sounds
> > > > like
> > > > >> that excitement was misplaced :)
> > > > >>
> > > > >> Dan
> > > > >>
> > > > >> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <
> > > > bbeaudreault@apache.org
> > > > >>>
> > > > >> wrote:
> > > > >>
> > > > >>> Hello,
> > > > >>>
> > > > >>> As the comment above the netty version change says, this only
> > affects
> > > > the
> > > > >>> transitive netty dependencies from thirdparty dependencies like
> > > > zookeeper
> > > > >>> and hadoop. HBase's internal netty usage (i.e. for HBase's RPC
> > > > protocol)
> > > > >>> uses the shaded netty provided by hbase-thirdparty.
> > > > >>>
> > > > >>> While you're generally correct that in maven you'd expect a
> version
> > > > >> defined
> > > > >>> in dependencyManagement to affect all transitive dependencies,
> that
> > > is
> > > > >> not
> > > > >>> the case for hbase-thirdparty due to the shading we do there. At
> > the
> > > > time
> > > > >>> of building hbase-thirdparty, the defined netty version there is
> > > pulled
> > > > >> in
> > > > >>> and relocated to org.apache.hbase.thirdparty.io.netty and
> published
> > > as
> > > > a
> > > > >>> new maven module named hbase-shaded-netty. As such, the
> > > > >>> dependencyManagement has no effect on it.
> > > > >>>
> > > > >>> I hope this helps
> > > > >>>
> > > > >>> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff
> > <dan.huff@dremio.com.invalid
> > > >
> > > > >>> wrote:
> > > > >>>
> > > > >>>> Hello there Hbase Devs--
> > > > >>>>
> > > > >>>> I have been investigating taking an update to Hbase 2.5.7 after
> > the
> > > > >>> release
> > > > >>>> last week and have what I hope is a quick question about commit
> > > > 7639345
> > > > >>>> <
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> > > > >>>>>
> > > > >>>> in
> > > > >>>> branch-2.5.
> > > > >>>>
> > > > >>>> Am I correct in believing that the direct inclusion of netty
> > > > >>> 4.1.100.Final
> > > > >>>> in Hbase's pom.xml will override the 4.1.97.Final version that
> is
> > > > >>>> specified in hbase-thirdparty
> > > > >>>> <
> > > > >>
> > > https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
> > > > >>>> ?
> > > > >>>> I
> > > > >>>> see 4.1.100.Final listed on
> > > > >>>> https://hbase.apache.org/dependency-management.html which to me
> > > > >> suggests
> > > > >>>> that I am understanding this correctly that issues flagged
> against
> > > > >>>> 4.1.97.Final can be ignored since Hbase will now just use
> > > > >> 4.1.100.Final.
> > > > >>>>
> > > > >>>> Thanks so much for your time,
> > > > >>>>
> > > > >>>> Dan Huff
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Unrest, ignorance distilled, nihilistic imbeciles -
> >     It's what we’ve earned
> > Welcome, apocalypse, what’s taken you so long?
> > Bring us the fitting end that we’ve been counting on
> >    - A23, Welcome, Apocalypse
> >
>


-- 
Best regards,
Andrew

Unrest, ignorance distilled, nihilistic imbeciles -
    It's what we’ve earned
Welcome, apocalypse, what’s taken you so long?
Bring us the fitting end that we’ve been counting on
   - A23, Welcome, Apocalypse

Re: Hbase, Hbase-thirdparty, and netty

Posted by Bryan Beaudreault <bb...@apache.org>.

To clarify, I was referring to "It’s weird to have netty dependency
versions diverging." Which I do agree with, but was explaining my
understanding of the rationale.

Otherwise, I agree with everything else. I have the PR ready for updating
netty in hbase-thirdparty. I can also vote and do the bump of
hbase-thirdparty.version in hbase once the release is made.

On Thu, Jan 4, 2024 at 3:15 PM Andrew Purtell <ap...@apache.org> wrote:

> We should not ship a netty with known CVEs in hbase-thirdparty. Likewise
> for the other components shipped in hbase-thirdparty. As with all things
> this is a guideline, not a rule, because each situation is different and
> people do not always share the same opinion.
>
> I am of the opinion that moderate to high scoring CVEs in a dependency --
> and it doesn't matter if direct, transitive, or shaded -- is going to be a
> problem for many users or potential users simply if they exist in our bill
> of materials. At my employer we need to juggle the "cleanliness" of our
> software bill of materials among priorities and I do not think we are
> exceptional in any way. We do it in a fork but I have been meaning to do a
> pass over the public open source project's dependency set, if this would
> have some value for the project (which I believe there is).
>
> I can do a thirdparty release now if everyone else is busy.
>
> On Thu, Jan 4, 2024 at 10:50 AM Bryan Beaudreault <bbeaudreault@apache.org
> >
> wrote:
>
> > I think we explicitly don't want our hbase-thirdparty netty version to be
> > locked to the one for transitive dependencies. That's sort of why we have
> > it in thirdparty/shaded at all, right?
> >
> > I created https://issues.apache.org/jira/browse/HBASE-28291 to update
> > pom.xml in hbase-thirdparty. I will handle that. We will need to do an
> > hbase-thirdparty release, which I'm not sure I'll have time for given I'm
> > already behind on the 2.6.0 release
> >
> > On Thu, Jan 4, 2024 at 11:58 AM Andrew Purtell <andrew.purtell@gmail.com
> >
> > wrote:
> >
> > > We should do that bump to hbase-thirdparty and spin another release to
> > > keep our house in order. It isn’t urgent but would be good to address
> > this
> > > in the normal release cadence. That has been about once per fiscal
> > quarter
> > > recently.  It’s weird to have netty dependency versions diverging.
> > >
> > > I will make a note as RM to look at hbase-thirdparty versions with
> > respect
> > > to the base POM and known security issues, using snyk probably, and
> > update
> > > it ahead of 2.5.8. As well as direct dependencies in the base POM.
> > > Unfortunately I can’t promise to do anything about transitive issues
> > > imported from something that impacts operational compatibility. Those
> > must
> > > be weighed case by case.
> > >
> > > > On Jan 4, 2024, at 8:31 AM, Bryan Beaudreault <
> bbeaudreault@apache.org
> > >
> > > wrote:
> > > >
> > > > It looks like that CVE only affects io.netty:netty-codec-http2.
> Since
> > > our
> > > > hbase-shaded-netty depends on netty-all, that module is included.
> > > However,
> > > > I don't think we use anything from netty-codec-http2. So I don't
> think
> > > the
> > > > CVE is a risk for this usage, unless you are building an app using
> the
> > > > org.apache.hbase.thirdparty.io.netty classes. This would not be
> > advised.
> > > >
> > > > That said, we could try to bump hbase-thirdparty to 4.1.100+ and
> > include
> > > > that in the upcoming 2.6.0 or 2.5.8 when that happens. If the CVE
> were
> > > > critical we could rush out another minor release, but I don't think
> > it's
> > > > necessary here? I also wonder if we should update hbase-shaded-netty
> to
> > > > only pull in the netty modules we actually use.
> > > >
> > > >> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff
> <dan.huff@dremio.com.invalid
> > >
> > > >> wrote:
> > > >>
> > > >> Thanks Bryan. That does help explain things. I have been looking at
> > > >>
> > https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p
> > > and
> > > >> have been trying to determine if hbase is vulnerable to this attack
> > > vector
> > > >> or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it
> sounds
> > > like
> > > >> that excitement was misplaced :)
> > > >>
> > > >> Dan
> > > >>
> > > >> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <
> > > bbeaudreault@apache.org
> > > >>>
> > > >> wrote:
> > > >>
> > > >>> Hello,
> > > >>>
> > > >>> As the comment above the netty version change says, this only
> affects
> > > the
> > > >>> transitive netty dependencies from thirdparty dependencies like
> > > zookeeper
> > > >>> and hadoop. HBase's internal netty usage (i.e. for HBase's RPC
> > > protocol)
> > > >>> uses the shaded netty provided by hbase-thirdparty.
> > > >>>
> > > >>> While you're generally correct that in maven you'd expect a version
> > > >> defined
> > > >>> in dependencyManagement to affect all transitive dependencies, that
> > is
> > > >> not
> > > >>> the case for hbase-thirdparty due to the shading we do there. At
> the
> > > time
> > > >>> of building hbase-thirdparty, the defined netty version there is
> > pulled
> > > >> in
> > > >>> and relocated to org.apache.hbase.thirdparty.io.netty and published
> > as
> > > a
> > > >>> new maven module named hbase-shaded-netty. As such, the
> > > >>> dependencyManagement has no effect on it.
> > > >>>
> > > >>> I hope this helps
> > > >>>
> > > >>> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff
> <dan.huff@dremio.com.invalid
> > >
> > > >>> wrote:
> > > >>>
> > > >>>> Hello there Hbase Devs--
> > > >>>>
> > > >>>> I have been investigating taking an update to Hbase 2.5.7 after
> the
> > > >>> release
> > > >>>> last week and have what I hope is a quick question about commit
> > > 7639345
> > > >>>> <
> > > >>>>
> > > >>>
> > > >>
> > >
> >
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> > > >>>>>
> > > >>>> in
> > > >>>> branch-2.5.
> > > >>>>
> > > >>>> Am I correct in believing that the direct inclusion of netty
> > > >>> 4.1.100.Final
> > > >>>> in Hbase's pom.xml will override the 4.1.97.Final version that is
> > > >>>> specified in hbase-thirdparty
> > > >>>> <
> > > >>
> > https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
> > > >>>> ?
> > > >>>> I
> > > >>>> see 4.1.100.Final listed on
> > > >>>> https://hbase.apache.org/dependency-management.html which to me
> > > >> suggests
> > > >>>> that I am understanding this correctly that issues flagged against
> > > >>>> 4.1.97.Final can be ignored since Hbase will now just use
> > > >> 4.1.100.Final.
> > > >>>>
> > > >>>> Thanks so much for your time,
> > > >>>>
> > > >>>> Dan Huff
> > > >>>>
> > > >>>
> > > >>
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Unrest, ignorance distilled, nihilistic imbeciles -
>     It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
>    - A23, Welcome, Apocalypse
>

Re: Hbase, Hbase-thirdparty, and netty

Posted by Andrew Purtell <ap...@apache.org>.

We should not ship a netty with known CVEs in hbase-thirdparty. Likewise
for the other components shipped in hbase-thirdparty. As with all things
this is a guideline, not a rule, because each situation is different and
people do not always share the same opinion.

I am of the opinion that moderate to high scoring CVEs in a dependency --
and it doesn't matter if direct, transitive, or shaded -- is going to be a
problem for many users or potential users simply if they exist in our bill
of materials. At my employer we need to juggle the "cleanliness" of our
software bill of materials among priorities and I do not think we are
exceptional in any way. We do it in a fork but I have been meaning to do a
pass over the public open source project's dependency set, if this would
have some value for the project (which I believe there is).

I can do a thirdparty release now if everyone else is busy.

On Thu, Jan 4, 2024 at 10:50 AM Bryan Beaudreault <bb...@apache.org>
wrote:

> I think we explicitly don't want our hbase-thirdparty netty version to be
> locked to the one for transitive dependencies. That's sort of why we have
> it in thirdparty/shaded at all, right?
>
> I created https://issues.apache.org/jira/browse/HBASE-28291 to update
> pom.xml in hbase-thirdparty. I will handle that. We will need to do an
> hbase-thirdparty release, which I'm not sure I'll have time for given I'm
> already behind on the 2.6.0 release
>
> On Thu, Jan 4, 2024 at 11:58 AM Andrew Purtell <an...@gmail.com>
> wrote:
>
> > We should do that bump to hbase-thirdparty and spin another release to
> > keep our house in order. It isn’t urgent but would be good to address
> this
> > in the normal release cadence. That has been about once per fiscal
> quarter
> > recently.  It’s weird to have netty dependency versions diverging.
> >
> > I will make a note as RM to look at hbase-thirdparty versions with
> respect
> > to the base POM and known security issues, using snyk probably, and
> update
> > it ahead of 2.5.8. As well as direct dependencies in the base POM.
> > Unfortunately I can’t promise to do anything about transitive issues
> > imported from something that impacts operational compatibility. Those
> must
> > be weighed case by case.
> >
> > > On Jan 4, 2024, at 8:31 AM, Bryan Beaudreault <bbeaudreault@apache.org
> >
> > wrote:
> > >
> > > It looks like that CVE only affects io.netty:netty-codec-http2. Since
> > our
> > > hbase-shaded-netty depends on netty-all, that module is included.
> > However,
> > > I don't think we use anything from netty-codec-http2. So I don't think
> > the
> > > CVE is a risk for this usage, unless you are building an app using the
> > > org.apache.hbase.thirdparty.io.netty classes. This would not be
> advised.
> > >
> > > That said, we could try to bump hbase-thirdparty to 4.1.100+ and
> include
> > > that in the upcoming 2.6.0 or 2.5.8 when that happens. If the CVE were
> > > critical we could rush out another minor release, but I don't think
> it's
> > > necessary here? I also wonder if we should update hbase-shaded-netty to
> > > only pull in the netty modules we actually use.
> > >
> > >> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff <dan.huff@dremio.com.invalid
> >
> > >> wrote:
> > >>
> > >> Thanks Bryan. That does help explain things. I have been looking at
> > >>
> https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p
> > and
> > >> have been trying to determine if hbase is vulnerable to this attack
> > vector
> > >> or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it sounds
> > like
> > >> that excitement was misplaced :)
> > >>
> > >> Dan
> > >>
> > >> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <
> > bbeaudreault@apache.org
> > >>>
> > >> wrote:
> > >>
> > >>> Hello,
> > >>>
> > >>> As the comment above the netty version change says, this only affects
> > the
> > >>> transitive netty dependencies from thirdparty dependencies like
> > zookeeper
> > >>> and hadoop. HBase's internal netty usage (i.e. for HBase's RPC
> > protocol)
> > >>> uses the shaded netty provided by hbase-thirdparty.
> > >>>
> > >>> While you're generally correct that in maven you'd expect a version
> > >> defined
> > >>> in dependencyManagement to affect all transitive dependencies, that
> is
> > >> not
> > >>> the case for hbase-thirdparty due to the shading we do there. At the
> > time
> > >>> of building hbase-thirdparty, the defined netty version there is
> pulled
> > >> in
> > >>> and relocated to org.apache.hbase.thirdparty.io.netty and published
> as
> > a
> > >>> new maven module named hbase-shaded-netty. As such, the
> > >>> dependencyManagement has no effect on it.
> > >>>
> > >>> I hope this helps
> > >>>
> > >>> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff <dan.huff@dremio.com.invalid
> >
> > >>> wrote:
> > >>>
> > >>>> Hello there Hbase Devs--
> > >>>>
> > >>>> I have been investigating taking an update to Hbase 2.5.7 after the
> > >>> release
> > >>>> last week and have what I hope is a quick question about commit
> > 7639345
> > >>>> <
> > >>>>
> > >>>
> > >>
> >
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> > >>>>>
> > >>>> in
> > >>>> branch-2.5.
> > >>>>
> > >>>> Am I correct in believing that the direct inclusion of netty
> > >>> 4.1.100.Final
> > >>>> in Hbase's pom.xml will override the 4.1.97.Final version that is
> > >>>> specified in hbase-thirdparty
> > >>>> <
> > >>
> https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
> > >>>> ?
> > >>>> I
> > >>>> see 4.1.100.Final listed on
> > >>>> https://hbase.apache.org/dependency-management.html which to me
> > >> suggests
> > >>>> that I am understanding this correctly that issues flagged against
> > >>>> 4.1.97.Final can be ignored since Hbase will now just use
> > >> 4.1.100.Final.
> > >>>>
> > >>>> Thanks so much for your time,
> > >>>>
> > >>>> Dan Huff
> > >>>>
> > >>>
> > >>
> >
>


-- 
Best regards,
Andrew

Unrest, ignorance distilled, nihilistic imbeciles -
    It's what we’ve earned
Welcome, apocalypse, what’s taken you so long?
Bring us the fitting end that we’ve been counting on
   - A23, Welcome, Apocalypse

Re: Hbase, Hbase-thirdparty, and netty

Posted by Bryan Beaudreault <bb...@apache.org>.

I think we explicitly don't want our hbase-thirdparty netty version to be
locked to the one for transitive dependencies. That's sort of why we have
it in thirdparty/shaded at all, right?

I created https://issues.apache.org/jira/browse/HBASE-28291 to update
pom.xml in hbase-thirdparty. I will handle that. We will need to do an
hbase-thirdparty release, which I'm not sure I'll have time for given I'm
already behind on the 2.6.0 release

On Thu, Jan 4, 2024 at 11:58 AM Andrew Purtell <an...@gmail.com>
wrote:

> We should do that bump to hbase-thirdparty and spin another release to
> keep our house in order. It isn’t urgent but would be good to address this
> in the normal release cadence. That has been about once per fiscal quarter
> recently.  It’s weird to have netty dependency versions diverging.
>
> I will make a note as RM to look at hbase-thirdparty versions with respect
> to the base POM and known security issues, using snyk probably, and update
> it ahead of 2.5.8. As well as direct dependencies in the base POM.
> Unfortunately I can’t promise to do anything about transitive issues
> imported from something that impacts operational compatibility. Those must
> be weighed case by case.
>
> > On Jan 4, 2024, at 8:31 AM, Bryan Beaudreault <bb...@apache.org>
> wrote:
> >
> > It looks like that CVE only affects io.netty:netty-codec-http2. Since
> our
> > hbase-shaded-netty depends on netty-all, that module is included.
> However,
> > I don't think we use anything from netty-codec-http2. So I don't think
> the
> > CVE is a risk for this usage, unless you are building an app using the
> > org.apache.hbase.thirdparty.io.netty classes. This would not be advised.
> >
> > That said, we could try to bump hbase-thirdparty to 4.1.100+ and include
> > that in the upcoming 2.6.0 or 2.5.8 when that happens. If the CVE were
> > critical we could rush out another minor release, but I don't think it's
> > necessary here? I also wonder if we should update hbase-shaded-netty to
> > only pull in the netty modules we actually use.
> >
> >> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff <da...@dremio.com.invalid>
> >> wrote:
> >>
> >> Thanks Bryan. That does help explain things. I have been looking at
> >> https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p
> and
> >> have been trying to determine if hbase is vulnerable to this attack
> vector
> >> or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it sounds
> like
> >> that excitement was misplaced :)
> >>
> >> Dan
> >>
> >> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <
> bbeaudreault@apache.org
> >>>
> >> wrote:
> >>
> >>> Hello,
> >>>
> >>> As the comment above the netty version change says, this only affects
> the
> >>> transitive netty dependencies from thirdparty dependencies like
> zookeeper
> >>> and hadoop. HBase's internal netty usage (i.e. for HBase's RPC
> protocol)
> >>> uses the shaded netty provided by hbase-thirdparty.
> >>>
> >>> While you're generally correct that in maven you'd expect a version
> >> defined
> >>> in dependencyManagement to affect all transitive dependencies, that is
> >> not
> >>> the case for hbase-thirdparty due to the shading we do there. At the
> time
> >>> of building hbase-thirdparty, the defined netty version there is pulled
> >> in
> >>> and relocated to org.apache.hbase.thirdparty.io.netty and published as
> a
> >>> new maven module named hbase-shaded-netty. As such, the
> >>> dependencyManagement has no effect on it.
> >>>
> >>> I hope this helps
> >>>
> >>> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff <da...@dremio.com.invalid>
> >>> wrote:
> >>>
> >>>> Hello there Hbase Devs--
> >>>>
> >>>> I have been investigating taking an update to Hbase 2.5.7 after the
> >>> release
> >>>> last week and have what I hope is a quick question about commit
> 7639345
> >>>> <
> >>>>
> >>>
> >>
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> >>>>>
> >>>> in
> >>>> branch-2.5.
> >>>>
> >>>> Am I correct in believing that the direct inclusion of netty
> >>> 4.1.100.Final
> >>>> in Hbase's pom.xml will override the 4.1.97.Final version that is
> >>>> specified in hbase-thirdparty
> >>>> <
> >> https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
> >>>> ?
> >>>> I
> >>>> see 4.1.100.Final listed on
> >>>> https://hbase.apache.org/dependency-management.html which to me
> >> suggests
> >>>> that I am understanding this correctly that issues flagged against
> >>>> 4.1.97.Final can be ignored since Hbase will now just use
> >> 4.1.100.Final.
> >>>>
> >>>> Thanks so much for your time,
> >>>>
> >>>> Dan Huff
> >>>>
> >>>
> >>
>

Re: Hbase, Hbase-thirdparty, and netty

Posted by Andrew Purtell <an...@gmail.com>.

We should do that bump to hbase-thirdparty and spin another release to keep our house in order. It isn’t urgent but would be good to address this in the normal release cadence. That has been about once per fiscal quarter recently.  It’s weird to have netty dependency versions diverging. 

I will make a note as RM to look at hbase-thirdparty versions with respect to the base POM and known security issues, using snyk probably, and update it ahead of 2.5.8. As well as direct dependencies in the base POM. Unfortunately I can’t promise to do anything about transitive issues imported from something that impacts operational compatibility. Those must be weighed case by case. 

> On Jan 4, 2024, at 8:31 AM, Bryan Beaudreault <bb...@apache.org> wrote:
> 
> It looks like that CVE only affects io.netty:netty-codec-http2. Since our
> hbase-shaded-netty depends on netty-all, that module is included. However,
> I don't think we use anything from netty-codec-http2. So I don't think the
> CVE is a risk for this usage, unless you are building an app using the
> org.apache.hbase.thirdparty.io.netty classes. This would not be advised.
> 
> That said, we could try to bump hbase-thirdparty to 4.1.100+ and include
> that in the upcoming 2.6.0 or 2.5.8 when that happens. If the CVE were
> critical we could rush out another minor release, but I don't think it's
> necessary here? I also wonder if we should update hbase-shaded-netty to
> only pull in the netty modules we actually use.
> 
>> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff <da...@dremio.com.invalid>
>> wrote:
>> 
>> Thanks Bryan. That does help explain things. I have been looking at
>> https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p and
>> have been trying to determine if hbase is vulnerable to this attack vector
>> or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it sounds like
>> that excitement was misplaced :)
>> 
>> Dan
>> 
>> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <bbeaudreault@apache.org
>>> 
>> wrote:
>> 
>>> Hello,
>>> 
>>> As the comment above the netty version change says, this only affects the
>>> transitive netty dependencies from thirdparty dependencies like zookeeper
>>> and hadoop. HBase's internal netty usage (i.e. for HBase's RPC protocol)
>>> uses the shaded netty provided by hbase-thirdparty.
>>> 
>>> While you're generally correct that in maven you'd expect a version
>> defined
>>> in dependencyManagement to affect all transitive dependencies, that is
>> not
>>> the case for hbase-thirdparty due to the shading we do there. At the time
>>> of building hbase-thirdparty, the defined netty version there is pulled
>> in
>>> and relocated to org.apache.hbase.thirdparty.io.netty and published as a
>>> new maven module named hbase-shaded-netty. As such, the
>>> dependencyManagement has no effect on it.
>>> 
>>> I hope this helps
>>> 
>>> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff <da...@dremio.com.invalid>
>>> wrote:
>>> 
>>>> Hello there Hbase Devs--
>>>> 
>>>> I have been investigating taking an update to Hbase 2.5.7 after the
>>> release
>>>> last week and have what I hope is a quick question about commit 7639345
>>>> <
>>>> 
>>> 
>> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
>>>>> 
>>>> in
>>>> branch-2.5.
>>>> 
>>>> Am I correct in believing that the direct inclusion of netty
>>> 4.1.100.Final
>>>> in Hbase's pom.xml will override the 4.1.97.Final version that is
>>>> specified in hbase-thirdparty
>>>> <
>> https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
>>>> ?
>>>> I
>>>> see 4.1.100.Final listed on
>>>> https://hbase.apache.org/dependency-management.html which to me
>> suggests
>>>> that I am understanding this correctly that issues flagged against
>>>> 4.1.97.Final can be ignored since Hbase will now just use
>> 4.1.100.Final.
>>>> 
>>>> Thanks so much for your time,
>>>> 
>>>> Dan Huff
>>>> 
>>> 
>>

Re: Hbase, Hbase-thirdparty, and netty

Posted by Dan Huff <da...@dremio.com.INVALID>.

I have been perusing through the hbase code the last couple of days and
have not been able to find anywhere it seems to reference
io.netty.handler.codec.http2 so it seems it may be in the clear. It would
be awesome to limit hbase-shaded-netty to the actually used modules but I'm
sure that's easier said than done.

Thanks so much for the discussion. I think this gets me what I've been
looking for. Please feel free to correct me if I'm wrong about anything.

Dan

On Thu, Jan 4, 2024 at 9:30 AM Bryan Beaudreault <bb...@apache.org>
wrote:

> It looks like that CVE only affects io.netty:netty-codec-http2. Since our
> hbase-shaded-netty depends on netty-all, that module is included. However,
> I don't think we use anything from netty-codec-http2. So I don't think the
> CVE is a risk for this usage, unless you are building an app using the
> org.apache.hbase.thirdparty.io.netty classes. This would not be advised.
>
> That said, we could try to bump hbase-thirdparty to 4.1.100+ and include
> that in the upcoming 2.6.0 or 2.5.8 when that happens. If the CVE were
> critical we could rush out another minor release, but I don't think it's
> necessary here? I also wonder if we should update hbase-shaded-netty to
> only pull in the netty modules we actually use.
>
> On Thu, Jan 4, 2024 at 11:14 AM Dan Huff <da...@dremio.com.invalid>
> wrote:
>
> > Thanks Bryan. That does help explain things. I have been looking at
> > https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p
> and
> > have been trying to determine if hbase is vulnerable to this attack
> vector
> > or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it sounds
> like
> > that excitement was misplaced :)
> >
> > Dan
> >
> > On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <
> bbeaudreault@apache.org
> > >
> > wrote:
> >
> > > Hello,
> > >
> > > As the comment above the netty version change says, this only affects
> the
> > > transitive netty dependencies from thirdparty dependencies like
> zookeeper
> > > and hadoop. HBase's internal netty usage (i.e. for HBase's RPC
> protocol)
> > > uses the shaded netty provided by hbase-thirdparty.
> > >
> > > While you're generally correct that in maven you'd expect a version
> > defined
> > > in dependencyManagement to affect all transitive dependencies, that is
> > not
> > > the case for hbase-thirdparty due to the shading we do there. At the
> time
> > > of building hbase-thirdparty, the defined netty version there is pulled
> > in
> > > and relocated to org.apache.hbase.thirdparty.io.netty and published as
> a
> > > new maven module named hbase-shaded-netty. As such, the
> > > dependencyManagement has no effect on it.
> > >
> > > I hope this helps
> > >
> > > On Tue, Jan 2, 2024 at 2:40 PM Dan Huff <da...@dremio.com.invalid>
> > > wrote:
> > >
> > > > Hello there Hbase Devs--
> > > >
> > > > I have been investigating taking an update to Hbase 2.5.7 after the
> > > release
> > > > last week and have what I hope is a quick question about commit
> 7639345
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> > > > >
> > > > in
> > > > branch-2.5.
> > > >
> > > > Am I correct in believing that the direct inclusion of netty
> > > 4.1.100.Final
> > > > in Hbase's pom.xml will override the 4.1.97.Final version that is
> > > > specified in hbase-thirdparty
> > > > <
> > https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
> > > >?
> > > > I
> > > > see 4.1.100.Final listed on
> > > > https://hbase.apache.org/dependency-management.html which to me
> > suggests
> > > > that I am understanding this correctly that issues flagged against
> > > > 4.1.97.Final can be ignored since Hbase will now just use
> > 4.1.100.Final.
> > > >
> > > > Thanks so much for your time,
> > > >
> > > > Dan Huff
> > > >
> > >
> >
>

Re: Hbase, Hbase-thirdparty, and netty

Posted by Bryan Beaudreault <bb...@apache.org>.

It looks like that CVE only affects io.netty:netty-codec-http2. Since our
hbase-shaded-netty depends on netty-all, that module is included. However,
I don't think we use anything from netty-codec-http2. So I don't think the
CVE is a risk for this usage, unless you are building an app using the
org.apache.hbase.thirdparty.io.netty classes. This would not be advised.

That said, we could try to bump hbase-thirdparty to 4.1.100+ and include
that in the upcoming 2.6.0 or 2.5.8 when that happens. If the CVE were
critical we could rush out another minor release, but I don't think it's
necessary here? I also wonder if we should update hbase-shaded-netty to
only pull in the netty modules we actually use.

On Thu, Jan 4, 2024 at 11:14 AM Dan Huff <da...@dremio.com.invalid>
wrote:

> Thanks Bryan. That does help explain things. I have been looking at
> https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p and
> have been trying to determine if hbase is vulnerable to this attack vector
> or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it sounds like
> that excitement was misplaced :)
>
> Dan
>
> On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <bbeaudreault@apache.org
> >
> wrote:
>
> > Hello,
> >
> > As the comment above the netty version change says, this only affects the
> > transitive netty dependencies from thirdparty dependencies like zookeeper
> > and hadoop. HBase's internal netty usage (i.e. for HBase's RPC protocol)
> > uses the shaded netty provided by hbase-thirdparty.
> >
> > While you're generally correct that in maven you'd expect a version
> defined
> > in dependencyManagement to affect all transitive dependencies, that is
> not
> > the case for hbase-thirdparty due to the shading we do there. At the time
> > of building hbase-thirdparty, the defined netty version there is pulled
> in
> > and relocated to org.apache.hbase.thirdparty.io.netty and published as a
> > new maven module named hbase-shaded-netty. As such, the
> > dependencyManagement has no effect on it.
> >
> > I hope this helps
> >
> > On Tue, Jan 2, 2024 at 2:40 PM Dan Huff <da...@dremio.com.invalid>
> > wrote:
> >
> > > Hello there Hbase Devs--
> > >
> > > I have been investigating taking an update to Hbase 2.5.7 after the
> > release
> > > last week and have what I hope is a quick question about commit 7639345
> > > <
> > >
> >
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> > > >
> > > in
> > > branch-2.5.
> > >
> > > Am I correct in believing that the direct inclusion of netty
> > 4.1.100.Final
> > > in Hbase's pom.xml will override the 4.1.97.Final version that is
> > > specified in hbase-thirdparty
> > > <
> https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
> > >?
> > > I
> > > see 4.1.100.Final listed on
> > > https://hbase.apache.org/dependency-management.html which to me
> suggests
> > > that I am understanding this correctly that issues flagged against
> > > 4.1.97.Final can be ignored since Hbase will now just use
> 4.1.100.Final.
> > >
> > > Thanks so much for your time,
> > >
> > > Dan Huff
> > >
> >
>

Re: Hbase, Hbase-thirdparty, and netty

Posted by Dan Huff <da...@dremio.com.INVALID>.

Thanks Bryan. That does help explain things. I have been looking at
https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p and
have been trying to determine if hbase is vulnerable to this attack vector
or not. I got excited when I saw 4.1.100.Final in 2.5.7 but it sounds like
that excitement was misplaced :)

Dan

On Tue, Jan 2, 2024 at 12:54 PM Bryan Beaudreault <bb...@apache.org>
wrote:

> Hello,
>
> As the comment above the netty version change says, this only affects the
> transitive netty dependencies from thirdparty dependencies like zookeeper
> and hadoop. HBase's internal netty usage (i.e. for HBase's RPC protocol)
> uses the shaded netty provided by hbase-thirdparty.
>
> While you're generally correct that in maven you'd expect a version defined
> in dependencyManagement to affect all transitive dependencies, that is not
> the case for hbase-thirdparty due to the shading we do there. At the time
> of building hbase-thirdparty, the defined netty version there is pulled in
> and relocated to org.apache.hbase.thirdparty.io.netty and published as a
> new maven module named hbase-shaded-netty. As such, the
> dependencyManagement has no effect on it.
>
> I hope this helps
>
> On Tue, Jan 2, 2024 at 2:40 PM Dan Huff <da...@dremio.com.invalid>
> wrote:
>
> > Hello there Hbase Devs--
> >
> > I have been investigating taking an update to Hbase 2.5.7 after the
> release
> > last week and have what I hope is a quick question about commit 7639345
> > <
> >
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> > >
> > in
> > branch-2.5.
> >
> > Am I correct in believing that the direct inclusion of netty
> 4.1.100.Final
> > in Hbase's pom.xml will override the 4.1.97.Final version that is
> > specified in hbase-thirdparty
> > <https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137
> >?
> > I
> > see 4.1.100.Final listed on
> > https://hbase.apache.org/dependency-management.html which to me suggests
> > that I am understanding this correctly that issues flagged against
> > 4.1.97.Final can be ignored since Hbase will now just use 4.1.100.Final.
> >
> > Thanks so much for your time,
> >
> > Dan Huff
> >
>

Re: Hbase, Hbase-thirdparty, and netty

Posted by Bryan Beaudreault <bb...@apache.org>.

Hello,

As the comment above the netty version change says, this only affects the
transitive netty dependencies from thirdparty dependencies like zookeeper
and hadoop. HBase's internal netty usage (i.e. for HBase's RPC protocol)
uses the shaded netty provided by hbase-thirdparty.

While you're generally correct that in maven you'd expect a version defined
in dependencyManagement to affect all transitive dependencies, that is not
the case for hbase-thirdparty due to the shading we do there. At the time
of building hbase-thirdparty, the defined netty version there is pulled in
and relocated to org.apache.hbase.thirdparty.io.netty and published as a
new maven module named hbase-shaded-netty. As such, the
dependencyManagement has no effect on it.

I hope this helps

On Tue, Jan 2, 2024 at 2:40 PM Dan Huff <da...@dremio.com.invalid> wrote:

> Hello there Hbase Devs--
>
> I have been investigating taking an update to Hbase 2.5.7 after the release
> last week and have what I hope is a quick question about commit 7639345
> <
> https://github.com/apache/hbase/commit/7639345a970636e7a9eb7adf6d84dadd6f3fccb9
> >
> in
> branch-2.5.
>
> Am I correct in believing that the direct inclusion of netty 4.1.100.Final
> in Hbase's pom.xml will override the 4.1.97.Final version that is
> specified in hbase-thirdparty
> <https://github.com/apache/hbase-thirdparty/blob/rel/4.1.5/pom.xml#L137>?
> I
> see 4.1.100.Final listed on
> https://hbase.apache.org/dependency-management.html which to me suggests
> that I am understanding this correctly that issues flagged against
> 4.1.97.Final can be ignored since Hbase will now just use 4.1.100.Final.
>
> Thanks so much for your time,
>
> Dan Huff
>