You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pekko.apache.org by kerr <he...@gmail.com> on 2023/09/08 09:10:35 UTC
Re: Remove the old classical transport or just upgrade it to netty v4?
I did a poll on reddit and get the result of
[image: image.png]
何品
kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
> > Is it really worth spending any time to support classic remoting? Users
> would be better served to switch to Artery.
>
> I'm one of the user and the China Scala User group may not talk much on
> web but most of us is using the netty transport, even Lightbend say the
> `Artery is the best`.
>
> Another online product with large scale nodes(here) is running with Akka
> 2.4.x with the classic transport, and Flink is using the netty.tpc
> transport at least for now too.
>
> I think the community can choose when they migrate the nodes to artery is
> essential, especially when otherwise your cluster need a hard reboot.
>
> AFAIK, the Artery transport is not wire compatible with the classic one.
>
> I suggest:
> 1. Start to send PRs to open sources projects like Flink with the help for
> migrating to the artery transport.
> 2. Upgrade to the new version of Netty for bugfix and better performance.
> 3. Suggest all the new community move to the Artery transport.
>
> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
>
> Anyway, different company has different requirements, that just my 2 cents.
>
> 何品
>
>
> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
>
>> This will end up in a vote.
>>
>> For me, we shipped 1.0.x. Users who don't want big changes can use
>> that version. We can agree to support 1.0.x with critical fixes.
>>
>> We can now start improving the code for the improvement release
>> (v1.1.x or v2.0.x). Improvements include getting rid of deprecated
>> code and most of us appear to believe that classic remoting is long
>> deprecated.
>>
>>
>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
>> <ma...@aiven.io.invalid> wrote:
>> >
>> > > Is it really worth spending any time to support classic remoting?
>> Users
>> > would be better served to switch to Artery.
>> >
>> > That depends on what you mean by "support". If agreed on, I would
>> > propose marking classic remoting as "to be removed" which would
>> > also mean that the code wouldn't be touched (i.e. no feature
>> > changes and no bug fixes unless extremely critical).
>> >
>> > I am in no way suggesting that we support remoting in a traditional
>> sense
>> > and again the only reason why I am even contemplating this is because
>> > of the CVE's.
>> >
>> > > I don't think it is likely that a significant portion of Akka 2.4
>> users
>> > would update to Pekko any time soon if the couldn't update to Akka
>> 2.5/2.6.
>> >
>> > I also had this impression but it's already come up a few times that
>> > people still use Akka 2.4.x. They likely haven't bothered updating
>> > because they didn't see a need to (quite famously Akka is quite stable
>> > and there are cases of companies using it for years without needing to
>> > update, as stated by Lightbend's CEO) but with the license change
>> > it created a catalyst/trigger for some users to consider moving to
>> Pekko.
>> >
>> > On this note I would also be wary in making assumptions on what versions
>> > of Akka people happen to be using. As a corollary in the discussion on
>> > dropping JDK 8, when we suggested that we quite quickly got feedback
>> that
>> > there are people still using JDK 8. And while I know all of the
>> arguments
>> > that you really should not be using JDK 8 can be carried over to Akka.
>> >
>> > Admittedly it's quite hard to get good feedback on what versions people
>> are
>> > using, just saying we should be a bit more careful in making these
>> > assumptions.
>> > After all, people are still using JDK 8 ;)
>> >
>> >
>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <nv...@gmail.com>
>> wrote:
>> >
>> > > Akka 2.4 has been EoL since end of 2017.
>> > > I don't think it is likely that a significant portion of Akka 2.4
>> users
>> > > would update to Pekko any time soon if the couldn't update to Akka
>> 2.5/2.6.
>> > >
>> > > Is it really worth spending any time to support classic remoting?
>> Users
>> > > would be better served to switch to Artery.
>> > >
>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
>> > > <ma...@aiven.io.invalid> wrote:
>> > >
>> > > > > If you manage to move to Pekko and were still using classic
>> remoting,
>> > > > it's likely you are still on Akka 2.4 or 2.5. If you manage the
>> update
>> > > > to Pekko, going to Artery TCP is a small step.
>> > > >
>> > > > One of the reasons why I was saying this is that I was under the
>> > > impression
>> > > > that there are a non-trivial amount of users still on Akka 2.4 (not
>> sure
>> > > > why
>> > > > this is the case but kerr was telling me this).
>> > > >
>> > > > > If people are interested in keeping it around and also opt in to
>> > > > maintaining it (which in the first place means, making the tests for
>> > > > it work *now*), than fine, but otherwise, being able to use it on
>> > > > 1.0.x is already a big benefit.
>> > > >
>> > > > Since classic remoting is currently using Netty 3 which has CVE's
>> > > > I don't think it's wise to encourage people that still happen to be
>> > > > using classic remoting to stay on 1.0.x. If it wasn't for final
>> > > > version of netty 3 having CVE's then I would have no qualms
>> > > > for dropping it in the 1.1.x series (in fact if there was a
>> > > > hypothetical netty 3 without CVE's we would have already
>> > > > updated to it in the 1.0.x series and dropped classic remoting in
>> > > > 1.1.x without a heartbeat).
>> > > >
>> > > > I would propose that if we do accept keeping classic remoting
>> > > > and updating to Netty 4, we would mark the feature to be
>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade that
>> > > > part of code would be untouched unless there are critical
>> > > > bugs/regressions.
>> > > >
>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
>> > > > johannes.rudolph@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > We definitely should remove it for 1.1.x. There's no technical
>> reason
>> > > > > to keep it because the newer artery TCP transport just supersedes
>> it.
>> > > > > As discussed before, the problem with keeping old features around
>> is
>> > > > > "death by a thousand cuts". The classic remoting backend is the
>> prime
>> > > > > example of it because it is basically unmaintained for years and
>> > > > > there's basically no expert knowledge available about how to
>> maintain
>> > > > > it or how to fix the existing bugs. Testing remoting backends has
>> > > > > shown to be very complicated and often unreliable and being able
>> to
>> > > > > remove this particular component from the test matrix will be a
>> huge
>> > > > > improvement.
>> > > > >
>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
>> > > > > <ma...@aiven.io.invalid> wrote:
>> > > > > > I would prefer adding in netty 4 support for the classic
>> transport
>> > > > > > mechanism because it gives
>> > > > > > people an upgrade path while allowing them to resolve the CVE
>> issues.
>> > > > If
>> > > > > > Pekko 1.1.x didn't
>> > > > > > support the classic transport mechanism then it forces Pekko
>> users to
>> > > > be
>> > > > > > stuck on 1.0.x
>> > > > > > with no easy upgrade path.
>> > > > >
>> > > > > We are not responsible for giving people an easy upgrade path when
>> > > > > they were stuck on features like classic remoting which has
>> basically
>> > > > > been outdated for years. It is very important that, going forward,
>> > > > > that we focus our attention on the most important features and
>> make
>> > > > > sure to not get stuck in maintaining old parts that we cannot (or
>> > > > > don't want to) care for.
>> > > > >
>> > > > > If people are interested in keeping it around and also opt in to
>> > > > > maintaining it (which in the first place means, making the tests
>> for
>> > > > > it work *now*), than fine, but otherwise, being able to use it on
>> > > > > 1.0.x is already a big benefit.
>> > > > >
>> > > > > Apart from that the update path is pretty clear:
>> > > > > * move to Pekko 1.0.x
>> > > > > * move to Arterty TCP
>> > > > > * move to Pekko 1.1.x
>> > > > >
>> > > > > If you manage to move to Pekko and were still using classic
>> remoting,
>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you manage the
>> update
>> > > > > to Pekko, going to Artery TCP is a small step.
>> > > > >
>> > > > > Johannes
>> > > > >
>> > > > >
>> ---------------------------------------------------------------------
>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
>> > > > >
>> > > > >
>> > > >
>> > > > --
>> > > >
>> > > > Matthew de Detrich
>> > > >
>> > > > *Aiven Deutschland GmbH*
>> > > >
>> > > > Immanuelkirchstraße 26, 10405 Berlin
>> > > >
>> > > > Amtsgericht Charlottenburg, HRB 209739 B
>> > > >
>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>> > > >
>> > > > *m:* +491603708037
>> > > >
>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>> > > >
>> > >
>> >
>> >
>> > --
>> >
>> > Matthew de Detrich
>> >
>> > *Aiven Deutschland GmbH*
>> >
>> > Immanuelkirchstraße 26, 10405 Berlin
>> >
>> > Amtsgericht Charlottenburg, HRB 209739 B
>> >
>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>> >
>> > *m:* +491603708037
>> >
>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>> For additional commands, e-mail: dev-help@pekko.apache.org
>>
>>
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by Matthew de Detrich <ma...@aiven.io.INVALID>.
> Not upgrading to Netty3
I meant not upgrading to Netty4
On Mon, Sep 11, 2023 at 1:34 PM Matthew de Detrich <
matthew.dedetrich@aiven.io> wrote:
> While I know that people have reservations for keeping the current
> classical transport,
> I just want to relay what I have written at
> https://github.com/apache/incubator-pekko/pull/643
> i.e.
>
> - Upgrading the classical transport from Netty 3 to Netty 4 was in context
> quite trivial
> - Dropping classical transport means that we are technically breaking
> semver (we can only do this in Pekko 2.0.x)
> - Not upgrading to Netty3 means that we are shipping code with known CVE's
>
> For these reasons I personally would prefer to merge the PR and just get
> it done with. Even
> with the PR merged, if we change our minds later nothing is stopping us
> from dropping it
> later (or even reverting the PR when testing is done but I doubt that will
> occur).
>
> There is of course a risk in that updating to Netty 4 might create some
> regressions
> and hence pekko users would complain but I would then argue that this is a
> bug,
> not a feature because we then get actual feedback as to how many people
> are using
> the classical transport which allows us to make more comprehensive
> decisions in
> the future.
>
> On Fri, Sep 8, 2023 at 5:56 PM kerr <he...@gmail.com> wrote:
>
>> And I gathered this result on reddit
>> [image: image.png]
>> 何品
>>
>>
>> kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
>>
>>> I just read
>>>
>>> To switch a full cluster restart is required and any overrides for
>>> classic remoting need to be ported to Artery configuration. Artery has a
>>> completely different protocol, which means that a rolling update is not
>>> supported.
>>> So for Flink use case is that ok?
>>> 何品
>>>
>>>
>>> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
>>>
>>>> I did a poll on reddit and get the result of
>>>> [image: image.png]
>>>> 何品
>>>>
>>>>
>>>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
>>>>
>>>>> > Is it really worth spending any time to support classic remoting?
>>>>> Users
>>>>> would be better served to switch to Artery.
>>>>>
>>>>> I'm one of the user and the China Scala User group may not talk much
>>>>> on web but most of us is using the netty transport, even Lightbend say the
>>>>> `Artery is the best`.
>>>>>
>>>>> Another online product with large scale nodes(here) is running with
>>>>> Akka 2.4.x with the classic transport, and Flink is using the netty.tpc
>>>>> transport at least for now too.
>>>>>
>>>>> I think the community can choose when they migrate the nodes to artery
>>>>> is essential, especially when otherwise your cluster need a hard reboot.
>>>>>
>>>>> AFAIK, the Artery transport is not wire compatible with the
>>>>> classic one.
>>>>>
>>>>> I suggest:
>>>>> 1. Start to send PRs to open sources projects like Flink with the help
>>>>> for migrating to the artery transport.
>>>>> 2. Upgrade to the new version of Netty for bugfix and
>>>>> better performance.
>>>>> 3. Suggest all the new community move to the Artery transport.
>>>>>
>>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
>>>>>
>>>>> Anyway, different company has different requirements, that just my 2
>>>>> cents.
>>>>>
>>>>> 何品
>>>>>
>>>>>
>>>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
>>>>>
>>>>>> This will end up in a vote.
>>>>>>
>>>>>> For me, we shipped 1.0.x. Users who don't want big changes can use
>>>>>> that version. We can agree to support 1.0.x with critical fixes.
>>>>>>
>>>>>> We can now start improving the code for the improvement release
>>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of deprecated
>>>>>> code and most of us appear to believe that classic remoting is long
>>>>>> deprecated.
>>>>>>
>>>>>>
>>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
>>>>>> <ma...@aiven.io.invalid> wrote:
>>>>>> >
>>>>>> > > Is it really worth spending any time to support classic remoting?
>>>>>> Users
>>>>>> > would be better served to switch to Artery.
>>>>>> >
>>>>>> > That depends on what you mean by "support". If agreed on, I would
>>>>>> > propose marking classic remoting as "to be removed" which would
>>>>>> > also mean that the code wouldn't be touched (i.e. no feature
>>>>>> > changes and no bug fixes unless extremely critical).
>>>>>> >
>>>>>> > I am in no way suggesting that we support remoting in a traditional
>>>>>> sense
>>>>>> > and again the only reason why I am even contemplating this is
>>>>>> because
>>>>>> > of the CVE's.
>>>>>> >
>>>>>> > > I don't think it is likely that a significant portion of Akka 2.4
>>>>>> users
>>>>>> > would update to Pekko any time soon if the couldn't update to Akka
>>>>>> 2.5/2.6.
>>>>>> >
>>>>>> > I also had this impression but it's already come up a few times that
>>>>>> > people still use Akka 2.4.x. They likely haven't bothered updating
>>>>>> > because they didn't see a need to (quite famously Akka is quite
>>>>>> stable
>>>>>> > and there are cases of companies using it for years without needing
>>>>>> to
>>>>>> > update, as stated by Lightbend's CEO) but with the license change
>>>>>> > it created a catalyst/trigger for some users to consider moving to
>>>>>> Pekko.
>>>>>> >
>>>>>> > On this note I would also be wary in making assumptions on what
>>>>>> versions
>>>>>> > of Akka people happen to be using. As a corollary in the discussion
>>>>>> on
>>>>>> > dropping JDK 8, when we suggested that we quite quickly got
>>>>>> feedback that
>>>>>> > there are people still using JDK 8. And while I know all of the
>>>>>> arguments
>>>>>> > that you really should not be using JDK 8 can be carried over to
>>>>>> Akka.
>>>>>> >
>>>>>> > Admittedly it's quite hard to get good feedback on what versions
>>>>>> people are
>>>>>> > using, just saying we should be a bit more careful in making these
>>>>>> > assumptions.
>>>>>> > After all, people are still using JDK 8 ;)
>>>>>> >
>>>>>> >
>>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <nv...@gmail.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > > Akka 2.4 has been EoL since end of 2017.
>>>>>> > > I don't think it is likely that a significant portion of Akka 2.4
>>>>>> users
>>>>>> > > would update to Pekko any time soon if the couldn't update to
>>>>>> Akka 2.5/2.6.
>>>>>> > >
>>>>>> > > Is it really worth spending any time to support classic remoting?
>>>>>> Users
>>>>>> > > would be better served to switch to Artery.
>>>>>> > >
>>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
>>>>>> > > <ma...@aiven.io.invalid> wrote:
>>>>>> > >
>>>>>> > > > > If you manage to move to Pekko and were still using classic
>>>>>> remoting,
>>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you manage the
>>>>>> update
>>>>>> > > > to Pekko, going to Artery TCP is a small step.
>>>>>> > > >
>>>>>> > > > One of the reasons why I was saying this is that I was under the
>>>>>> > > impression
>>>>>> > > > that there are a non-trivial amount of users still on Akka 2.4
>>>>>> (not sure
>>>>>> > > > why
>>>>>> > > > this is the case but kerr was telling me this).
>>>>>> > > >
>>>>>> > > > > If people are interested in keeping it around and also opt in
>>>>>> to
>>>>>> > > > maintaining it (which in the first place means, making the
>>>>>> tests for
>>>>>> > > > it work *now*), than fine, but otherwise, being able to use it
>>>>>> on
>>>>>> > > > 1.0.x is already a big benefit.
>>>>>> > > >
>>>>>> > > > Since classic remoting is currently using Netty 3 which has
>>>>>> CVE's
>>>>>> > > > I don't think it's wise to encourage people that still happen
>>>>>> to be
>>>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't for final
>>>>>> > > > version of netty 3 having CVE's then I would have no qualms
>>>>>> > > > for dropping it in the 1.1.x series (in fact if there was a
>>>>>> > > > hypothetical netty 3 without CVE's we would have already
>>>>>> > > > updated to it in the 1.0.x series and dropped classic remoting
>>>>>> in
>>>>>> > > > 1.1.x without a heartbeat).
>>>>>> > > >
>>>>>> > > > I would propose that if we do accept keeping classic remoting
>>>>>> > > > and updating to Netty 4, we would mark the feature to be
>>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade that
>>>>>> > > > part of code would be untouched unless there are critical
>>>>>> > > > bugs/regressions.
>>>>>> > > >
>>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
>>>>>> > > > johannes.rudolph@gmail.com>
>>>>>> > > > wrote:
>>>>>> > > >
>>>>>> > > > > We definitely should remove it for 1.1.x. There's no
>>>>>> technical reason
>>>>>> > > > > to keep it because the newer artery TCP transport just
>>>>>> supersedes it.
>>>>>> > > > > As discussed before, the problem with keeping old features
>>>>>> around is
>>>>>> > > > > "death by a thousand cuts". The classic remoting backend is
>>>>>> the prime
>>>>>> > > > > example of it because it is basically unmaintained for years
>>>>>> and
>>>>>> > > > > there's basically no expert knowledge available about how to
>>>>>> maintain
>>>>>> > > > > it or how to fix the existing bugs. Testing remoting backends
>>>>>> has
>>>>>> > > > > shown to be very complicated and often unreliable and being
>>>>>> able to
>>>>>> > > > > remove this particular component from the test matrix will be
>>>>>> a huge
>>>>>> > > > > improvement.
>>>>>> > > > >
>>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
>>>>>> > > > > <ma...@aiven.io.invalid> wrote:
>>>>>> > > > > > I would prefer adding in netty 4 support for the classic
>>>>>> transport
>>>>>> > > > > > mechanism because it gives
>>>>>> > > > > > people an upgrade path while allowing them to resolve the
>>>>>> CVE issues.
>>>>>> > > > If
>>>>>> > > > > > Pekko 1.1.x didn't
>>>>>> > > > > > support the classic transport mechanism then it forces
>>>>>> Pekko users to
>>>>>> > > > be
>>>>>> > > > > > stuck on 1.0.x
>>>>>> > > > > > with no easy upgrade path.
>>>>>> > > > >
>>>>>> > > > > We are not responsible for giving people an easy upgrade path
>>>>>> when
>>>>>> > > > > they were stuck on features like classic remoting which has
>>>>>> basically
>>>>>> > > > > been outdated for years. It is very important that, going
>>>>>> forward,
>>>>>> > > > > that we focus our attention on the most important features
>>>>>> and make
>>>>>> > > > > sure to not get stuck in maintaining old parts that we cannot
>>>>>> (or
>>>>>> > > > > don't want to) care for.
>>>>>> > > > >
>>>>>> > > > > If people are interested in keeping it around and also opt in
>>>>>> to
>>>>>> > > > > maintaining it (which in the first place means, making the
>>>>>> tests for
>>>>>> > > > > it work *now*), than fine, but otherwise, being able to use
>>>>>> it on
>>>>>> > > > > 1.0.x is already a big benefit.
>>>>>> > > > >
>>>>>> > > > > Apart from that the update path is pretty clear:
>>>>>> > > > > * move to Pekko 1.0.x
>>>>>> > > > > * move to Arterty TCP
>>>>>> > > > > * move to Pekko 1.1.x
>>>>>> > > > >
>>>>>> > > > > If you manage to move to Pekko and were still using classic
>>>>>> remoting,
>>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you manage
>>>>>> the update
>>>>>> > > > > to Pekko, going to Artery TCP is a small step.
>>>>>> > > > >
>>>>>> > > > > Johannes
>>>>>> > > > >
>>>>>> > > > >
>>>>>> ---------------------------------------------------------------------
>>>>>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>>>>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
>>>>>> > > > >
>>>>>> > > > >
>>>>>> > > >
>>>>>> > > > --
>>>>>> > > >
>>>>>> > > > Matthew de Detrich
>>>>>> > > >
>>>>>> > > > *Aiven Deutschland GmbH*
>>>>>> > > >
>>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
>>>>>> > > >
>>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
>>>>>> > > >
>>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>>>>> > > >
>>>>>> > > > *m:* +491603708037
>>>>>> > > >
>>>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>>>>> > > >
>>>>>> > >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> >
>>>>>> > Matthew de Detrich
>>>>>> >
>>>>>> > *Aiven Deutschland GmbH*
>>>>>> >
>>>>>> > Immanuelkirchstraße 26, 10405 Berlin
>>>>>> >
>>>>>> > Amtsgericht Charlottenburg, HRB 209739 B
>>>>>> >
>>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>>>>> >
>>>>>> > *m:* +491603708037
>>>>>> >
>>>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>>>>> For additional commands, e-mail: dev-help@pekko.apache.org
>>>>>>
>>>>>>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>
--
Matthew de Detrich
*Aiven Deutschland GmbH*
Immanuelkirchstraße 26, 10405 Berlin
Amtsgericht Charlottenburg, HRB 209739 B
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
*m:* +491603708037
*w:* aiven.io *e:* matthew.dedetrich@aiven.io
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by kerr <he...@gmail.com>.
I have run sevral tests and multiJvm tests all passed.
During the tests, I find the artery areon test is unstable than the current
classic transport.
何品
Matthew de Detrich <ma...@aiven.io.invalid> 于2023年9月13日周三
15:09写道：
> So we got a response from the flink thread, I would recommend reading
> it but a quick summary is that Flink already planned to move
> from classic remoting to artery[1] however they hit issues which
> is why they are still on classic remoting.
>
> Hence they also have a preference for Pekko moving classic
> remoting to netty4 so that they can also get rid of the CVE's
> from netty3.
>
> Given this and the fact that He-Pin did excellent work with the upgrade
> in his PR, I don't think the arguments for not merging it are holding
> much weight
>
> [1]: https://github.com/apache/flink/pull/22271
>
> On Tue, Sep 12, 2023 at 10:48 AM Matthew de Detrich <
> matthew.dedetrich@aiven.io> wrote:
>
> > I created a thread on the Flink dev mailing list with a summary of the
> > situation at
> > https://lists.apache.org/thread/o774bkox2mvyl8r3xld6kkg5krstgrow
> >
> > On Mon, Sep 11, 2023 at 4:43 PM Matthew de Detrich <
> > matthew.dedetrich@aiven.io> wrote:
> >
> >> I am not sure there is any discussion on this, just the apparent fact
> >> that they happen to be currently using classic remoting
> >>
> >> On Mon, Sep 11, 2023 at 4:37 PM PJ Fanning <fa...@gmail.com> wrote:
> >>
> >>> Can we get a link to the Flink discussion?
> >>>
> >>> On Mon, 11 Sept 2023 at 15:25, Matthew de Detrich
> >>> <ma...@aiven.io.invalid> wrote:
> >>> >
> >>> > > Flink is using pekko now, and a hard reboot may not be ok in some
> >>> cases,
> >>> > and because Pekko ships a Netty3 with many CVEs, they may decide to
> >>> drop
> >>> > pekko and come out with a dedicated rpc implementation based on
> Netty4
> >>> too.
> >>> >
> >>> > Wasn't aware of this (I thought Flink used the new remoting
> protocol).
> >>> > I think this is a strong enough reason by itself to upgrade Netty4,
> but
> >>> > ofcourse
> >>> > deprecated classic remoting.
> >>> >
> >>> > On Mon, Sep 11, 2023 at 3:22 PM kerr <he...@gmail.com> wrote:
> >>> >
> >>> > > Netty 4 is much better than Netty 3 and we can schedule tests to
> run
> >>> for it
> >>> > > and make sure the tests all passed for servertimes.
> >>> > > Otherwise, we will have a CVEs old netty 3 in classpath.
> >>> > >
> >>> > > Flink is using pekko now, and a hard reboot may not be ok in some
> >>> cases,
> >>> > > and because Pekko ships a Netty3 with many CVEs, they may decide to
> >>> drop
> >>> > > pekko and come out with a dedicated rpc implementation based on
> >>> Netty4 too.
> >>> > > But after we migrate to Netty 4, they can defer that, I will ask
> >>> around the
> >>> > > PMC of Flink at here to let them taking a look.
> >>> > >
> >>> > > 何品
> >>> > >
> >>> > >
> >>> > > Matthew de Detrich <ma...@aiven.io.invalid>
> >>> 于2023年9月11日周一
> >>> > > 19:35写道：
> >>> > >
> >>> > > > > that this is a bug, not a feature
> >>> > > >
> >>> > > > Also meant to say that this is a feature, not a bug
> >>> > > >
> >>> > > > On Mon, Sep 11, 2023 at 1:34 PM Matthew de Detrich <
> >>> > > > matthew.dedetrich@aiven.io> wrote:
> >>> > > >
> >>> > > > > While I know that people have reservations for keeping the
> >>> current
> >>> > > > > classical transport,
> >>> > > > > I just want to relay what I have written at
> >>> > > > > https://github.com/apache/incubator-pekko/pull/643
> >>> > > > > i.e.
> >>> > > > >
> >>> > > > > - Upgrading the classical transport from Netty 3 to Netty 4 was
> >>> in
> >>> > > > context
> >>> > > > > quite trivial
> >>> > > > > - Dropping classical transport means that we are technically
> >>> breaking
> >>> > > > > semver (we can only do this in Pekko 2.0.x)
> >>> > > > > - Not upgrading to Netty3 means that we are shipping code with
> >>> known
> >>> > > > CVE's
> >>> > > > >
> >>> > > > > For these reasons I personally would prefer to merge the PR and
> >>> just
> >>> > > get
> >>> > > > > it done with. Even
> >>> > > > > with the PR merged, if we change our minds later nothing is
> >>> stopping us
> >>> > > > > from dropping it
> >>> > > > > later (or even reverting the PR when testing is done but I
> doubt
> >>> that
> >>> > > > will
> >>> > > > > occur).
> >>> > > > >
> >>> > > > > There is of course a risk in that updating to Netty 4 might
> >>> create some
> >>> > > > > regressions
> >>> > > > > and hence pekko users would complain but I would then argue
> that
> >>> this
> >>> > > is
> >>> > > > a
> >>> > > > > bug,
> >>> > > > > not a feature because we then get actual feedback as to how
> many
> >>> people
> >>> > > > > are using
> >>> > > > > the classical transport which allows us to make more
> >>> comprehensive
> >>> > > > > decisions in
> >>> > > > > the future.
> >>> > > > >
> >>> > > > > On Fri, Sep 8, 2023 at 5:56 PM kerr <he...@gmail.com>
> wrote:
> >>> > > > >
> >>> > > > >> And I gathered this result on reddit
> >>> > > > >> [image: image.png]
> >>> > > > >> 何品
> >>> > > > >>
> >>> > > > >>
> >>> > > > >> kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
> >>> > > > >>
> >>> > > > >>> I just read
> >>> > > > >>>
> >>> > > > >>> To switch a full cluster restart is required and any
> overrides
> >>> for
> >>> > > > >>> classic remoting need to be ported to Artery configuration.
> >>> Artery
> >>> > > has
> >>> > > > a
> >>> > > > >>> completely different protocol, which means that a rolling
> >>> update is
> >>> > > not
> >>> > > > >>> supported.
> >>> > > > >>> So for Flink use case is that ok?
> >>> > > > >>> 何品
> >>> > > > >>>
> >>> > > > >>>
> >>> > > > >>> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
> >>> > > > >>>
> >>> > > > >>>> I did a poll on reddit and get the result of
> >>> > > > >>>> [image: image.png]
> >>> > > > >>>> 何品
> >>> > > > >>>>
> >>> > > > >>>>
> >>> > > > >>>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
> >>> > > > >>>>
> >>> > > > >>>>> > Is it really worth spending any time to support classic
> >>> remoting?
> >>> > > > >>>>> Users
> >>> > > > >>>>> would be better served to switch to Artery.
> >>> > > > >>>>>
> >>> > > > >>>>> I'm one of the user and the China Scala User group may not
> >>> talk
> >>> > > much
> >>> > > > >>>>> on web but most of us is using the netty transport, even
> >>> Lightbend
> >>> > > > say the
> >>> > > > >>>>> `Artery is the best`.
> >>> > > > >>>>>
> >>> > > > >>>>> Another online product with large scale nodes(here) is
> >>> running with
> >>> > > > >>>>> Akka 2.4.x with the classic transport, and Flink is using
> the
> >>> > > > netty.tpc
> >>> > > > >>>>> transport at least for now too.
> >>> > > > >>>>>
> >>> > > > >>>>> I think the community can choose when they migrate the
> nodes
> >>> to
> >>> > > > artery
> >>> > > > >>>>> is essential, especially when otherwise your cluster need a
> >>> hard
> >>> > > > reboot.
> >>> > > > >>>>>
> >>> > > > >>>>> AFAIK, the Artery transport is not wire compatible with the
> >>> > > > >>>>> classic one.
> >>> > > > >>>>>
> >>> > > > >>>>> I suggest:
> >>> > > > >>>>> 1. Start to send PRs to open sources projects like Flink
> >>> with the
> >>> > > > help
> >>> > > > >>>>> for migrating to the artery transport.
> >>> > > > >>>>> 2. Upgrade to the new version of Netty for bugfix and
> >>> > > > >>>>> better performance.
> >>> > > > >>>>> 3. Suggest all the new community move to the Artery
> >>> transport.
> >>> > > > >>>>>
> >>> > > > >>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
> >>> > > > >>>>>
> >>> > > > >>>>> Anyway, different company has different requirements, that
> >>> just my
> >>> > > 2
> >>> > > > >>>>> cents.
> >>> > > > >>>>>
> >>> > > > >>>>> 何品
> >>> > > > >>>>>
> >>> > > > >>>>>
> >>> > > > >>>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
> >>> > > > >>>>>
> >>> > > > >>>>>> This will end up in a vote.
> >>> > > > >>>>>>
> >>> > > > >>>>>> For me, we shipped 1.0.x. Users who don't want big changes
> >>> can use
> >>> > > > >>>>>> that version. We can agree to support 1.0.x with critical
> >>> fixes.
> >>> > > > >>>>>>
> >>> > > > >>>>>> We can now start improving the code for the improvement
> >>> release
> >>> > > > >>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of
> >>> deprecated
> >>> > > > >>>>>> code and most of us appear to believe that classic
> remoting
> >>> is
> >>> > > long
> >>> > > > >>>>>> deprecated.
> >>> > > > >>>>>>
> >>> > > > >>>>>>
> >>> > > > >>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
> >>> > > > >>>>>> <ma...@aiven.io.invalid> wrote:
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > > Is it really worth spending any time to support
> classic
> >>> > > > remoting?
> >>> > > > >>>>>> Users
> >>> > > > >>>>>> > would be better served to switch to Artery.
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > That depends on what you mean by "support". If agreed
> on,
> >>> I
> >>> > > would
> >>> > > > >>>>>> > propose marking classic remoting as "to be removed"
> which
> >>> would
> >>> > > > >>>>>> > also mean that the code wouldn't be touched (i.e. no
> >>> feature
> >>> > > > >>>>>> > changes and no bug fixes unless extremely critical).
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > I am in no way suggesting that we support remoting in a
> >>> > > > traditional
> >>> > > > >>>>>> sense
> >>> > > > >>>>>> > and again the only reason why I am even contemplating
> >>> this is
> >>> > > > >>>>>> because
> >>> > > > >>>>>> > of the CVE's.
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > > I don't think it is likely that a significant portion
> >>> of Akka
> >>> > > > 2.4
> >>> > > > >>>>>> users
> >>> > > > >>>>>> > would update to Pekko any time soon if the couldn't
> >>> update to
> >>> > > Akka
> >>> > > > >>>>>> 2.5/2.6.
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > I also had this impression but it's already come up a
> few
> >>> times
> >>> > > > that
> >>> > > > >>>>>> > people still use Akka 2.4.x. They likely haven't
> bothered
> >>> > > updating
> >>> > > > >>>>>> > because they didn't see a need to (quite famously Akka
> is
> >>> quite
> >>> > > > >>>>>> stable
> >>> > > > >>>>>> > and there are cases of companies using it for years
> >>> without
> >>> > > > needing
> >>> > > > >>>>>> to
> >>> > > > >>>>>> > update, as stated by Lightbend's CEO) but with the
> license
> >>> > > change
> >>> > > > >>>>>> > it created a catalyst/trigger for some users to consider
> >>> moving
> >>> > > to
> >>> > > > >>>>>> Pekko.
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > On this note I would also be wary in making assumptions
> >>> on what
> >>> > > > >>>>>> versions
> >>> > > > >>>>>> > of Akka people happen to be using. As a corollary in the
> >>> > > > discussion
> >>> > > > >>>>>> on
> >>> > > > >>>>>> > dropping JDK 8, when we suggested that we quite quickly
> >>> got
> >>> > > > >>>>>> feedback that
> >>> > > > >>>>>> > there are people still using JDK 8. And while I know all
> >>> of the
> >>> > > > >>>>>> arguments
> >>> > > > >>>>>> > that you really should not be using JDK 8 can be carried
> >>> over to
> >>> > > > >>>>>> Akka.
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > Admittedly it's quite hard to get good feedback on what
> >>> versions
> >>> > > > >>>>>> people are
> >>> > > > >>>>>> > using, just saying we should be a bit more careful in
> >>> making
> >>> > > these
> >>> > > > >>>>>> > assumptions.
> >>> > > > >>>>>> > After all, people are still using JDK 8 ;)
> >>> > > > >>>>>> >
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <
> >>> > > > nvollmar@gmail.com>
> >>> > > > >>>>>> wrote:
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > > Akka 2.4 has been EoL since end of 2017.
> >>> > > > >>>>>> > > I don't think it is likely that a significant portion
> >>> of Akka
> >>> > > > 2.4
> >>> > > > >>>>>> users
> >>> > > > >>>>>> > > would update to Pekko any time soon if the couldn't
> >>> update to
> >>> > > > >>>>>> Akka 2.5/2.6.
> >>> > > > >>>>>> > >
> >>> > > > >>>>>> > > Is it really worth spending any time to support
> classic
> >>> > > > remoting?
> >>> > > > >>>>>> Users
> >>> > > > >>>>>> > > would be better served to switch to Artery.
> >>> > > > >>>>>> > >
> >>> > > > >>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
> >>> > > > >>>>>> > > <ma...@aiven.io.invalid> wrote:
> >>> > > > >>>>>> > >
> >>> > > > >>>>>> > > > > If you manage to move to Pekko and were still
> using
> >>> > > classic
> >>> > > > >>>>>> remoting,
> >>> > > > >>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you
> >>> manage
> >>> > > > the
> >>> > > > >>>>>> update
> >>> > > > >>>>>> > > > to Pekko, going to Artery TCP is a small step.
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > One of the reasons why I was saying this is that I
> >>> was under
> >>> > > > the
> >>> > > > >>>>>> > > impression
> >>> > > > >>>>>> > > > that there are a non-trivial amount of users still
> on
> >>> Akka
> >>> > > 2.4
> >>> > > > >>>>>> (not sure
> >>> > > > >>>>>> > > > why
> >>> > > > >>>>>> > > > this is the case but kerr was telling me this).
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > > If people are interested in keeping it around and
> >>> also opt
> >>> > > > in
> >>> > > > >>>>>> to
> >>> > > > >>>>>> > > > maintaining it (which in the first place means,
> >>> making the
> >>> > > > >>>>>> tests for
> >>> > > > >>>>>> > > > it work *now*), than fine, but otherwise, being able
> >>> to use
> >>> > > it
> >>> > > > >>>>>> on
> >>> > > > >>>>>> > > > 1.0.x is already a big benefit.
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > Since classic remoting is currently using Netty 3
> >>> which has
> >>> > > > >>>>>> CVE's
> >>> > > > >>>>>> > > > I don't think it's wise to encourage people that
> still
> >>> > > happen
> >>> > > > >>>>>> to be
> >>> > > > >>>>>> > > > using classic remoting to stay on 1.0.x. If it
> wasn't
> >>> for
> >>> > > > final
> >>> > > > >>>>>> > > > version of netty 3 having CVE's then I would have no
> >>> qualms
> >>> > > > >>>>>> > > > for dropping it in the 1.1.x series (in fact if
> there
> >>> was a
> >>> > > > >>>>>> > > > hypothetical netty 3 without CVE's we would have
> >>> already
> >>> > > > >>>>>> > > > updated to it in the 1.0.x series and dropped
> classic
> >>> > > remoting
> >>> > > > >>>>>> in
> >>> > > > >>>>>> > > > 1.1.x without a heartbeat).
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > I would propose that if we do accept keeping classic
> >>> > > remoting
> >>> > > > >>>>>> > > > and updating to Netty 4, we would mark the feature
> to
> >>> be
> >>> > > > >>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade
> >>> that
> >>> > > > >>>>>> > > > part of code would be untouched unless there are
> >>> critical
> >>> > > > >>>>>> > > > bugs/regressions.
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
> >>> > > > >>>>>> > > > johannes.rudolph@gmail.com>
> >>> > > > >>>>>> > > > wrote:
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > > We definitely should remove it for 1.1.x. There's
> no
> >>> > > > >>>>>> technical reason
> >>> > > > >>>>>> > > > > to keep it because the newer artery TCP transport
> >>> just
> >>> > > > >>>>>> supersedes it.
> >>> > > > >>>>>> > > > > As discussed before, the problem with keeping old
> >>> features
> >>> > > > >>>>>> around is
> >>> > > > >>>>>> > > > > "death by a thousand cuts". The classic remoting
> >>> backend
> >>> > > is
> >>> > > > >>>>>> the prime
> >>> > > > >>>>>> > > > > example of it because it is basically unmaintained
> >>> for
> >>> > > years
> >>> > > > >>>>>> and
> >>> > > > >>>>>> > > > > there's basically no expert knowledge available
> >>> about how
> >>> > > to
> >>> > > > >>>>>> maintain
> >>> > > > >>>>>> > > > > it or how to fix the existing bugs. Testing
> remoting
> >>> > > > backends
> >>> > > > >>>>>> has
> >>> > > > >>>>>> > > > > shown to be very complicated and often unreliable
> >>> and
> >>> > > being
> >>> > > > >>>>>> able to
> >>> > > > >>>>>> > > > > remove this particular component from the test
> >>> matrix will
> >>> > > > be
> >>> > > > >>>>>> a huge
> >>> > > > >>>>>> > > > > improvement.
> >>> > > > >>>>>> > > > >
> >>> > > > >>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de
> Detrich
> >>> > > > >>>>>> > > > > <ma...@aiven.io.invalid> wrote:
> >>> > > > >>>>>> > > > > > I would prefer adding in netty 4 support for the
> >>> classic
> >>> > > > >>>>>> transport
> >>> > > > >>>>>> > > > > > mechanism because it gives
> >>> > > > >>>>>> > > > > > people an upgrade path while allowing them to
> >>> resolve
> >>> > > the
> >>> > > > >>>>>> CVE issues.
> >>> > > > >>>>>> > > > If
> >>> > > > >>>>>> > > > > > Pekko 1.1.x didn't
> >>> > > > >>>>>> > > > > > support the classic transport mechanism then it
> >>> forces
> >>> > > > >>>>>> Pekko users to
> >>> > > > >>>>>> > > > be
> >>> > > > >>>>>> > > > > > stuck on 1.0.x
> >>> > > > >>>>>> > > > > > with no easy upgrade path.
> >>> > > > >>>>>> > > > >
> >>> > > > >>>>>> > > > > We are not responsible for giving people an easy
> >>> upgrade
> >>> > > > path
> >>> > > > >>>>>> when
> >>> > > > >>>>>> > > > > they were stuck on features like classic remoting
> >>> which
> >>> > > has
> >>> > > > >>>>>> basically
> >>> > > > >>>>>> > > > > been outdated for years. It is very important
> that,
> >>> going
> >>> > > > >>>>>> forward,
> >>> > > > >>>>>> > > > > that we focus our attention on the most important
> >>> features
> >>> > > > >>>>>> and make
> >>> > > > >>>>>> > > > > sure to not get stuck in maintaining old parts
> that
> >>> we
> >>> > > > cannot
> >>> > > > >>>>>> (or
> >>> > > > >>>>>> > > > > don't want to) care for.
> >>> > > > >>>>>> > > > >
> >>> > > > >>>>>> > > > > If people are interested in keeping it around and
> >>> also opt
> >>> > > > in
> >>> > > > >>>>>> to
> >>> > > > >>>>>> > > > > maintaining it (which in the first place means,
> >>> making the
> >>> > > > >>>>>> tests for
> >>> > > > >>>>>> > > > > it work *now*), than fine, but otherwise, being
> >>> able to
> >>> > > use
> >>> > > > >>>>>> it on
> >>> > > > >>>>>> > > > > 1.0.x is already a big benefit.
> >>> > > > >>>>>> > > > >
> >>> > > > >>>>>> > > > > Apart from that the update path is pretty clear:
> >>> > > > >>>>>> > > > > * move to Pekko 1.0.x
> >>> > > > >>>>>> > > > > * move to Arterty TCP
> >>> > > > >>>>>> > > > > * move to Pekko 1.1.x
> >>> > > > >>>>>> > > > >
> >>> > > > >>>>>> > > > > If you manage to move to Pekko and were still
> using
> >>> > > classic
> >>> > > > >>>>>> remoting,
> >>> > > > >>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If
> you
> >>> > > manage
> >>> > > > >>>>>> the update
> >>> > > > >>>>>> > > > > to Pekko, going to Artery TCP is a small step.
> >>> > > > >>>>>> > > > >
> >>> > > > >>>>>> > > > > Johannes
> >>> > > > >>>>>> > > > >
> >>> > > > >>>>>> > > > >
> >>> > > > >>>>>>
> >>> > > >
> >>> ---------------------------------------------------------------------
> >>> > > > >>>>>> > > > > To unsubscribe, e-mail:
> >>> dev-unsubscribe@pekko.apache.org
> >>> > > > >>>>>> > > > > For additional commands, e-mail:
> >>> > > dev-help@pekko.apache.org
> >>> > > > >>>>>> > > > >
> >>> > > > >>>>>> > > > >
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > --
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > Matthew de Detrich
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > *Aiven Deutschland GmbH*
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > *m:* +491603708037
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >>> > > > >>>>>> > > >
> >>> > > > >>>>>> > >
> >>> > > > >>>>>> >
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > --
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > Matthew de Detrich
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > *Aiven Deutschland GmbH*
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > Immanuelkirchstraße 26, 10405 Berlin
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > Amtsgericht Charlottenburg, HRB 209739 B
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > *m:* +491603708037
> >>> > > > >>>>>> >
> >>> > > > >>>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >>> > > > >>>>>>
> >>> > > > >>>>>>
> >>> > > >
> >>> ---------------------------------------------------------------------
> >>> > > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> >>> > > > >>>>>> For additional commands, e-mail:
> dev-help@pekko.apache.org
> >>> > > > >>>>>>
> >>> > > > >>>>>>
> >>> > > > >
> >>> > > > > --
> >>> > > > >
> >>> > > > > Matthew de Detrich
> >>> > > > >
> >>> > > > > *Aiven Deutschland GmbH*
> >>> > > > >
> >>> > > > > Immanuelkirchstraße 26, 10405 Berlin
> >>> > > > >
> >>> > > > > Amtsgericht Charlottenburg, HRB 209739 B
> >>> > > > >
> >>> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >>> > > > >
> >>> > > > > *m:* +491603708037
> >>> > > > >
> >>> > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >>> > > > >
> >>> > > >
> >>> > > >
> >>> > > > --
> >>> > > >
> >>> > > > Matthew de Detrich
> >>> > > >
> >>> > > > *Aiven Deutschland GmbH*
> >>> > > >
> >>> > > > Immanuelkirchstraße 26, 10405 Berlin
> >>> > > >
> >>> > > > Amtsgericht Charlottenburg, HRB 209739 B
> >>> > > >
> >>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >>> > > >
> >>> > > > *m:* +491603708037
> >>> > > >
> >>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >>> > > >
> >>> > >
> >>> >
> >>> >
> >>> > --
> >>> >
> >>> > Matthew de Detrich
> >>> >
> >>> > *Aiven Deutschland GmbH*
> >>> >
> >>> > Immanuelkirchstraße 26, 10405 Berlin
> >>> >
> >>> > Amtsgericht Charlottenburg, HRB 209739 B
> >>> >
> >>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >>> >
> >>> > *m:* +491603708037
> >>> >
> >>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> >>> For additional commands, e-mail: dev-help@pekko.apache.org
> >>>
> >>>
> >>
> >> --
> >>
> >> Matthew de Detrich
> >>
> >> *Aiven Deutschland GmbH*
> >>
> >> Immanuelkirchstraße 26, 10405 Berlin
> >>
> >> Amtsgericht Charlottenburg, HRB 209739 B
> >>
> >> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >>
> >> *m:* +491603708037
> >>
> >> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >>
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by Matthew de Detrich <ma...@aiven.io.INVALID>.
So we got a response from the flink thread, I would recommend reading
it but a quick summary is that Flink already planned to move
from classic remoting to artery[1] however they hit issues which
is why they are still on classic remoting.
Hence they also have a preference for Pekko moving classic
remoting to netty4 so that they can also get rid of the CVE's
from netty3.
Given this and the fact that He-Pin did excellent work with the upgrade
in his PR, I don't think the arguments for not merging it are holding
much weight
[1]: https://github.com/apache/flink/pull/22271
On Tue, Sep 12, 2023 at 10:48 AM Matthew de Detrich <
matthew.dedetrich@aiven.io> wrote:
> I created a thread on the Flink dev mailing list with a summary of the
> situation at
> https://lists.apache.org/thread/o774bkox2mvyl8r3xld6kkg5krstgrow
>
> On Mon, Sep 11, 2023 at 4:43 PM Matthew de Detrich <
> matthew.dedetrich@aiven.io> wrote:
>
>> I am not sure there is any discussion on this, just the apparent fact
>> that they happen to be currently using classic remoting
>>
>> On Mon, Sep 11, 2023 at 4:37 PM PJ Fanning <fa...@gmail.com> wrote:
>>
>>> Can we get a link to the Flink discussion?
>>>
>>> On Mon, 11 Sept 2023 at 15:25, Matthew de Detrich
>>> <ma...@aiven.io.invalid> wrote:
>>> >
>>> > > Flink is using pekko now, and a hard reboot may not be ok in some
>>> cases,
>>> > and because Pekko ships a Netty3 with many CVEs, they may decide to
>>> drop
>>> > pekko and come out with a dedicated rpc implementation based on Netty4
>>> too.
>>> >
>>> > Wasn't aware of this (I thought Flink used the new remoting protocol).
>>> > I think this is a strong enough reason by itself to upgrade Netty4, but
>>> > ofcourse
>>> > deprecated classic remoting.
>>> >
>>> > On Mon, Sep 11, 2023 at 3:22 PM kerr <he...@gmail.com> wrote:
>>> >
>>> > > Netty 4 is much better than Netty 3 and we can schedule tests to run
>>> for it
>>> > > and make sure the tests all passed for servertimes.
>>> > > Otherwise, we will have a CVEs old netty 3 in classpath.
>>> > >
>>> > > Flink is using pekko now, and a hard reboot may not be ok in some
>>> cases,
>>> > > and because Pekko ships a Netty3 with many CVEs, they may decide to
>>> drop
>>> > > pekko and come out with a dedicated rpc implementation based on
>>> Netty4 too.
>>> > > But after we migrate to Netty 4, they can defer that, I will ask
>>> around the
>>> > > PMC of Flink at here to let them taking a look.
>>> > >
>>> > > 何品
>>> > >
>>> > >
>>> > > Matthew de Detrich <ma...@aiven.io.invalid>
>>> 于2023年9月11日周一
>>> > > 19:35写道：
>>> > >
>>> > > > > that this is a bug, not a feature
>>> > > >
>>> > > > Also meant to say that this is a feature, not a bug
>>> > > >
>>> > > > On Mon, Sep 11, 2023 at 1:34 PM Matthew de Detrich <
>>> > > > matthew.dedetrich@aiven.io> wrote:
>>> > > >
>>> > > > > While I know that people have reservations for keeping the
>>> current
>>> > > > > classical transport,
>>> > > > > I just want to relay what I have written at
>>> > > > > https://github.com/apache/incubator-pekko/pull/643
>>> > > > > i.e.
>>> > > > >
>>> > > > > - Upgrading the classical transport from Netty 3 to Netty 4 was
>>> in
>>> > > > context
>>> > > > > quite trivial
>>> > > > > - Dropping classical transport means that we are technically
>>> breaking
>>> > > > > semver (we can only do this in Pekko 2.0.x)
>>> > > > > - Not upgrading to Netty3 means that we are shipping code with
>>> known
>>> > > > CVE's
>>> > > > >
>>> > > > > For these reasons I personally would prefer to merge the PR and
>>> just
>>> > > get
>>> > > > > it done with. Even
>>> > > > > with the PR merged, if we change our minds later nothing is
>>> stopping us
>>> > > > > from dropping it
>>> > > > > later (or even reverting the PR when testing is done but I doubt
>>> that
>>> > > > will
>>> > > > > occur).
>>> > > > >
>>> > > > > There is of course a risk in that updating to Netty 4 might
>>> create some
>>> > > > > regressions
>>> > > > > and hence pekko users would complain but I would then argue that
>>> this
>>> > > is
>>> > > > a
>>> > > > > bug,
>>> > > > > not a feature because we then get actual feedback as to how many
>>> people
>>> > > > > are using
>>> > > > > the classical transport which allows us to make more
>>> comprehensive
>>> > > > > decisions in
>>> > > > > the future.
>>> > > > >
>>> > > > > On Fri, Sep 8, 2023 at 5:56 PM kerr <he...@gmail.com> wrote:
>>> > > > >
>>> > > > >> And I gathered this result on reddit
>>> > > > >> [image: image.png]
>>> > > > >> 何品
>>> > > > >>
>>> > > > >>
>>> > > > >> kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
>>> > > > >>
>>> > > > >>> I just read
>>> > > > >>>
>>> > > > >>> To switch a full cluster restart is required and any overrides
>>> for
>>> > > > >>> classic remoting need to be ported to Artery configuration.
>>> Artery
>>> > > has
>>> > > > a
>>> > > > >>> completely different protocol, which means that a rolling
>>> update is
>>> > > not
>>> > > > >>> supported.
>>> > > > >>> So for Flink use case is that ok?
>>> > > > >>> 何品
>>> > > > >>>
>>> > > > >>>
>>> > > > >>> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
>>> > > > >>>
>>> > > > >>>> I did a poll on reddit and get the result of
>>> > > > >>>> [image: image.png]
>>> > > > >>>> 何品
>>> > > > >>>>
>>> > > > >>>>
>>> > > > >>>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
>>> > > > >>>>
>>> > > > >>>>> > Is it really worth spending any time to support classic
>>> remoting?
>>> > > > >>>>> Users
>>> > > > >>>>> would be better served to switch to Artery.
>>> > > > >>>>>
>>> > > > >>>>> I'm one of the user and the China Scala User group may not
>>> talk
>>> > > much
>>> > > > >>>>> on web but most of us is using the netty transport, even
>>> Lightbend
>>> > > > say the
>>> > > > >>>>> `Artery is the best`.
>>> > > > >>>>>
>>> > > > >>>>> Another online product with large scale nodes(here) is
>>> running with
>>> > > > >>>>> Akka 2.4.x with the classic transport, and Flink is using the
>>> > > > netty.tpc
>>> > > > >>>>> transport at least for now too.
>>> > > > >>>>>
>>> > > > >>>>> I think the community can choose when they migrate the nodes
>>> to
>>> > > > artery
>>> > > > >>>>> is essential, especially when otherwise your cluster need a
>>> hard
>>> > > > reboot.
>>> > > > >>>>>
>>> > > > >>>>> AFAIK, the Artery transport is not wire compatible with the
>>> > > > >>>>> classic one.
>>> > > > >>>>>
>>> > > > >>>>> I suggest:
>>> > > > >>>>> 1. Start to send PRs to open sources projects like Flink
>>> with the
>>> > > > help
>>> > > > >>>>> for migrating to the artery transport.
>>> > > > >>>>> 2. Upgrade to the new version of Netty for bugfix and
>>> > > > >>>>> better performance.
>>> > > > >>>>> 3. Suggest all the new community move to the Artery
>>> transport.
>>> > > > >>>>>
>>> > > > >>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
>>> > > > >>>>>
>>> > > > >>>>> Anyway, different company has different requirements, that
>>> just my
>>> > > 2
>>> > > > >>>>> cents.
>>> > > > >>>>>
>>> > > > >>>>> 何品
>>> > > > >>>>>
>>> > > > >>>>>
>>> > > > >>>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
>>> > > > >>>>>
>>> > > > >>>>>> This will end up in a vote.
>>> > > > >>>>>>
>>> > > > >>>>>> For me, we shipped 1.0.x. Users who don't want big changes
>>> can use
>>> > > > >>>>>> that version. We can agree to support 1.0.x with critical
>>> fixes.
>>> > > > >>>>>>
>>> > > > >>>>>> We can now start improving the code for the improvement
>>> release
>>> > > > >>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of
>>> deprecated
>>> > > > >>>>>> code and most of us appear to believe that classic remoting
>>> is
>>> > > long
>>> > > > >>>>>> deprecated.
>>> > > > >>>>>>
>>> > > > >>>>>>
>>> > > > >>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
>>> > > > >>>>>> <ma...@aiven.io.invalid> wrote:
>>> > > > >>>>>> >
>>> > > > >>>>>> > > Is it really worth spending any time to support classic
>>> > > > remoting?
>>> > > > >>>>>> Users
>>> > > > >>>>>> > would be better served to switch to Artery.
>>> > > > >>>>>> >
>>> > > > >>>>>> > That depends on what you mean by "support". If agreed on,
>>> I
>>> > > would
>>> > > > >>>>>> > propose marking classic remoting as "to be removed" which
>>> would
>>> > > > >>>>>> > also mean that the code wouldn't be touched (i.e. no
>>> feature
>>> > > > >>>>>> > changes and no bug fixes unless extremely critical).
>>> > > > >>>>>> >
>>> > > > >>>>>> > I am in no way suggesting that we support remoting in a
>>> > > > traditional
>>> > > > >>>>>> sense
>>> > > > >>>>>> > and again the only reason why I am even contemplating
>>> this is
>>> > > > >>>>>> because
>>> > > > >>>>>> > of the CVE's.
>>> > > > >>>>>> >
>>> > > > >>>>>> > > I don't think it is likely that a significant portion
>>> of Akka
>>> > > > 2.4
>>> > > > >>>>>> users
>>> > > > >>>>>> > would update to Pekko any time soon if the couldn't
>>> update to
>>> > > Akka
>>> > > > >>>>>> 2.5/2.6.
>>> > > > >>>>>> >
>>> > > > >>>>>> > I also had this impression but it's already come up a few
>>> times
>>> > > > that
>>> > > > >>>>>> > people still use Akka 2.4.x. They likely haven't bothered
>>> > > updating
>>> > > > >>>>>> > because they didn't see a need to (quite famously Akka is
>>> quite
>>> > > > >>>>>> stable
>>> > > > >>>>>> > and there are cases of companies using it for years
>>> without
>>> > > > needing
>>> > > > >>>>>> to
>>> > > > >>>>>> > update, as stated by Lightbend's CEO) but with the license
>>> > > change
>>> > > > >>>>>> > it created a catalyst/trigger for some users to consider
>>> moving
>>> > > to
>>> > > > >>>>>> Pekko.
>>> > > > >>>>>> >
>>> > > > >>>>>> > On this note I would also be wary in making assumptions
>>> on what
>>> > > > >>>>>> versions
>>> > > > >>>>>> > of Akka people happen to be using. As a corollary in the
>>> > > > discussion
>>> > > > >>>>>> on
>>> > > > >>>>>> > dropping JDK 8, when we suggested that we quite quickly
>>> got
>>> > > > >>>>>> feedback that
>>> > > > >>>>>> > there are people still using JDK 8. And while I know all
>>> of the
>>> > > > >>>>>> arguments
>>> > > > >>>>>> > that you really should not be using JDK 8 can be carried
>>> over to
>>> > > > >>>>>> Akka.
>>> > > > >>>>>> >
>>> > > > >>>>>> > Admittedly it's quite hard to get good feedback on what
>>> versions
>>> > > > >>>>>> people are
>>> > > > >>>>>> > using, just saying we should be a bit more careful in
>>> making
>>> > > these
>>> > > > >>>>>> > assumptions.
>>> > > > >>>>>> > After all, people are still using JDK 8 ;)
>>> > > > >>>>>> >
>>> > > > >>>>>> >
>>> > > > >>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <
>>> > > > nvollmar@gmail.com>
>>> > > > >>>>>> wrote:
>>> > > > >>>>>> >
>>> > > > >>>>>> > > Akka 2.4 has been EoL since end of 2017.
>>> > > > >>>>>> > > I don't think it is likely that a significant portion
>>> of Akka
>>> > > > 2.4
>>> > > > >>>>>> users
>>> > > > >>>>>> > > would update to Pekko any time soon if the couldn't
>>> update to
>>> > > > >>>>>> Akka 2.5/2.6.
>>> > > > >>>>>> > >
>>> > > > >>>>>> > > Is it really worth spending any time to support classic
>>> > > > remoting?
>>> > > > >>>>>> Users
>>> > > > >>>>>> > > would be better served to switch to Artery.
>>> > > > >>>>>> > >
>>> > > > >>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
>>> > > > >>>>>> > > <ma...@aiven.io.invalid> wrote:
>>> > > > >>>>>> > >
>>> > > > >>>>>> > > > > If you manage to move to Pekko and were still using
>>> > > classic
>>> > > > >>>>>> remoting,
>>> > > > >>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you
>>> manage
>>> > > > the
>>> > > > >>>>>> update
>>> > > > >>>>>> > > > to Pekko, going to Artery TCP is a small step.
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > One of the reasons why I was saying this is that I
>>> was under
>>> > > > the
>>> > > > >>>>>> > > impression
>>> > > > >>>>>> > > > that there are a non-trivial amount of users still on
>>> Akka
>>> > > 2.4
>>> > > > >>>>>> (not sure
>>> > > > >>>>>> > > > why
>>> > > > >>>>>> > > > this is the case but kerr was telling me this).
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > > If people are interested in keeping it around and
>>> also opt
>>> > > > in
>>> > > > >>>>>> to
>>> > > > >>>>>> > > > maintaining it (which in the first place means,
>>> making the
>>> > > > >>>>>> tests for
>>> > > > >>>>>> > > > it work *now*), than fine, but otherwise, being able
>>> to use
>>> > > it
>>> > > > >>>>>> on
>>> > > > >>>>>> > > > 1.0.x is already a big benefit.
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > Since classic remoting is currently using Netty 3
>>> which has
>>> > > > >>>>>> CVE's
>>> > > > >>>>>> > > > I don't think it's wise to encourage people that still
>>> > > happen
>>> > > > >>>>>> to be
>>> > > > >>>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't
>>> for
>>> > > > final
>>> > > > >>>>>> > > > version of netty 3 having CVE's then I would have no
>>> qualms
>>> > > > >>>>>> > > > for dropping it in the 1.1.x series (in fact if there
>>> was a
>>> > > > >>>>>> > > > hypothetical netty 3 without CVE's we would have
>>> already
>>> > > > >>>>>> > > > updated to it in the 1.0.x series and dropped classic
>>> > > remoting
>>> > > > >>>>>> in
>>> > > > >>>>>> > > > 1.1.x without a heartbeat).
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > I would propose that if we do accept keeping classic
>>> > > remoting
>>> > > > >>>>>> > > > and updating to Netty 4, we would mark the feature to
>>> be
>>> > > > >>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade
>>> that
>>> > > > >>>>>> > > > part of code would be untouched unless there are
>>> critical
>>> > > > >>>>>> > > > bugs/regressions.
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
>>> > > > >>>>>> > > > johannes.rudolph@gmail.com>
>>> > > > >>>>>> > > > wrote:
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > > We definitely should remove it for 1.1.x. There's no
>>> > > > >>>>>> technical reason
>>> > > > >>>>>> > > > > to keep it because the newer artery TCP transport
>>> just
>>> > > > >>>>>> supersedes it.
>>> > > > >>>>>> > > > > As discussed before, the problem with keeping old
>>> features
>>> > > > >>>>>> around is
>>> > > > >>>>>> > > > > "death by a thousand cuts". The classic remoting
>>> backend
>>> > > is
>>> > > > >>>>>> the prime
>>> > > > >>>>>> > > > > example of it because it is basically unmaintained
>>> for
>>> > > years
>>> > > > >>>>>> and
>>> > > > >>>>>> > > > > there's basically no expert knowledge available
>>> about how
>>> > > to
>>> > > > >>>>>> maintain
>>> > > > >>>>>> > > > > it or how to fix the existing bugs. Testing remoting
>>> > > > backends
>>> > > > >>>>>> has
>>> > > > >>>>>> > > > > shown to be very complicated and often unreliable
>>> and
>>> > > being
>>> > > > >>>>>> able to
>>> > > > >>>>>> > > > > remove this particular component from the test
>>> matrix will
>>> > > > be
>>> > > > >>>>>> a huge
>>> > > > >>>>>> > > > > improvement.
>>> > > > >>>>>> > > > >
>>> > > > >>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
>>> > > > >>>>>> > > > > <ma...@aiven.io.invalid> wrote:
>>> > > > >>>>>> > > > > > I would prefer adding in netty 4 support for the
>>> classic
>>> > > > >>>>>> transport
>>> > > > >>>>>> > > > > > mechanism because it gives
>>> > > > >>>>>> > > > > > people an upgrade path while allowing them to
>>> resolve
>>> > > the
>>> > > > >>>>>> CVE issues.
>>> > > > >>>>>> > > > If
>>> > > > >>>>>> > > > > > Pekko 1.1.x didn't
>>> > > > >>>>>> > > > > > support the classic transport mechanism then it
>>> forces
>>> > > > >>>>>> Pekko users to
>>> > > > >>>>>> > > > be
>>> > > > >>>>>> > > > > > stuck on 1.0.x
>>> > > > >>>>>> > > > > > with no easy upgrade path.
>>> > > > >>>>>> > > > >
>>> > > > >>>>>> > > > > We are not responsible for giving people an easy
>>> upgrade
>>> > > > path
>>> > > > >>>>>> when
>>> > > > >>>>>> > > > > they were stuck on features like classic remoting
>>> which
>>> > > has
>>> > > > >>>>>> basically
>>> > > > >>>>>> > > > > been outdated for years. It is very important that,
>>> going
>>> > > > >>>>>> forward,
>>> > > > >>>>>> > > > > that we focus our attention on the most important
>>> features
>>> > > > >>>>>> and make
>>> > > > >>>>>> > > > > sure to not get stuck in maintaining old parts that
>>> we
>>> > > > cannot
>>> > > > >>>>>> (or
>>> > > > >>>>>> > > > > don't want to) care for.
>>> > > > >>>>>> > > > >
>>> > > > >>>>>> > > > > If people are interested in keeping it around and
>>> also opt
>>> > > > in
>>> > > > >>>>>> to
>>> > > > >>>>>> > > > > maintaining it (which in the first place means,
>>> making the
>>> > > > >>>>>> tests for
>>> > > > >>>>>> > > > > it work *now*), than fine, but otherwise, being
>>> able to
>>> > > use
>>> > > > >>>>>> it on
>>> > > > >>>>>> > > > > 1.0.x is already a big benefit.
>>> > > > >>>>>> > > > >
>>> > > > >>>>>> > > > > Apart from that the update path is pretty clear:
>>> > > > >>>>>> > > > > * move to Pekko 1.0.x
>>> > > > >>>>>> > > > > * move to Arterty TCP
>>> > > > >>>>>> > > > > * move to Pekko 1.1.x
>>> > > > >>>>>> > > > >
>>> > > > >>>>>> > > > > If you manage to move to Pekko and were still using
>>> > > classic
>>> > > > >>>>>> remoting,
>>> > > > >>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you
>>> > > manage
>>> > > > >>>>>> the update
>>> > > > >>>>>> > > > > to Pekko, going to Artery TCP is a small step.
>>> > > > >>>>>> > > > >
>>> > > > >>>>>> > > > > Johannes
>>> > > > >>>>>> > > > >
>>> > > > >>>>>> > > > >
>>> > > > >>>>>>
>>> > > >
>>> ---------------------------------------------------------------------
>>> > > > >>>>>> > > > > To unsubscribe, e-mail:
>>> dev-unsubscribe@pekko.apache.org
>>> > > > >>>>>> > > > > For additional commands, e-mail:
>>> > > dev-help@pekko.apache.org
>>> > > > >>>>>> > > > >
>>> > > > >>>>>> > > > >
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > --
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > Matthew de Detrich
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > *Aiven Deutschland GmbH*
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > *m:* +491603708037
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>> > > > >>>>>> > > >
>>> > > > >>>>>> > >
>>> > > > >>>>>> >
>>> > > > >>>>>> >
>>> > > > >>>>>> > --
>>> > > > >>>>>> >
>>> > > > >>>>>> > Matthew de Detrich
>>> > > > >>>>>> >
>>> > > > >>>>>> > *Aiven Deutschland GmbH*
>>> > > > >>>>>> >
>>> > > > >>>>>> > Immanuelkirchstraße 26, 10405 Berlin
>>> > > > >>>>>> >
>>> > > > >>>>>> > Amtsgericht Charlottenburg, HRB 209739 B
>>> > > > >>>>>> >
>>> > > > >>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>> > > > >>>>>> >
>>> > > > >>>>>> > *m:* +491603708037
>>> > > > >>>>>> >
>>> > > > >>>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>> > > > >>>>>>
>>> > > > >>>>>>
>>> > > >
>>> ---------------------------------------------------------------------
>>> > > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>> > > > >>>>>> For additional commands, e-mail: dev-help@pekko.apache.org
>>> > > > >>>>>>
>>> > > > >>>>>>
>>> > > > >
>>> > > > > --
>>> > > > >
>>> > > > > Matthew de Detrich
>>> > > > >
>>> > > > > *Aiven Deutschland GmbH*
>>> > > > >
>>> > > > > Immanuelkirchstraße 26, 10405 Berlin
>>> > > > >
>>> > > > > Amtsgericht Charlottenburg, HRB 209739 B
>>> > > > >
>>> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>> > > > >
>>> > > > > *m:* +491603708037
>>> > > > >
>>> > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>> > > > >
>>> > > >
>>> > > >
>>> > > > --
>>> > > >
>>> > > > Matthew de Detrich
>>> > > >
>>> > > > *Aiven Deutschland GmbH*
>>> > > >
>>> > > > Immanuelkirchstraße 26, 10405 Berlin
>>> > > >
>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
>>> > > >
>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>> > > >
>>> > > > *m:* +491603708037
>>> > > >
>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>> > > >
>>> > >
>>> >
>>> >
>>> > --
>>> >
>>> > Matthew de Detrich
>>> >
>>> > *Aiven Deutschland GmbH*
>>> >
>>> > Immanuelkirchstraße 26, 10405 Berlin
>>> >
>>> > Amtsgericht Charlottenburg, HRB 209739 B
>>> >
>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>> >
>>> > *m:* +491603708037
>>> >
>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>> For additional commands, e-mail: dev-help@pekko.apache.org
>>>
>>>
>>
>> --
>>
>> Matthew de Detrich
>>
>> *Aiven Deutschland GmbH*
>>
>> Immanuelkirchstraße 26, 10405 Berlin
>>
>> Amtsgericht Charlottenburg, HRB 209739 B
>>
>> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>
>> *m:* +491603708037
>>
>> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>
--
Matthew de Detrich
*Aiven Deutschland GmbH*
Immanuelkirchstraße 26, 10405 Berlin
Amtsgericht Charlottenburg, HRB 209739 B
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
*m:* +491603708037
*w:* aiven.io *e:* matthew.dedetrich@aiven.io
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by Matthew de Detrich <ma...@aiven.io.INVALID>.
I created a thread on the Flink dev mailing list with a summary of the
situation at
https://lists.apache.org/thread/o774bkox2mvyl8r3xld6kkg5krstgrow
On Mon, Sep 11, 2023 at 4:43 PM Matthew de Detrich <
matthew.dedetrich@aiven.io> wrote:
> I am not sure there is any discussion on this, just the apparent fact
> that they happen to be currently using classic remoting
>
> On Mon, Sep 11, 2023 at 4:37 PM PJ Fanning <fa...@gmail.com> wrote:
>
>> Can we get a link to the Flink discussion?
>>
>> On Mon, 11 Sept 2023 at 15:25, Matthew de Detrich
>> <ma...@aiven.io.invalid> wrote:
>> >
>> > > Flink is using pekko now, and a hard reboot may not be ok in some
>> cases,
>> > and because Pekko ships a Netty3 with many CVEs, they may decide to drop
>> > pekko and come out with a dedicated rpc implementation based on Netty4
>> too.
>> >
>> > Wasn't aware of this (I thought Flink used the new remoting protocol).
>> > I think this is a strong enough reason by itself to upgrade Netty4, but
>> > ofcourse
>> > deprecated classic remoting.
>> >
>> > On Mon, Sep 11, 2023 at 3:22 PM kerr <he...@gmail.com> wrote:
>> >
>> > > Netty 4 is much better than Netty 3 and we can schedule tests to run
>> for it
>> > > and make sure the tests all passed for servertimes.
>> > > Otherwise, we will have a CVEs old netty 3 in classpath.
>> > >
>> > > Flink is using pekko now, and a hard reboot may not be ok in some
>> cases,
>> > > and because Pekko ships a Netty3 with many CVEs, they may decide to
>> drop
>> > > pekko and come out with a dedicated rpc implementation based on
>> Netty4 too.
>> > > But after we migrate to Netty 4, they can defer that, I will ask
>> around the
>> > > PMC of Flink at here to let them taking a look.
>> > >
>> > > 何品
>> > >
>> > >
>> > > Matthew de Detrich <ma...@aiven.io.invalid> 于2023年9月11日周一
>> > > 19:35写道：
>> > >
>> > > > > that this is a bug, not a feature
>> > > >
>> > > > Also meant to say that this is a feature, not a bug
>> > > >
>> > > > On Mon, Sep 11, 2023 at 1:34 PM Matthew de Detrich <
>> > > > matthew.dedetrich@aiven.io> wrote:
>> > > >
>> > > > > While I know that people have reservations for keeping the current
>> > > > > classical transport,
>> > > > > I just want to relay what I have written at
>> > > > > https://github.com/apache/incubator-pekko/pull/643
>> > > > > i.e.
>> > > > >
>> > > > > - Upgrading the classical transport from Netty 3 to Netty 4 was in
>> > > > context
>> > > > > quite trivial
>> > > > > - Dropping classical transport means that we are technically
>> breaking
>> > > > > semver (we can only do this in Pekko 2.0.x)
>> > > > > - Not upgrading to Netty3 means that we are shipping code with
>> known
>> > > > CVE's
>> > > > >
>> > > > > For these reasons I personally would prefer to merge the PR and
>> just
>> > > get
>> > > > > it done with. Even
>> > > > > with the PR merged, if we change our minds later nothing is
>> stopping us
>> > > > > from dropping it
>> > > > > later (or even reverting the PR when testing is done but I doubt
>> that
>> > > > will
>> > > > > occur).
>> > > > >
>> > > > > There is of course a risk in that updating to Netty 4 might
>> create some
>> > > > > regressions
>> > > > > and hence pekko users would complain but I would then argue that
>> this
>> > > is
>> > > > a
>> > > > > bug,
>> > > > > not a feature because we then get actual feedback as to how many
>> people
>> > > > > are using
>> > > > > the classical transport which allows us to make more comprehensive
>> > > > > decisions in
>> > > > > the future.
>> > > > >
>> > > > > On Fri, Sep 8, 2023 at 5:56 PM kerr <he...@gmail.com> wrote:
>> > > > >
>> > > > >> And I gathered this result on reddit
>> > > > >> [image: image.png]
>> > > > >> 何品
>> > > > >>
>> > > > >>
>> > > > >> kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
>> > > > >>
>> > > > >>> I just read
>> > > > >>>
>> > > > >>> To switch a full cluster restart is required and any overrides
>> for
>> > > > >>> classic remoting need to be ported to Artery configuration.
>> Artery
>> > > has
>> > > > a
>> > > > >>> completely different protocol, which means that a rolling
>> update is
>> > > not
>> > > > >>> supported.
>> > > > >>> So for Flink use case is that ok?
>> > > > >>> 何品
>> > > > >>>
>> > > > >>>
>> > > > >>> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
>> > > > >>>
>> > > > >>>> I did a poll on reddit and get the result of
>> > > > >>>> [image: image.png]
>> > > > >>>> 何品
>> > > > >>>>
>> > > > >>>>
>> > > > >>>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
>> > > > >>>>
>> > > > >>>>> > Is it really worth spending any time to support classic
>> remoting?
>> > > > >>>>> Users
>> > > > >>>>> would be better served to switch to Artery.
>> > > > >>>>>
>> > > > >>>>> I'm one of the user and the China Scala User group may not
>> talk
>> > > much
>> > > > >>>>> on web but most of us is using the netty transport, even
>> Lightbend
>> > > > say the
>> > > > >>>>> `Artery is the best`.
>> > > > >>>>>
>> > > > >>>>> Another online product with large scale nodes(here) is
>> running with
>> > > > >>>>> Akka 2.4.x with the classic transport, and Flink is using the
>> > > > netty.tpc
>> > > > >>>>> transport at least for now too.
>> > > > >>>>>
>> > > > >>>>> I think the community can choose when they migrate the nodes
>> to
>> > > > artery
>> > > > >>>>> is essential, especially when otherwise your cluster need a
>> hard
>> > > > reboot.
>> > > > >>>>>
>> > > > >>>>> AFAIK, the Artery transport is not wire compatible with the
>> > > > >>>>> classic one.
>> > > > >>>>>
>> > > > >>>>> I suggest:
>> > > > >>>>> 1. Start to send PRs to open sources projects like Flink with
>> the
>> > > > help
>> > > > >>>>> for migrating to the artery transport.
>> > > > >>>>> 2. Upgrade to the new version of Netty for bugfix and
>> > > > >>>>> better performance.
>> > > > >>>>> 3. Suggest all the new community move to the Artery transport.
>> > > > >>>>>
>> > > > >>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
>> > > > >>>>>
>> > > > >>>>> Anyway, different company has different requirements, that
>> just my
>> > > 2
>> > > > >>>>> cents.
>> > > > >>>>>
>> > > > >>>>> 何品
>> > > > >>>>>
>> > > > >>>>>
>> > > > >>>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
>> > > > >>>>>
>> > > > >>>>>> This will end up in a vote.
>> > > > >>>>>>
>> > > > >>>>>> For me, we shipped 1.0.x. Users who don't want big changes
>> can use
>> > > > >>>>>> that version. We can agree to support 1.0.x with critical
>> fixes.
>> > > > >>>>>>
>> > > > >>>>>> We can now start improving the code for the improvement
>> release
>> > > > >>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of
>> deprecated
>> > > > >>>>>> code and most of us appear to believe that classic remoting
>> is
>> > > long
>> > > > >>>>>> deprecated.
>> > > > >>>>>>
>> > > > >>>>>>
>> > > > >>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
>> > > > >>>>>> <ma...@aiven.io.invalid> wrote:
>> > > > >>>>>> >
>> > > > >>>>>> > > Is it really worth spending any time to support classic
>> > > > remoting?
>> > > > >>>>>> Users
>> > > > >>>>>> > would be better served to switch to Artery.
>> > > > >>>>>> >
>> > > > >>>>>> > That depends on what you mean by "support". If agreed on, I
>> > > would
>> > > > >>>>>> > propose marking classic remoting as "to be removed" which
>> would
>> > > > >>>>>> > also mean that the code wouldn't be touched (i.e. no
>> feature
>> > > > >>>>>> > changes and no bug fixes unless extremely critical).
>> > > > >>>>>> >
>> > > > >>>>>> > I am in no way suggesting that we support remoting in a
>> > > > traditional
>> > > > >>>>>> sense
>> > > > >>>>>> > and again the only reason why I am even contemplating this
>> is
>> > > > >>>>>> because
>> > > > >>>>>> > of the CVE's.
>> > > > >>>>>> >
>> > > > >>>>>> > > I don't think it is likely that a significant portion of
>> Akka
>> > > > 2.4
>> > > > >>>>>> users
>> > > > >>>>>> > would update to Pekko any time soon if the couldn't update
>> to
>> > > Akka
>> > > > >>>>>> 2.5/2.6.
>> > > > >>>>>> >
>> > > > >>>>>> > I also had this impression but it's already come up a few
>> times
>> > > > that
>> > > > >>>>>> > people still use Akka 2.4.x. They likely haven't bothered
>> > > updating
>> > > > >>>>>> > because they didn't see a need to (quite famously Akka is
>> quite
>> > > > >>>>>> stable
>> > > > >>>>>> > and there are cases of companies using it for years without
>> > > > needing
>> > > > >>>>>> to
>> > > > >>>>>> > update, as stated by Lightbend's CEO) but with the license
>> > > change
>> > > > >>>>>> > it created a catalyst/trigger for some users to consider
>> moving
>> > > to
>> > > > >>>>>> Pekko.
>> > > > >>>>>> >
>> > > > >>>>>> > On this note I would also be wary in making assumptions on
>> what
>> > > > >>>>>> versions
>> > > > >>>>>> > of Akka people happen to be using. As a corollary in the
>> > > > discussion
>> > > > >>>>>> on
>> > > > >>>>>> > dropping JDK 8, when we suggested that we quite quickly got
>> > > > >>>>>> feedback that
>> > > > >>>>>> > there are people still using JDK 8. And while I know all
>> of the
>> > > > >>>>>> arguments
>> > > > >>>>>> > that you really should not be using JDK 8 can be carried
>> over to
>> > > > >>>>>> Akka.
>> > > > >>>>>> >
>> > > > >>>>>> > Admittedly it's quite hard to get good feedback on what
>> versions
>> > > > >>>>>> people are
>> > > > >>>>>> > using, just saying we should be a bit more careful in
>> making
>> > > these
>> > > > >>>>>> > assumptions.
>> > > > >>>>>> > After all, people are still using JDK 8 ;)
>> > > > >>>>>> >
>> > > > >>>>>> >
>> > > > >>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <
>> > > > nvollmar@gmail.com>
>> > > > >>>>>> wrote:
>> > > > >>>>>> >
>> > > > >>>>>> > > Akka 2.4 has been EoL since end of 2017.
>> > > > >>>>>> > > I don't think it is likely that a significant portion of
>> Akka
>> > > > 2.4
>> > > > >>>>>> users
>> > > > >>>>>> > > would update to Pekko any time soon if the couldn't
>> update to
>> > > > >>>>>> Akka 2.5/2.6.
>> > > > >>>>>> > >
>> > > > >>>>>> > > Is it really worth spending any time to support classic
>> > > > remoting?
>> > > > >>>>>> Users
>> > > > >>>>>> > > would be better served to switch to Artery.
>> > > > >>>>>> > >
>> > > > >>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
>> > > > >>>>>> > > <ma...@aiven.io.invalid> wrote:
>> > > > >>>>>> > >
>> > > > >>>>>> > > > > If you manage to move to Pekko and were still using
>> > > classic
>> > > > >>>>>> remoting,
>> > > > >>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you
>> manage
>> > > > the
>> > > > >>>>>> update
>> > > > >>>>>> > > > to Pekko, going to Artery TCP is a small step.
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > One of the reasons why I was saying this is that I was
>> under
>> > > > the
>> > > > >>>>>> > > impression
>> > > > >>>>>> > > > that there are a non-trivial amount of users still on
>> Akka
>> > > 2.4
>> > > > >>>>>> (not sure
>> > > > >>>>>> > > > why
>> > > > >>>>>> > > > this is the case but kerr was telling me this).
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > > If people are interested in keeping it around and
>> also opt
>> > > > in
>> > > > >>>>>> to
>> > > > >>>>>> > > > maintaining it (which in the first place means, making
>> the
>> > > > >>>>>> tests for
>> > > > >>>>>> > > > it work *now*), than fine, but otherwise, being able
>> to use
>> > > it
>> > > > >>>>>> on
>> > > > >>>>>> > > > 1.0.x is already a big benefit.
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > Since classic remoting is currently using Netty 3
>> which has
>> > > > >>>>>> CVE's
>> > > > >>>>>> > > > I don't think it's wise to encourage people that still
>> > > happen
>> > > > >>>>>> to be
>> > > > >>>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't
>> for
>> > > > final
>> > > > >>>>>> > > > version of netty 3 having CVE's then I would have no
>> qualms
>> > > > >>>>>> > > > for dropping it in the 1.1.x series (in fact if there
>> was a
>> > > > >>>>>> > > > hypothetical netty 3 without CVE's we would have
>> already
>> > > > >>>>>> > > > updated to it in the 1.0.x series and dropped classic
>> > > remoting
>> > > > >>>>>> in
>> > > > >>>>>> > > > 1.1.x without a heartbeat).
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > I would propose that if we do accept keeping classic
>> > > remoting
>> > > > >>>>>> > > > and updating to Netty 4, we would mark the feature to
>> be
>> > > > >>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade
>> that
>> > > > >>>>>> > > > part of code would be untouched unless there are
>> critical
>> > > > >>>>>> > > > bugs/regressions.
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
>> > > > >>>>>> > > > johannes.rudolph@gmail.com>
>> > > > >>>>>> > > > wrote:
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > > We definitely should remove it for 1.1.x. There's no
>> > > > >>>>>> technical reason
>> > > > >>>>>> > > > > to keep it because the newer artery TCP transport
>> just
>> > > > >>>>>> supersedes it.
>> > > > >>>>>> > > > > As discussed before, the problem with keeping old
>> features
>> > > > >>>>>> around is
>> > > > >>>>>> > > > > "death by a thousand cuts". The classic remoting
>> backend
>> > > is
>> > > > >>>>>> the prime
>> > > > >>>>>> > > > > example of it because it is basically unmaintained
>> for
>> > > years
>> > > > >>>>>> and
>> > > > >>>>>> > > > > there's basically no expert knowledge available
>> about how
>> > > to
>> > > > >>>>>> maintain
>> > > > >>>>>> > > > > it or how to fix the existing bugs. Testing remoting
>> > > > backends
>> > > > >>>>>> has
>> > > > >>>>>> > > > > shown to be very complicated and often unreliable and
>> > > being
>> > > > >>>>>> able to
>> > > > >>>>>> > > > > remove this particular component from the test
>> matrix will
>> > > > be
>> > > > >>>>>> a huge
>> > > > >>>>>> > > > > improvement.
>> > > > >>>>>> > > > >
>> > > > >>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
>> > > > >>>>>> > > > > <ma...@aiven.io.invalid> wrote:
>> > > > >>>>>> > > > > > I would prefer adding in netty 4 support for the
>> classic
>> > > > >>>>>> transport
>> > > > >>>>>> > > > > > mechanism because it gives
>> > > > >>>>>> > > > > > people an upgrade path while allowing them to
>> resolve
>> > > the
>> > > > >>>>>> CVE issues.
>> > > > >>>>>> > > > If
>> > > > >>>>>> > > > > > Pekko 1.1.x didn't
>> > > > >>>>>> > > > > > support the classic transport mechanism then it
>> forces
>> > > > >>>>>> Pekko users to
>> > > > >>>>>> > > > be
>> > > > >>>>>> > > > > > stuck on 1.0.x
>> > > > >>>>>> > > > > > with no easy upgrade path.
>> > > > >>>>>> > > > >
>> > > > >>>>>> > > > > We are not responsible for giving people an easy
>> upgrade
>> > > > path
>> > > > >>>>>> when
>> > > > >>>>>> > > > > they were stuck on features like classic remoting
>> which
>> > > has
>> > > > >>>>>> basically
>> > > > >>>>>> > > > > been outdated for years. It is very important that,
>> going
>> > > > >>>>>> forward,
>> > > > >>>>>> > > > > that we focus our attention on the most important
>> features
>> > > > >>>>>> and make
>> > > > >>>>>> > > > > sure to not get stuck in maintaining old parts that
>> we
>> > > > cannot
>> > > > >>>>>> (or
>> > > > >>>>>> > > > > don't want to) care for.
>> > > > >>>>>> > > > >
>> > > > >>>>>> > > > > If people are interested in keeping it around and
>> also opt
>> > > > in
>> > > > >>>>>> to
>> > > > >>>>>> > > > > maintaining it (which in the first place means,
>> making the
>> > > > >>>>>> tests for
>> > > > >>>>>> > > > > it work *now*), than fine, but otherwise, being able
>> to
>> > > use
>> > > > >>>>>> it on
>> > > > >>>>>> > > > > 1.0.x is already a big benefit.
>> > > > >>>>>> > > > >
>> > > > >>>>>> > > > > Apart from that the update path is pretty clear:
>> > > > >>>>>> > > > > * move to Pekko 1.0.x
>> > > > >>>>>> > > > > * move to Arterty TCP
>> > > > >>>>>> > > > > * move to Pekko 1.1.x
>> > > > >>>>>> > > > >
>> > > > >>>>>> > > > > If you manage to move to Pekko and were still using
>> > > classic
>> > > > >>>>>> remoting,
>> > > > >>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you
>> > > manage
>> > > > >>>>>> the update
>> > > > >>>>>> > > > > to Pekko, going to Artery TCP is a small step.
>> > > > >>>>>> > > > >
>> > > > >>>>>> > > > > Johannes
>> > > > >>>>>> > > > >
>> > > > >>>>>> > > > >
>> > > > >>>>>>
>> > > >
>> ---------------------------------------------------------------------
>> > > > >>>>>> > > > > To unsubscribe, e-mail:
>> dev-unsubscribe@pekko.apache.org
>> > > > >>>>>> > > > > For additional commands, e-mail:
>> > > dev-help@pekko.apache.org
>> > > > >>>>>> > > > >
>> > > > >>>>>> > > > >
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > --
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > Matthew de Detrich
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > *Aiven Deutschland GmbH*
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > *m:* +491603708037
>> > > > >>>>>> > > >
>> > > > >>>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>> > > > >>>>>> > > >
>> > > > >>>>>> > >
>> > > > >>>>>> >
>> > > > >>>>>> >
>> > > > >>>>>> > --
>> > > > >>>>>> >
>> > > > >>>>>> > Matthew de Detrich
>> > > > >>>>>> >
>> > > > >>>>>> > *Aiven Deutschland GmbH*
>> > > > >>>>>> >
>> > > > >>>>>> > Immanuelkirchstraße 26, 10405 Berlin
>> > > > >>>>>> >
>> > > > >>>>>> > Amtsgericht Charlottenburg, HRB 209739 B
>> > > > >>>>>> >
>> > > > >>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>> > > > >>>>>> >
>> > > > >>>>>> > *m:* +491603708037
>> > > > >>>>>> >
>> > > > >>>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>> > > > >>>>>>
>> > > > >>>>>>
>> > > >
>> ---------------------------------------------------------------------
>> > > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>> > > > >>>>>> For additional commands, e-mail: dev-help@pekko.apache.org
>> > > > >>>>>>
>> > > > >>>>>>
>> > > > >
>> > > > > --
>> > > > >
>> > > > > Matthew de Detrich
>> > > > >
>> > > > > *Aiven Deutschland GmbH*
>> > > > >
>> > > > > Immanuelkirchstraße 26, 10405 Berlin
>> > > > >
>> > > > > Amtsgericht Charlottenburg, HRB 209739 B
>> > > > >
>> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>> > > > >
>> > > > > *m:* +491603708037
>> > > > >
>> > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > >
>> > > > Matthew de Detrich
>> > > >
>> > > > *Aiven Deutschland GmbH*
>> > > >
>> > > > Immanuelkirchstraße 26, 10405 Berlin
>> > > >
>> > > > Amtsgericht Charlottenburg, HRB 209739 B
>> > > >
>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>> > > >
>> > > > *m:* +491603708037
>> > > >
>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>> > > >
>> > >
>> >
>> >
>> > --
>> >
>> > Matthew de Detrich
>> >
>> > *Aiven Deutschland GmbH*
>> >
>> > Immanuelkirchstraße 26, 10405 Berlin
>> >
>> > Amtsgericht Charlottenburg, HRB 209739 B
>> >
>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>> >
>> > *m:* +491603708037
>> >
>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>> For additional commands, e-mail: dev-help@pekko.apache.org
>>
>>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>
--
Matthew de Detrich
*Aiven Deutschland GmbH*
Immanuelkirchstraße 26, 10405 Berlin
Amtsgericht Charlottenburg, HRB 209739 B
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
*m:* +491603708037
*w:* aiven.io *e:* matthew.dedetrich@aiven.io
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by Matthew de Detrich <ma...@aiven.io.INVALID>.
I am not sure there is any discussion on this, just the apparent fact
that they happen to be currently using classic remoting
On Mon, Sep 11, 2023 at 4:37 PM PJ Fanning <fa...@gmail.com> wrote:
> Can we get a link to the Flink discussion?
>
> On Mon, 11 Sept 2023 at 15:25, Matthew de Detrich
> <ma...@aiven.io.invalid> wrote:
> >
> > > Flink is using pekko now, and a hard reboot may not be ok in some
> cases,
> > and because Pekko ships a Netty3 with many CVEs, they may decide to drop
> > pekko and come out with a dedicated rpc implementation based on Netty4
> too.
> >
> > Wasn't aware of this (I thought Flink used the new remoting protocol).
> > I think this is a strong enough reason by itself to upgrade Netty4, but
> > ofcourse
> > deprecated classic remoting.
> >
> > On Mon, Sep 11, 2023 at 3:22 PM kerr <he...@gmail.com> wrote:
> >
> > > Netty 4 is much better than Netty 3 and we can schedule tests to run
> for it
> > > and make sure the tests all passed for servertimes.
> > > Otherwise, we will have a CVEs old netty 3 in classpath.
> > >
> > > Flink is using pekko now, and a hard reboot may not be ok in some
> cases,
> > > and because Pekko ships a Netty3 with many CVEs, they may decide to
> drop
> > > pekko and come out with a dedicated rpc implementation based on Netty4
> too.
> > > But after we migrate to Netty 4, they can defer that, I will ask
> around the
> > > PMC of Flink at here to let them taking a look.
> > >
> > > 何品
> > >
> > >
> > > Matthew de Detrich <ma...@aiven.io.invalid> 于2023年9月11日周一
> > > 19:35写道：
> > >
> > > > > that this is a bug, not a feature
> > > >
> > > > Also meant to say that this is a feature, not a bug
> > > >
> > > > On Mon, Sep 11, 2023 at 1:34 PM Matthew de Detrich <
> > > > matthew.dedetrich@aiven.io> wrote:
> > > >
> > > > > While I know that people have reservations for keeping the current
> > > > > classical transport,
> > > > > I just want to relay what I have written at
> > > > > https://github.com/apache/incubator-pekko/pull/643
> > > > > i.e.
> > > > >
> > > > > - Upgrading the classical transport from Netty 3 to Netty 4 was in
> > > > context
> > > > > quite trivial
> > > > > - Dropping classical transport means that we are technically
> breaking
> > > > > semver (we can only do this in Pekko 2.0.x)
> > > > > - Not upgrading to Netty3 means that we are shipping code with
> known
> > > > CVE's
> > > > >
> > > > > For these reasons I personally would prefer to merge the PR and
> just
> > > get
> > > > > it done with. Even
> > > > > with the PR merged, if we change our minds later nothing is
> stopping us
> > > > > from dropping it
> > > > > later (or even reverting the PR when testing is done but I doubt
> that
> > > > will
> > > > > occur).
> > > > >
> > > > > There is of course a risk in that updating to Netty 4 might create
> some
> > > > > regressions
> > > > > and hence pekko users would complain but I would then argue that
> this
> > > is
> > > > a
> > > > > bug,
> > > > > not a feature because we then get actual feedback as to how many
> people
> > > > > are using
> > > > > the classical transport which allows us to make more comprehensive
> > > > > decisions in
> > > > > the future.
> > > > >
> > > > > On Fri, Sep 8, 2023 at 5:56 PM kerr <he...@gmail.com> wrote:
> > > > >
> > > > >> And I gathered this result on reddit
> > > > >> [image: image.png]
> > > > >> 何品
> > > > >>
> > > > >>
> > > > >> kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
> > > > >>
> > > > >>> I just read
> > > > >>>
> > > > >>> To switch a full cluster restart is required and any overrides
> for
> > > > >>> classic remoting need to be ported to Artery configuration.
> Artery
> > > has
> > > > a
> > > > >>> completely different protocol, which means that a rolling update
> is
> > > not
> > > > >>> supported.
> > > > >>> So for Flink use case is that ok?
> > > > >>> 何品
> > > > >>>
> > > > >>>
> > > > >>> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
> > > > >>>
> > > > >>>> I did a poll on reddit and get the result of
> > > > >>>> [image: image.png]
> > > > >>>> 何品
> > > > >>>>
> > > > >>>>
> > > > >>>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
> > > > >>>>
> > > > >>>>> > Is it really worth spending any time to support classic
> remoting?
> > > > >>>>> Users
> > > > >>>>> would be better served to switch to Artery.
> > > > >>>>>
> > > > >>>>> I'm one of the user and the China Scala User group may not talk
> > > much
> > > > >>>>> on web but most of us is using the netty transport, even
> Lightbend
> > > > say the
> > > > >>>>> `Artery is the best`.
> > > > >>>>>
> > > > >>>>> Another online product with large scale nodes(here) is running
> with
> > > > >>>>> Akka 2.4.x with the classic transport, and Flink is using the
> > > > netty.tpc
> > > > >>>>> transport at least for now too.
> > > > >>>>>
> > > > >>>>> I think the community can choose when they migrate the nodes to
> > > > artery
> > > > >>>>> is essential, especially when otherwise your cluster need a
> hard
> > > > reboot.
> > > > >>>>>
> > > > >>>>> AFAIK, the Artery transport is not wire compatible with the
> > > > >>>>> classic one.
> > > > >>>>>
> > > > >>>>> I suggest:
> > > > >>>>> 1. Start to send PRs to open sources projects like Flink with
> the
> > > > help
> > > > >>>>> for migrating to the artery transport.
> > > > >>>>> 2. Upgrade to the new version of Netty for bugfix and
> > > > >>>>> better performance.
> > > > >>>>> 3. Suggest all the new community move to the Artery transport.
> > > > >>>>>
> > > > >>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
> > > > >>>>>
> > > > >>>>> Anyway, different company has different requirements, that
> just my
> > > 2
> > > > >>>>> cents.
> > > > >>>>>
> > > > >>>>> 何品
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
> > > > >>>>>
> > > > >>>>>> This will end up in a vote.
> > > > >>>>>>
> > > > >>>>>> For me, we shipped 1.0.x. Users who don't want big changes
> can use
> > > > >>>>>> that version. We can agree to support 1.0.x with critical
> fixes.
> > > > >>>>>>
> > > > >>>>>> We can now start improving the code for the improvement
> release
> > > > >>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of
> deprecated
> > > > >>>>>> code and most of us appear to believe that classic remoting is
> > > long
> > > > >>>>>> deprecated.
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
> > > > >>>>>> <ma...@aiven.io.invalid> wrote:
> > > > >>>>>> >
> > > > >>>>>> > > Is it really worth spending any time to support classic
> > > > remoting?
> > > > >>>>>> Users
> > > > >>>>>> > would be better served to switch to Artery.
> > > > >>>>>> >
> > > > >>>>>> > That depends on what you mean by "support". If agreed on, I
> > > would
> > > > >>>>>> > propose marking classic remoting as "to be removed" which
> would
> > > > >>>>>> > also mean that the code wouldn't be touched (i.e. no feature
> > > > >>>>>> > changes and no bug fixes unless extremely critical).
> > > > >>>>>> >
> > > > >>>>>> > I am in no way suggesting that we support remoting in a
> > > > traditional
> > > > >>>>>> sense
> > > > >>>>>> > and again the only reason why I am even contemplating this
> is
> > > > >>>>>> because
> > > > >>>>>> > of the CVE's.
> > > > >>>>>> >
> > > > >>>>>> > > I don't think it is likely that a significant portion of
> Akka
> > > > 2.4
> > > > >>>>>> users
> > > > >>>>>> > would update to Pekko any time soon if the couldn't update
> to
> > > Akka
> > > > >>>>>> 2.5/2.6.
> > > > >>>>>> >
> > > > >>>>>> > I also had this impression but it's already come up a few
> times
> > > > that
> > > > >>>>>> > people still use Akka 2.4.x. They likely haven't bothered
> > > updating
> > > > >>>>>> > because they didn't see a need to (quite famously Akka is
> quite
> > > > >>>>>> stable
> > > > >>>>>> > and there are cases of companies using it for years without
> > > > needing
> > > > >>>>>> to
> > > > >>>>>> > update, as stated by Lightbend's CEO) but with the license
> > > change
> > > > >>>>>> > it created a catalyst/trigger for some users to consider
> moving
> > > to
> > > > >>>>>> Pekko.
> > > > >>>>>> >
> > > > >>>>>> > On this note I would also be wary in making assumptions on
> what
> > > > >>>>>> versions
> > > > >>>>>> > of Akka people happen to be using. As a corollary in the
> > > > discussion
> > > > >>>>>> on
> > > > >>>>>> > dropping JDK 8, when we suggested that we quite quickly got
> > > > >>>>>> feedback that
> > > > >>>>>> > there are people still using JDK 8. And while I know all of
> the
> > > > >>>>>> arguments
> > > > >>>>>> > that you really should not be using JDK 8 can be carried
> over to
> > > > >>>>>> Akka.
> > > > >>>>>> >
> > > > >>>>>> > Admittedly it's quite hard to get good feedback on what
> versions
> > > > >>>>>> people are
> > > > >>>>>> > using, just saying we should be a bit more careful in making
> > > these
> > > > >>>>>> > assumptions.
> > > > >>>>>> > After all, people are still using JDK 8 ;)
> > > > >>>>>> >
> > > > >>>>>> >
> > > > >>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <
> > > > nvollmar@gmail.com>
> > > > >>>>>> wrote:
> > > > >>>>>> >
> > > > >>>>>> > > Akka 2.4 has been EoL since end of 2017.
> > > > >>>>>> > > I don't think it is likely that a significant portion of
> Akka
> > > > 2.4
> > > > >>>>>> users
> > > > >>>>>> > > would update to Pekko any time soon if the couldn't
> update to
> > > > >>>>>> Akka 2.5/2.6.
> > > > >>>>>> > >
> > > > >>>>>> > > Is it really worth spending any time to support classic
> > > > remoting?
> > > > >>>>>> Users
> > > > >>>>>> > > would be better served to switch to Artery.
> > > > >>>>>> > >
> > > > >>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
> > > > >>>>>> > > <ma...@aiven.io.invalid> wrote:
> > > > >>>>>> > >
> > > > >>>>>> > > > > If you manage to move to Pekko and were still using
> > > classic
> > > > >>>>>> remoting,
> > > > >>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you
> manage
> > > > the
> > > > >>>>>> update
> > > > >>>>>> > > > to Pekko, going to Artery TCP is a small step.
> > > > >>>>>> > > >
> > > > >>>>>> > > > One of the reasons why I was saying this is that I was
> under
> > > > the
> > > > >>>>>> > > impression
> > > > >>>>>> > > > that there are a non-trivial amount of users still on
> Akka
> > > 2.4
> > > > >>>>>> (not sure
> > > > >>>>>> > > > why
> > > > >>>>>> > > > this is the case but kerr was telling me this).
> > > > >>>>>> > > >
> > > > >>>>>> > > > > If people are interested in keeping it around and
> also opt
> > > > in
> > > > >>>>>> to
> > > > >>>>>> > > > maintaining it (which in the first place means, making
> the
> > > > >>>>>> tests for
> > > > >>>>>> > > > it work *now*), than fine, but otherwise, being able to
> use
> > > it
> > > > >>>>>> on
> > > > >>>>>> > > > 1.0.x is already a big benefit.
> > > > >>>>>> > > >
> > > > >>>>>> > > > Since classic remoting is currently using Netty 3 which
> has
> > > > >>>>>> CVE's
> > > > >>>>>> > > > I don't think it's wise to encourage people that still
> > > happen
> > > > >>>>>> to be
> > > > >>>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't
> for
> > > > final
> > > > >>>>>> > > > version of netty 3 having CVE's then I would have no
> qualms
> > > > >>>>>> > > > for dropping it in the 1.1.x series (in fact if there
> was a
> > > > >>>>>> > > > hypothetical netty 3 without CVE's we would have already
> > > > >>>>>> > > > updated to it in the 1.0.x series and dropped classic
> > > remoting
> > > > >>>>>> in
> > > > >>>>>> > > > 1.1.x without a heartbeat).
> > > > >>>>>> > > >
> > > > >>>>>> > > > I would propose that if we do accept keeping classic
> > > remoting
> > > > >>>>>> > > > and updating to Netty 4, we would mark the feature to be
> > > > >>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade
> that
> > > > >>>>>> > > > part of code would be untouched unless there are
> critical
> > > > >>>>>> > > > bugs/regressions.
> > > > >>>>>> > > >
> > > > >>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
> > > > >>>>>> > > > johannes.rudolph@gmail.com>
> > > > >>>>>> > > > wrote:
> > > > >>>>>> > > >
> > > > >>>>>> > > > > We definitely should remove it for 1.1.x. There's no
> > > > >>>>>> technical reason
> > > > >>>>>> > > > > to keep it because the newer artery TCP transport just
> > > > >>>>>> supersedes it.
> > > > >>>>>> > > > > As discussed before, the problem with keeping old
> features
> > > > >>>>>> around is
> > > > >>>>>> > > > > "death by a thousand cuts". The classic remoting
> backend
> > > is
> > > > >>>>>> the prime
> > > > >>>>>> > > > > example of it because it is basically unmaintained for
> > > years
> > > > >>>>>> and
> > > > >>>>>> > > > > there's basically no expert knowledge available about
> how
> > > to
> > > > >>>>>> maintain
> > > > >>>>>> > > > > it or how to fix the existing bugs. Testing remoting
> > > > backends
> > > > >>>>>> has
> > > > >>>>>> > > > > shown to be very complicated and often unreliable and
> > > being
> > > > >>>>>> able to
> > > > >>>>>> > > > > remove this particular component from the test matrix
> will
> > > > be
> > > > >>>>>> a huge
> > > > >>>>>> > > > > improvement.
> > > > >>>>>> > > > >
> > > > >>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
> > > > >>>>>> > > > > <ma...@aiven.io.invalid> wrote:
> > > > >>>>>> > > > > > I would prefer adding in netty 4 support for the
> classic
> > > > >>>>>> transport
> > > > >>>>>> > > > > > mechanism because it gives
> > > > >>>>>> > > > > > people an upgrade path while allowing them to
> resolve
> > > the
> > > > >>>>>> CVE issues.
> > > > >>>>>> > > > If
> > > > >>>>>> > > > > > Pekko 1.1.x didn't
> > > > >>>>>> > > > > > support the classic transport mechanism then it
> forces
> > > > >>>>>> Pekko users to
> > > > >>>>>> > > > be
> > > > >>>>>> > > > > > stuck on 1.0.x
> > > > >>>>>> > > > > > with no easy upgrade path.
> > > > >>>>>> > > > >
> > > > >>>>>> > > > > We are not responsible for giving people an easy
> upgrade
> > > > path
> > > > >>>>>> when
> > > > >>>>>> > > > > they were stuck on features like classic remoting
> which
> > > has
> > > > >>>>>> basically
> > > > >>>>>> > > > > been outdated for years. It is very important that,
> going
> > > > >>>>>> forward,
> > > > >>>>>> > > > > that we focus our attention on the most important
> features
> > > > >>>>>> and make
> > > > >>>>>> > > > > sure to not get stuck in maintaining old parts that we
> > > > cannot
> > > > >>>>>> (or
> > > > >>>>>> > > > > don't want to) care for.
> > > > >>>>>> > > > >
> > > > >>>>>> > > > > If people are interested in keeping it around and
> also opt
> > > > in
> > > > >>>>>> to
> > > > >>>>>> > > > > maintaining it (which in the first place means,
> making the
> > > > >>>>>> tests for
> > > > >>>>>> > > > > it work *now*), than fine, but otherwise, being able
> to
> > > use
> > > > >>>>>> it on
> > > > >>>>>> > > > > 1.0.x is already a big benefit.
> > > > >>>>>> > > > >
> > > > >>>>>> > > > > Apart from that the update path is pretty clear:
> > > > >>>>>> > > > > * move to Pekko 1.0.x
> > > > >>>>>> > > > > * move to Arterty TCP
> > > > >>>>>> > > > > * move to Pekko 1.1.x
> > > > >>>>>> > > > >
> > > > >>>>>> > > > > If you manage to move to Pekko and were still using
> > > classic
> > > > >>>>>> remoting,
> > > > >>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you
> > > manage
> > > > >>>>>> the update
> > > > >>>>>> > > > > to Pekko, going to Artery TCP is a small step.
> > > > >>>>>> > > > >
> > > > >>>>>> > > > > Johannes
> > > > >>>>>> > > > >
> > > > >>>>>> > > > >
> > > > >>>>>>
> > > > ---------------------------------------------------------------------
> > > > >>>>>> > > > > To unsubscribe, e-mail:
> dev-unsubscribe@pekko.apache.org
> > > > >>>>>> > > > > For additional commands, e-mail:
> > > dev-help@pekko.apache.org
> > > > >>>>>> > > > >
> > > > >>>>>> > > > >
> > > > >>>>>> > > >
> > > > >>>>>> > > > --
> > > > >>>>>> > > >
> > > > >>>>>> > > > Matthew de Detrich
> > > > >>>>>> > > >
> > > > >>>>>> > > > *Aiven Deutschland GmbH*
> > > > >>>>>> > > >
> > > > >>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > >>>>>> > > >
> > > > >>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > >>>>>> > > >
> > > > >>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > >>>>>> > > >
> > > > >>>>>> > > > *m:* +491603708037
> > > > >>>>>> > > >
> > > > >>>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > >>>>>> > > >
> > > > >>>>>> > >
> > > > >>>>>> >
> > > > >>>>>> >
> > > > >>>>>> > --
> > > > >>>>>> >
> > > > >>>>>> > Matthew de Detrich
> > > > >>>>>> >
> > > > >>>>>> > *Aiven Deutschland GmbH*
> > > > >>>>>> >
> > > > >>>>>> > Immanuelkirchstraße 26, 10405 Berlin
> > > > >>>>>> >
> > > > >>>>>> > Amtsgericht Charlottenburg, HRB 209739 B
> > > > >>>>>> >
> > > > >>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > >>>>>> >
> > > > >>>>>> > *m:* +491603708037
> > > > >>>>>> >
> > > > >>>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > >>>>>>
> > > > >>>>>>
> > > > ---------------------------------------------------------------------
> > > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > >>>>>> For additional commands, e-mail: dev-help@pekko.apache.org
> > > > >>>>>>
> > > > >>>>>>
> > > > >
> > > > > --
> > > > >
> > > > > Matthew de Detrich
> > > > >
> > > > > *Aiven Deutschland GmbH*
> > > > >
> > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > >
> > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > >
> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > >
> > > > > *m:* +491603708037
> > > > >
> > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Matthew de Detrich
> > > >
> > > > *Aiven Deutschland GmbH*
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > *m:* +491603708037
> > > >
> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >
> > >
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>
>
--
Matthew de Detrich
*Aiven Deutschland GmbH*
Immanuelkirchstraße 26, 10405 Berlin
Amtsgericht Charlottenburg, HRB 209739 B
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
*m:* +491603708037
*w:* aiven.io *e:* matthew.dedetrich@aiven.io
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by PJ Fanning <fa...@gmail.com>.
Can we get a link to the Flink discussion?
On Mon, 11 Sept 2023 at 15:25, Matthew de Detrich
<ma...@aiven.io.invalid> wrote:
>
> > Flink is using pekko now, and a hard reboot may not be ok in some cases,
> and because Pekko ships a Netty3 with many CVEs, they may decide to drop
> pekko and come out with a dedicated rpc implementation based on Netty4 too.
>
> Wasn't aware of this (I thought Flink used the new remoting protocol).
> I think this is a strong enough reason by itself to upgrade Netty4, but
> ofcourse
> deprecated classic remoting.
>
> On Mon, Sep 11, 2023 at 3:22 PM kerr <he...@gmail.com> wrote:
>
> > Netty 4 is much better than Netty 3 and we can schedule tests to run for it
> > and make sure the tests all passed for servertimes.
> > Otherwise, we will have a CVEs old netty 3 in classpath.
> >
> > Flink is using pekko now, and a hard reboot may not be ok in some cases,
> > and because Pekko ships a Netty3 with many CVEs, they may decide to drop
> > pekko and come out with a dedicated rpc implementation based on Netty4 too.
> > But after we migrate to Netty 4, they can defer that, I will ask around the
> > PMC of Flink at here to let them taking a look.
> >
> > 何品
> >
> >
> > Matthew de Detrich <ma...@aiven.io.invalid> 于2023年9月11日周一
> > 19:35写道：
> >
> > > > that this is a bug, not a feature
> > >
> > > Also meant to say that this is a feature, not a bug
> > >
> > > On Mon, Sep 11, 2023 at 1:34 PM Matthew de Detrich <
> > > matthew.dedetrich@aiven.io> wrote:
> > >
> > > > While I know that people have reservations for keeping the current
> > > > classical transport,
> > > > I just want to relay what I have written at
> > > > https://github.com/apache/incubator-pekko/pull/643
> > > > i.e.
> > > >
> > > > - Upgrading the classical transport from Netty 3 to Netty 4 was in
> > > context
> > > > quite trivial
> > > > - Dropping classical transport means that we are technically breaking
> > > > semver (we can only do this in Pekko 2.0.x)
> > > > - Not upgrading to Netty3 means that we are shipping code with known
> > > CVE's
> > > >
> > > > For these reasons I personally would prefer to merge the PR and just
> > get
> > > > it done with. Even
> > > > with the PR merged, if we change our minds later nothing is stopping us
> > > > from dropping it
> > > > later (or even reverting the PR when testing is done but I doubt that
> > > will
> > > > occur).
> > > >
> > > > There is of course a risk in that updating to Netty 4 might create some
> > > > regressions
> > > > and hence pekko users would complain but I would then argue that this
> > is
> > > a
> > > > bug,
> > > > not a feature because we then get actual feedback as to how many people
> > > > are using
> > > > the classical transport which allows us to make more comprehensive
> > > > decisions in
> > > > the future.
> > > >
> > > > On Fri, Sep 8, 2023 at 5:56 PM kerr <he...@gmail.com> wrote:
> > > >
> > > >> And I gathered this result on reddit
> > > >> [image: image.png]
> > > >> 何品
> > > >>
> > > >>
> > > >> kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
> > > >>
> > > >>> I just read
> > > >>>
> > > >>> To switch a full cluster restart is required and any overrides for
> > > >>> classic remoting need to be ported to Artery configuration. Artery
> > has
> > > a
> > > >>> completely different protocol, which means that a rolling update is
> > not
> > > >>> supported.
> > > >>> So for Flink use case is that ok?
> > > >>> 何品
> > > >>>
> > > >>>
> > > >>> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
> > > >>>
> > > >>>> I did a poll on reddit and get the result of
> > > >>>> [image: image.png]
> > > >>>> 何品
> > > >>>>
> > > >>>>
> > > >>>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
> > > >>>>
> > > >>>>> > Is it really worth spending any time to support classic remoting?
> > > >>>>> Users
> > > >>>>> would be better served to switch to Artery.
> > > >>>>>
> > > >>>>> I'm one of the user and the China Scala User group may not talk
> > much
> > > >>>>> on web but most of us is using the netty transport, even Lightbend
> > > say the
> > > >>>>> `Artery is the best`.
> > > >>>>>
> > > >>>>> Another online product with large scale nodes(here) is running with
> > > >>>>> Akka 2.4.x with the classic transport, and Flink is using the
> > > netty.tpc
> > > >>>>> transport at least for now too.
> > > >>>>>
> > > >>>>> I think the community can choose when they migrate the nodes to
> > > artery
> > > >>>>> is essential, especially when otherwise your cluster need a hard
> > > reboot.
> > > >>>>>
> > > >>>>> AFAIK, the Artery transport is not wire compatible with the
> > > >>>>> classic one.
> > > >>>>>
> > > >>>>> I suggest:
> > > >>>>> 1. Start to send PRs to open sources projects like Flink with the
> > > help
> > > >>>>> for migrating to the artery transport.
> > > >>>>> 2. Upgrade to the new version of Netty for bugfix and
> > > >>>>> better performance.
> > > >>>>> 3. Suggest all the new community move to the Artery transport.
> > > >>>>>
> > > >>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
> > > >>>>>
> > > >>>>> Anyway, different company has different requirements, that just my
> > 2
> > > >>>>> cents.
> > > >>>>>
> > > >>>>> 何品
> > > >>>>>
> > > >>>>>
> > > >>>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
> > > >>>>>
> > > >>>>>> This will end up in a vote.
> > > >>>>>>
> > > >>>>>> For me, we shipped 1.0.x. Users who don't want big changes can use
> > > >>>>>> that version. We can agree to support 1.0.x with critical fixes.
> > > >>>>>>
> > > >>>>>> We can now start improving the code for the improvement release
> > > >>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of deprecated
> > > >>>>>> code and most of us appear to believe that classic remoting is
> > long
> > > >>>>>> deprecated.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
> > > >>>>>> <ma...@aiven.io.invalid> wrote:
> > > >>>>>> >
> > > >>>>>> > > Is it really worth spending any time to support classic
> > > remoting?
> > > >>>>>> Users
> > > >>>>>> > would be better served to switch to Artery.
> > > >>>>>> >
> > > >>>>>> > That depends on what you mean by "support". If agreed on, I
> > would
> > > >>>>>> > propose marking classic remoting as "to be removed" which would
> > > >>>>>> > also mean that the code wouldn't be touched (i.e. no feature
> > > >>>>>> > changes and no bug fixes unless extremely critical).
> > > >>>>>> >
> > > >>>>>> > I am in no way suggesting that we support remoting in a
> > > traditional
> > > >>>>>> sense
> > > >>>>>> > and again the only reason why I am even contemplating this is
> > > >>>>>> because
> > > >>>>>> > of the CVE's.
> > > >>>>>> >
> > > >>>>>> > > I don't think it is likely that a significant portion of Akka
> > > 2.4
> > > >>>>>> users
> > > >>>>>> > would update to Pekko any time soon if the couldn't update to
> > Akka
> > > >>>>>> 2.5/2.6.
> > > >>>>>> >
> > > >>>>>> > I also had this impression but it's already come up a few times
> > > that
> > > >>>>>> > people still use Akka 2.4.x. They likely haven't bothered
> > updating
> > > >>>>>> > because they didn't see a need to (quite famously Akka is quite
> > > >>>>>> stable
> > > >>>>>> > and there are cases of companies using it for years without
> > > needing
> > > >>>>>> to
> > > >>>>>> > update, as stated by Lightbend's CEO) but with the license
> > change
> > > >>>>>> > it created a catalyst/trigger for some users to consider moving
> > to
> > > >>>>>> Pekko.
> > > >>>>>> >
> > > >>>>>> > On this note I would also be wary in making assumptions on what
> > > >>>>>> versions
> > > >>>>>> > of Akka people happen to be using. As a corollary in the
> > > discussion
> > > >>>>>> on
> > > >>>>>> > dropping JDK 8, when we suggested that we quite quickly got
> > > >>>>>> feedback that
> > > >>>>>> > there are people still using JDK 8. And while I know all of the
> > > >>>>>> arguments
> > > >>>>>> > that you really should not be using JDK 8 can be carried over to
> > > >>>>>> Akka.
> > > >>>>>> >
> > > >>>>>> > Admittedly it's quite hard to get good feedback on what versions
> > > >>>>>> people are
> > > >>>>>> > using, just saying we should be a bit more careful in making
> > these
> > > >>>>>> > assumptions.
> > > >>>>>> > After all, people are still using JDK 8 ;)
> > > >>>>>> >
> > > >>>>>> >
> > > >>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <
> > > nvollmar@gmail.com>
> > > >>>>>> wrote:
> > > >>>>>> >
> > > >>>>>> > > Akka 2.4 has been EoL since end of 2017.
> > > >>>>>> > > I don't think it is likely that a significant portion of Akka
> > > 2.4
> > > >>>>>> users
> > > >>>>>> > > would update to Pekko any time soon if the couldn't update to
> > > >>>>>> Akka 2.5/2.6.
> > > >>>>>> > >
> > > >>>>>> > > Is it really worth spending any time to support classic
> > > remoting?
> > > >>>>>> Users
> > > >>>>>> > > would be better served to switch to Artery.
> > > >>>>>> > >
> > > >>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
> > > >>>>>> > > <ma...@aiven.io.invalid> wrote:
> > > >>>>>> > >
> > > >>>>>> > > > > If you manage to move to Pekko and were still using
> > classic
> > > >>>>>> remoting,
> > > >>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you manage
> > > the
> > > >>>>>> update
> > > >>>>>> > > > to Pekko, going to Artery TCP is a small step.
> > > >>>>>> > > >
> > > >>>>>> > > > One of the reasons why I was saying this is that I was under
> > > the
> > > >>>>>> > > impression
> > > >>>>>> > > > that there are a non-trivial amount of users still on Akka
> > 2.4
> > > >>>>>> (not sure
> > > >>>>>> > > > why
> > > >>>>>> > > > this is the case but kerr was telling me this).
> > > >>>>>> > > >
> > > >>>>>> > > > > If people are interested in keeping it around and also opt
> > > in
> > > >>>>>> to
> > > >>>>>> > > > maintaining it (which in the first place means, making the
> > > >>>>>> tests for
> > > >>>>>> > > > it work *now*), than fine, but otherwise, being able to use
> > it
> > > >>>>>> on
> > > >>>>>> > > > 1.0.x is already a big benefit.
> > > >>>>>> > > >
> > > >>>>>> > > > Since classic remoting is currently using Netty 3 which has
> > > >>>>>> CVE's
> > > >>>>>> > > > I don't think it's wise to encourage people that still
> > happen
> > > >>>>>> to be
> > > >>>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't for
> > > final
> > > >>>>>> > > > version of netty 3 having CVE's then I would have no qualms
> > > >>>>>> > > > for dropping it in the 1.1.x series (in fact if there was a
> > > >>>>>> > > > hypothetical netty 3 without CVE's we would have already
> > > >>>>>> > > > updated to it in the 1.0.x series and dropped classic
> > remoting
> > > >>>>>> in
> > > >>>>>> > > > 1.1.x without a heartbeat).
> > > >>>>>> > > >
> > > >>>>>> > > > I would propose that if we do accept keeping classic
> > remoting
> > > >>>>>> > > > and updating to Netty 4, we would mark the feature to be
> > > >>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade that
> > > >>>>>> > > > part of code would be untouched unless there are critical
> > > >>>>>> > > > bugs/regressions.
> > > >>>>>> > > >
> > > >>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
> > > >>>>>> > > > johannes.rudolph@gmail.com>
> > > >>>>>> > > > wrote:
> > > >>>>>> > > >
> > > >>>>>> > > > > We definitely should remove it for 1.1.x. There's no
> > > >>>>>> technical reason
> > > >>>>>> > > > > to keep it because the newer artery TCP transport just
> > > >>>>>> supersedes it.
> > > >>>>>> > > > > As discussed before, the problem with keeping old features
> > > >>>>>> around is
> > > >>>>>> > > > > "death by a thousand cuts". The classic remoting backend
> > is
> > > >>>>>> the prime
> > > >>>>>> > > > > example of it because it is basically unmaintained for
> > years
> > > >>>>>> and
> > > >>>>>> > > > > there's basically no expert knowledge available about how
> > to
> > > >>>>>> maintain
> > > >>>>>> > > > > it or how to fix the existing bugs. Testing remoting
> > > backends
> > > >>>>>> has
> > > >>>>>> > > > > shown to be very complicated and often unreliable and
> > being
> > > >>>>>> able to
> > > >>>>>> > > > > remove this particular component from the test matrix will
> > > be
> > > >>>>>> a huge
> > > >>>>>> > > > > improvement.
> > > >>>>>> > > > >
> > > >>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
> > > >>>>>> > > > > <ma...@aiven.io.invalid> wrote:
> > > >>>>>> > > > > > I would prefer adding in netty 4 support for the classic
> > > >>>>>> transport
> > > >>>>>> > > > > > mechanism because it gives
> > > >>>>>> > > > > > people an upgrade path while allowing them to resolve
> > the
> > > >>>>>> CVE issues.
> > > >>>>>> > > > If
> > > >>>>>> > > > > > Pekko 1.1.x didn't
> > > >>>>>> > > > > > support the classic transport mechanism then it forces
> > > >>>>>> Pekko users to
> > > >>>>>> > > > be
> > > >>>>>> > > > > > stuck on 1.0.x
> > > >>>>>> > > > > > with no easy upgrade path.
> > > >>>>>> > > > >
> > > >>>>>> > > > > We are not responsible for giving people an easy upgrade
> > > path
> > > >>>>>> when
> > > >>>>>> > > > > they were stuck on features like classic remoting which
> > has
> > > >>>>>> basically
> > > >>>>>> > > > > been outdated for years. It is very important that, going
> > > >>>>>> forward,
> > > >>>>>> > > > > that we focus our attention on the most important features
> > > >>>>>> and make
> > > >>>>>> > > > > sure to not get stuck in maintaining old parts that we
> > > cannot
> > > >>>>>> (or
> > > >>>>>> > > > > don't want to) care for.
> > > >>>>>> > > > >
> > > >>>>>> > > > > If people are interested in keeping it around and also opt
> > > in
> > > >>>>>> to
> > > >>>>>> > > > > maintaining it (which in the first place means, making the
> > > >>>>>> tests for
> > > >>>>>> > > > > it work *now*), than fine, but otherwise, being able to
> > use
> > > >>>>>> it on
> > > >>>>>> > > > > 1.0.x is already a big benefit.
> > > >>>>>> > > > >
> > > >>>>>> > > > > Apart from that the update path is pretty clear:
> > > >>>>>> > > > > * move to Pekko 1.0.x
> > > >>>>>> > > > > * move to Arterty TCP
> > > >>>>>> > > > > * move to Pekko 1.1.x
> > > >>>>>> > > > >
> > > >>>>>> > > > > If you manage to move to Pekko and were still using
> > classic
> > > >>>>>> remoting,
> > > >>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you
> > manage
> > > >>>>>> the update
> > > >>>>>> > > > > to Pekko, going to Artery TCP is a small step.
> > > >>>>>> > > > >
> > > >>>>>> > > > > Johannes
> > > >>>>>> > > > >
> > > >>>>>> > > > >
> > > >>>>>>
> > > ---------------------------------------------------------------------
> > > >>>>>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > >>>>>> > > > > For additional commands, e-mail:
> > dev-help@pekko.apache.org
> > > >>>>>> > > > >
> > > >>>>>> > > > >
> > > >>>>>> > > >
> > > >>>>>> > > > --
> > > >>>>>> > > >
> > > >>>>>> > > > Matthew de Detrich
> > > >>>>>> > > >
> > > >>>>>> > > > *Aiven Deutschland GmbH*
> > > >>>>>> > > >
> > > >>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >>>>>> > > >
> > > >>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >>>>>> > > >
> > > >>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >>>>>> > > >
> > > >>>>>> > > > *m:* +491603708037
> > > >>>>>> > > >
> > > >>>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >>>>>> > > >
> > > >>>>>> > >
> > > >>>>>> >
> > > >>>>>> >
> > > >>>>>> > --
> > > >>>>>> >
> > > >>>>>> > Matthew de Detrich
> > > >>>>>> >
> > > >>>>>> > *Aiven Deutschland GmbH*
> > > >>>>>> >
> > > >>>>>> > Immanuelkirchstraße 26, 10405 Berlin
> > > >>>>>> >
> > > >>>>>> > Amtsgericht Charlottenburg, HRB 209739 B
> > > >>>>>> >
> > > >>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >>>>>> >
> > > >>>>>> > *m:* +491603708037
> > > >>>>>> >
> > > >>>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >>>>>>
> > > >>>>>>
> > > ---------------------------------------------------------------------
> > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > >>>>>> For additional commands, e-mail: dev-help@pekko.apache.org
> > > >>>>>>
> > > >>>>>>
> > > >
> > > > --
> > > >
> > > > Matthew de Detrich
> > > >
> > > > *Aiven Deutschland GmbH*
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > *m:* +491603708037
> > > >
> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >
> > >
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by Matthew de Detrich <ma...@aiven.io.INVALID>.
> Flink is using pekko now, and a hard reboot may not be ok in some cases,
and because Pekko ships a Netty3 with many CVEs, they may decide to drop
pekko and come out with a dedicated rpc implementation based on Netty4 too.
Wasn't aware of this (I thought Flink used the new remoting protocol).
I think this is a strong enough reason by itself to upgrade Netty4, but
ofcourse
deprecated classic remoting.
On Mon, Sep 11, 2023 at 3:22 PM kerr <he...@gmail.com> wrote:
> Netty 4 is much better than Netty 3 and we can schedule tests to run for it
> and make sure the tests all passed for servertimes.
> Otherwise, we will have a CVEs old netty 3 in classpath.
>
> Flink is using pekko now, and a hard reboot may not be ok in some cases,
> and because Pekko ships a Netty3 with many CVEs, they may decide to drop
> pekko and come out with a dedicated rpc implementation based on Netty4 too.
> But after we migrate to Netty 4, they can defer that, I will ask around the
> PMC of Flink at here to let them taking a look.
>
> 何品
>
>
> Matthew de Detrich <ma...@aiven.io.invalid> 于2023年9月11日周一
> 19:35写道：
>
> > > that this is a bug, not a feature
> >
> > Also meant to say that this is a feature, not a bug
> >
> > On Mon, Sep 11, 2023 at 1:34 PM Matthew de Detrich <
> > matthew.dedetrich@aiven.io> wrote:
> >
> > > While I know that people have reservations for keeping the current
> > > classical transport,
> > > I just want to relay what I have written at
> > > https://github.com/apache/incubator-pekko/pull/643
> > > i.e.
> > >
> > > - Upgrading the classical transport from Netty 3 to Netty 4 was in
> > context
> > > quite trivial
> > > - Dropping classical transport means that we are technically breaking
> > > semver (we can only do this in Pekko 2.0.x)
> > > - Not upgrading to Netty3 means that we are shipping code with known
> > CVE's
> > >
> > > For these reasons I personally would prefer to merge the PR and just
> get
> > > it done with. Even
> > > with the PR merged, if we change our minds later nothing is stopping us
> > > from dropping it
> > > later (or even reverting the PR when testing is done but I doubt that
> > will
> > > occur).
> > >
> > > There is of course a risk in that updating to Netty 4 might create some
> > > regressions
> > > and hence pekko users would complain but I would then argue that this
> is
> > a
> > > bug,
> > > not a feature because we then get actual feedback as to how many people
> > > are using
> > > the classical transport which allows us to make more comprehensive
> > > decisions in
> > > the future.
> > >
> > > On Fri, Sep 8, 2023 at 5:56 PM kerr <he...@gmail.com> wrote:
> > >
> > >> And I gathered this result on reddit
> > >> [image: image.png]
> > >> 何品
> > >>
> > >>
> > >> kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
> > >>
> > >>> I just read
> > >>>
> > >>> To switch a full cluster restart is required and any overrides for
> > >>> classic remoting need to be ported to Artery configuration. Artery
> has
> > a
> > >>> completely different protocol, which means that a rolling update is
> not
> > >>> supported.
> > >>> So for Flink use case is that ok?
> > >>> 何品
> > >>>
> > >>>
> > >>> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
> > >>>
> > >>>> I did a poll on reddit and get the result of
> > >>>> [image: image.png]
> > >>>> 何品
> > >>>>
> > >>>>
> > >>>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
> > >>>>
> > >>>>> > Is it really worth spending any time to support classic remoting?
> > >>>>> Users
> > >>>>> would be better served to switch to Artery.
> > >>>>>
> > >>>>> I'm one of the user and the China Scala User group may not talk
> much
> > >>>>> on web but most of us is using the netty transport, even Lightbend
> > say the
> > >>>>> `Artery is the best`.
> > >>>>>
> > >>>>> Another online product with large scale nodes(here) is running with
> > >>>>> Akka 2.4.x with the classic transport, and Flink is using the
> > netty.tpc
> > >>>>> transport at least for now too.
> > >>>>>
> > >>>>> I think the community can choose when they migrate the nodes to
> > artery
> > >>>>> is essential, especially when otherwise your cluster need a hard
> > reboot.
> > >>>>>
> > >>>>> AFAIK, the Artery transport is not wire compatible with the
> > >>>>> classic one.
> > >>>>>
> > >>>>> I suggest:
> > >>>>> 1. Start to send PRs to open sources projects like Flink with the
> > help
> > >>>>> for migrating to the artery transport.
> > >>>>> 2. Upgrade to the new version of Netty for bugfix and
> > >>>>> better performance.
> > >>>>> 3. Suggest all the new community move to the Artery transport.
> > >>>>>
> > >>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
> > >>>>>
> > >>>>> Anyway, different company has different requirements, that just my
> 2
> > >>>>> cents.
> > >>>>>
> > >>>>> 何品
> > >>>>>
> > >>>>>
> > >>>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
> > >>>>>
> > >>>>>> This will end up in a vote.
> > >>>>>>
> > >>>>>> For me, we shipped 1.0.x. Users who don't want big changes can use
> > >>>>>> that version. We can agree to support 1.0.x with critical fixes.
> > >>>>>>
> > >>>>>> We can now start improving the code for the improvement release
> > >>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of deprecated
> > >>>>>> code and most of us appear to believe that classic remoting is
> long
> > >>>>>> deprecated.
> > >>>>>>
> > >>>>>>
> > >>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
> > >>>>>> <ma...@aiven.io.invalid> wrote:
> > >>>>>> >
> > >>>>>> > > Is it really worth spending any time to support classic
> > remoting?
> > >>>>>> Users
> > >>>>>> > would be better served to switch to Artery.
> > >>>>>> >
> > >>>>>> > That depends on what you mean by "support". If agreed on, I
> would
> > >>>>>> > propose marking classic remoting as "to be removed" which would
> > >>>>>> > also mean that the code wouldn't be touched (i.e. no feature
> > >>>>>> > changes and no bug fixes unless extremely critical).
> > >>>>>> >
> > >>>>>> > I am in no way suggesting that we support remoting in a
> > traditional
> > >>>>>> sense
> > >>>>>> > and again the only reason why I am even contemplating this is
> > >>>>>> because
> > >>>>>> > of the CVE's.
> > >>>>>> >
> > >>>>>> > > I don't think it is likely that a significant portion of Akka
> > 2.4
> > >>>>>> users
> > >>>>>> > would update to Pekko any time soon if the couldn't update to
> Akka
> > >>>>>> 2.5/2.6.
> > >>>>>> >
> > >>>>>> > I also had this impression but it's already come up a few times
> > that
> > >>>>>> > people still use Akka 2.4.x. They likely haven't bothered
> updating
> > >>>>>> > because they didn't see a need to (quite famously Akka is quite
> > >>>>>> stable
> > >>>>>> > and there are cases of companies using it for years without
> > needing
> > >>>>>> to
> > >>>>>> > update, as stated by Lightbend's CEO) but with the license
> change
> > >>>>>> > it created a catalyst/trigger for some users to consider moving
> to
> > >>>>>> Pekko.
> > >>>>>> >
> > >>>>>> > On this note I would also be wary in making assumptions on what
> > >>>>>> versions
> > >>>>>> > of Akka people happen to be using. As a corollary in the
> > discussion
> > >>>>>> on
> > >>>>>> > dropping JDK 8, when we suggested that we quite quickly got
> > >>>>>> feedback that
> > >>>>>> > there are people still using JDK 8. And while I know all of the
> > >>>>>> arguments
> > >>>>>> > that you really should not be using JDK 8 can be carried over to
> > >>>>>> Akka.
> > >>>>>> >
> > >>>>>> > Admittedly it's quite hard to get good feedback on what versions
> > >>>>>> people are
> > >>>>>> > using, just saying we should be a bit more careful in making
> these
> > >>>>>> > assumptions.
> > >>>>>> > After all, people are still using JDK 8 ;)
> > >>>>>> >
> > >>>>>> >
> > >>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <
> > nvollmar@gmail.com>
> > >>>>>> wrote:
> > >>>>>> >
> > >>>>>> > > Akka 2.4 has been EoL since end of 2017.
> > >>>>>> > > I don't think it is likely that a significant portion of Akka
> > 2.4
> > >>>>>> users
> > >>>>>> > > would update to Pekko any time soon if the couldn't update to
> > >>>>>> Akka 2.5/2.6.
> > >>>>>> > >
> > >>>>>> > > Is it really worth spending any time to support classic
> > remoting?
> > >>>>>> Users
> > >>>>>> > > would be better served to switch to Artery.
> > >>>>>> > >
> > >>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
> > >>>>>> > > <ma...@aiven.io.invalid> wrote:
> > >>>>>> > >
> > >>>>>> > > > > If you manage to move to Pekko and were still using
> classic
> > >>>>>> remoting,
> > >>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you manage
> > the
> > >>>>>> update
> > >>>>>> > > > to Pekko, going to Artery TCP is a small step.
> > >>>>>> > > >
> > >>>>>> > > > One of the reasons why I was saying this is that I was under
> > the
> > >>>>>> > > impression
> > >>>>>> > > > that there are a non-trivial amount of users still on Akka
> 2.4
> > >>>>>> (not sure
> > >>>>>> > > > why
> > >>>>>> > > > this is the case but kerr was telling me this).
> > >>>>>> > > >
> > >>>>>> > > > > If people are interested in keeping it around and also opt
> > in
> > >>>>>> to
> > >>>>>> > > > maintaining it (which in the first place means, making the
> > >>>>>> tests for
> > >>>>>> > > > it work *now*), than fine, but otherwise, being able to use
> it
> > >>>>>> on
> > >>>>>> > > > 1.0.x is already a big benefit.
> > >>>>>> > > >
> > >>>>>> > > > Since classic remoting is currently using Netty 3 which has
> > >>>>>> CVE's
> > >>>>>> > > > I don't think it's wise to encourage people that still
> happen
> > >>>>>> to be
> > >>>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't for
> > final
> > >>>>>> > > > version of netty 3 having CVE's then I would have no qualms
> > >>>>>> > > > for dropping it in the 1.1.x series (in fact if there was a
> > >>>>>> > > > hypothetical netty 3 without CVE's we would have already
> > >>>>>> > > > updated to it in the 1.0.x series and dropped classic
> remoting
> > >>>>>> in
> > >>>>>> > > > 1.1.x without a heartbeat).
> > >>>>>> > > >
> > >>>>>> > > > I would propose that if we do accept keeping classic
> remoting
> > >>>>>> > > > and updating to Netty 4, we would mark the feature to be
> > >>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade that
> > >>>>>> > > > part of code would be untouched unless there are critical
> > >>>>>> > > > bugs/regressions.
> > >>>>>> > > >
> > >>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
> > >>>>>> > > > johannes.rudolph@gmail.com>
> > >>>>>> > > > wrote:
> > >>>>>> > > >
> > >>>>>> > > > > We definitely should remove it for 1.1.x. There's no
> > >>>>>> technical reason
> > >>>>>> > > > > to keep it because the newer artery TCP transport just
> > >>>>>> supersedes it.
> > >>>>>> > > > > As discussed before, the problem with keeping old features
> > >>>>>> around is
> > >>>>>> > > > > "death by a thousand cuts". The classic remoting backend
> is
> > >>>>>> the prime
> > >>>>>> > > > > example of it because it is basically unmaintained for
> years
> > >>>>>> and
> > >>>>>> > > > > there's basically no expert knowledge available about how
> to
> > >>>>>> maintain
> > >>>>>> > > > > it or how to fix the existing bugs. Testing remoting
> > backends
> > >>>>>> has
> > >>>>>> > > > > shown to be very complicated and often unreliable and
> being
> > >>>>>> able to
> > >>>>>> > > > > remove this particular component from the test matrix will
> > be
> > >>>>>> a huge
> > >>>>>> > > > > improvement.
> > >>>>>> > > > >
> > >>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
> > >>>>>> > > > > <ma...@aiven.io.invalid> wrote:
> > >>>>>> > > > > > I would prefer adding in netty 4 support for the classic
> > >>>>>> transport
> > >>>>>> > > > > > mechanism because it gives
> > >>>>>> > > > > > people an upgrade path while allowing them to resolve
> the
> > >>>>>> CVE issues.
> > >>>>>> > > > If
> > >>>>>> > > > > > Pekko 1.1.x didn't
> > >>>>>> > > > > > support the classic transport mechanism then it forces
> > >>>>>> Pekko users to
> > >>>>>> > > > be
> > >>>>>> > > > > > stuck on 1.0.x
> > >>>>>> > > > > > with no easy upgrade path.
> > >>>>>> > > > >
> > >>>>>> > > > > We are not responsible for giving people an easy upgrade
> > path
> > >>>>>> when
> > >>>>>> > > > > they were stuck on features like classic remoting which
> has
> > >>>>>> basically
> > >>>>>> > > > > been outdated for years. It is very important that, going
> > >>>>>> forward,
> > >>>>>> > > > > that we focus our attention on the most important features
> > >>>>>> and make
> > >>>>>> > > > > sure to not get stuck in maintaining old parts that we
> > cannot
> > >>>>>> (or
> > >>>>>> > > > > don't want to) care for.
> > >>>>>> > > > >
> > >>>>>> > > > > If people are interested in keeping it around and also opt
> > in
> > >>>>>> to
> > >>>>>> > > > > maintaining it (which in the first place means, making the
> > >>>>>> tests for
> > >>>>>> > > > > it work *now*), than fine, but otherwise, being able to
> use
> > >>>>>> it on
> > >>>>>> > > > > 1.0.x is already a big benefit.
> > >>>>>> > > > >
> > >>>>>> > > > > Apart from that the update path is pretty clear:
> > >>>>>> > > > > * move to Pekko 1.0.x
> > >>>>>> > > > > * move to Arterty TCP
> > >>>>>> > > > > * move to Pekko 1.1.x
> > >>>>>> > > > >
> > >>>>>> > > > > If you manage to move to Pekko and were still using
> classic
> > >>>>>> remoting,
> > >>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you
> manage
> > >>>>>> the update
> > >>>>>> > > > > to Pekko, going to Artery TCP is a small step.
> > >>>>>> > > > >
> > >>>>>> > > > > Johannes
> > >>>>>> > > > >
> > >>>>>> > > > >
> > >>>>>>
> > ---------------------------------------------------------------------
> > >>>>>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > >>>>>> > > > > For additional commands, e-mail:
> dev-help@pekko.apache.org
> > >>>>>> > > > >
> > >>>>>> > > > >
> > >>>>>> > > >
> > >>>>>> > > > --
> > >>>>>> > > >
> > >>>>>> > > > Matthew de Detrich
> > >>>>>> > > >
> > >>>>>> > > > *Aiven Deutschland GmbH*
> > >>>>>> > > >
> > >>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
> > >>>>>> > > >
> > >>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > >>>>>> > > >
> > >>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >>>>>> > > >
> > >>>>>> > > > *m:* +491603708037
> > >>>>>> > > >
> > >>>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >>>>>> > > >
> > >>>>>> > >
> > >>>>>> >
> > >>>>>> >
> > >>>>>> > --
> > >>>>>> >
> > >>>>>> > Matthew de Detrich
> > >>>>>> >
> > >>>>>> > *Aiven Deutschland GmbH*
> > >>>>>> >
> > >>>>>> > Immanuelkirchstraße 26, 10405 Berlin
> > >>>>>> >
> > >>>>>> > Amtsgericht Charlottenburg, HRB 209739 B
> > >>>>>> >
> > >>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >>>>>> >
> > >>>>>> > *m:* +491603708037
> > >>>>>> >
> > >>>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >>>>>>
> > >>>>>>
> > ---------------------------------------------------------------------
> > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > >>>>>> For additional commands, e-mail: dev-help@pekko.apache.org
> > >>>>>>
> > >>>>>>
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>
--
Matthew de Detrich
*Aiven Deutschland GmbH*
Immanuelkirchstraße 26, 10405 Berlin
Amtsgericht Charlottenburg, HRB 209739 B
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
*m:* +491603708037
*w:* aiven.io *e:* matthew.dedetrich@aiven.io
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by kerr <he...@gmail.com>.
Netty 4 is much better than Netty 3 and we can schedule tests to run for it
and make sure the tests all passed for servertimes.
Otherwise, we will have a CVEs old netty 3 in classpath.
Flink is using pekko now, and a hard reboot may not be ok in some cases,
and because Pekko ships a Netty3 with many CVEs, they may decide to drop
pekko and come out with a dedicated rpc implementation based on Netty4 too.
But after we migrate to Netty 4, they can defer that, I will ask around the
PMC of Flink at here to let them taking a look.
何品
Matthew de Detrich <ma...@aiven.io.invalid> 于2023年9月11日周一
19:35写道：
> > that this is a bug, not a feature
>
> Also meant to say that this is a feature, not a bug
>
> On Mon, Sep 11, 2023 at 1:34 PM Matthew de Detrich <
> matthew.dedetrich@aiven.io> wrote:
>
> > While I know that people have reservations for keeping the current
> > classical transport,
> > I just want to relay what I have written at
> > https://github.com/apache/incubator-pekko/pull/643
> > i.e.
> >
> > - Upgrading the classical transport from Netty 3 to Netty 4 was in
> context
> > quite trivial
> > - Dropping classical transport means that we are technically breaking
> > semver (we can only do this in Pekko 2.0.x)
> > - Not upgrading to Netty3 means that we are shipping code with known
> CVE's
> >
> > For these reasons I personally would prefer to merge the PR and just get
> > it done with. Even
> > with the PR merged, if we change our minds later nothing is stopping us
> > from dropping it
> > later (or even reverting the PR when testing is done but I doubt that
> will
> > occur).
> >
> > There is of course a risk in that updating to Netty 4 might create some
> > regressions
> > and hence pekko users would complain but I would then argue that this is
> a
> > bug,
> > not a feature because we then get actual feedback as to how many people
> > are using
> > the classical transport which allows us to make more comprehensive
> > decisions in
> > the future.
> >
> > On Fri, Sep 8, 2023 at 5:56 PM kerr <he...@gmail.com> wrote:
> >
> >> And I gathered this result on reddit
> >> [image: image.png]
> >> 何品
> >>
> >>
> >> kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
> >>
> >>> I just read
> >>>
> >>> To switch a full cluster restart is required and any overrides for
> >>> classic remoting need to be ported to Artery configuration. Artery has
> a
> >>> completely different protocol, which means that a rolling update is not
> >>> supported.
> >>> So for Flink use case is that ok?
> >>> 何品
> >>>
> >>>
> >>> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
> >>>
> >>>> I did a poll on reddit and get the result of
> >>>> [image: image.png]
> >>>> 何品
> >>>>
> >>>>
> >>>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
> >>>>
> >>>>> > Is it really worth spending any time to support classic remoting?
> >>>>> Users
> >>>>> would be better served to switch to Artery.
> >>>>>
> >>>>> I'm one of the user and the China Scala User group may not talk much
> >>>>> on web but most of us is using the netty transport, even Lightbend
> say the
> >>>>> `Artery is the best`.
> >>>>>
> >>>>> Another online product with large scale nodes(here) is running with
> >>>>> Akka 2.4.x with the classic transport, and Flink is using the
> netty.tpc
> >>>>> transport at least for now too.
> >>>>>
> >>>>> I think the community can choose when they migrate the nodes to
> artery
> >>>>> is essential, especially when otherwise your cluster need a hard
> reboot.
> >>>>>
> >>>>> AFAIK, the Artery transport is not wire compatible with the
> >>>>> classic one.
> >>>>>
> >>>>> I suggest:
> >>>>> 1. Start to send PRs to open sources projects like Flink with the
> help
> >>>>> for migrating to the artery transport.
> >>>>> 2. Upgrade to the new version of Netty for bugfix and
> >>>>> better performance.
> >>>>> 3. Suggest all the new community move to the Artery transport.
> >>>>>
> >>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
> >>>>>
> >>>>> Anyway, different company has different requirements, that just my 2
> >>>>> cents.
> >>>>>
> >>>>> 何品
> >>>>>
> >>>>>
> >>>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
> >>>>>
> >>>>>> This will end up in a vote.
> >>>>>>
> >>>>>> For me, we shipped 1.0.x. Users who don't want big changes can use
> >>>>>> that version. We can agree to support 1.0.x with critical fixes.
> >>>>>>
> >>>>>> We can now start improving the code for the improvement release
> >>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of deprecated
> >>>>>> code and most of us appear to believe that classic remoting is long
> >>>>>> deprecated.
> >>>>>>
> >>>>>>
> >>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
> >>>>>> <ma...@aiven.io.invalid> wrote:
> >>>>>> >
> >>>>>> > > Is it really worth spending any time to support classic
> remoting?
> >>>>>> Users
> >>>>>> > would be better served to switch to Artery.
> >>>>>> >
> >>>>>> > That depends on what you mean by "support". If agreed on, I would
> >>>>>> > propose marking classic remoting as "to be removed" which would
> >>>>>> > also mean that the code wouldn't be touched (i.e. no feature
> >>>>>> > changes and no bug fixes unless extremely critical).
> >>>>>> >
> >>>>>> > I am in no way suggesting that we support remoting in a
> traditional
> >>>>>> sense
> >>>>>> > and again the only reason why I am even contemplating this is
> >>>>>> because
> >>>>>> > of the CVE's.
> >>>>>> >
> >>>>>> > > I don't think it is likely that a significant portion of Akka
> 2.4
> >>>>>> users
> >>>>>> > would update to Pekko any time soon if the couldn't update to Akka
> >>>>>> 2.5/2.6.
> >>>>>> >
> >>>>>> > I also had this impression but it's already come up a few times
> that
> >>>>>> > people still use Akka 2.4.x. They likely haven't bothered updating
> >>>>>> > because they didn't see a need to (quite famously Akka is quite
> >>>>>> stable
> >>>>>> > and there are cases of companies using it for years without
> needing
> >>>>>> to
> >>>>>> > update, as stated by Lightbend's CEO) but with the license change
> >>>>>> > it created a catalyst/trigger for some users to consider moving to
> >>>>>> Pekko.
> >>>>>> >
> >>>>>> > On this note I would also be wary in making assumptions on what
> >>>>>> versions
> >>>>>> > of Akka people happen to be using. As a corollary in the
> discussion
> >>>>>> on
> >>>>>> > dropping JDK 8, when we suggested that we quite quickly got
> >>>>>> feedback that
> >>>>>> > there are people still using JDK 8. And while I know all of the
> >>>>>> arguments
> >>>>>> > that you really should not be using JDK 8 can be carried over to
> >>>>>> Akka.
> >>>>>> >
> >>>>>> > Admittedly it's quite hard to get good feedback on what versions
> >>>>>> people are
> >>>>>> > using, just saying we should be a bit more careful in making these
> >>>>>> > assumptions.
> >>>>>> > After all, people are still using JDK 8 ;)
> >>>>>> >
> >>>>>> >
> >>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <
> nvollmar@gmail.com>
> >>>>>> wrote:
> >>>>>> >
> >>>>>> > > Akka 2.4 has been EoL since end of 2017.
> >>>>>> > > I don't think it is likely that a significant portion of Akka
> 2.4
> >>>>>> users
> >>>>>> > > would update to Pekko any time soon if the couldn't update to
> >>>>>> Akka 2.5/2.6.
> >>>>>> > >
> >>>>>> > > Is it really worth spending any time to support classic
> remoting?
> >>>>>> Users
> >>>>>> > > would be better served to switch to Artery.
> >>>>>> > >
> >>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
> >>>>>> > > <ma...@aiven.io.invalid> wrote:
> >>>>>> > >
> >>>>>> > > > > If you manage to move to Pekko and were still using classic
> >>>>>> remoting,
> >>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you manage
> the
> >>>>>> update
> >>>>>> > > > to Pekko, going to Artery TCP is a small step.
> >>>>>> > > >
> >>>>>> > > > One of the reasons why I was saying this is that I was under
> the
> >>>>>> > > impression
> >>>>>> > > > that there are a non-trivial amount of users still on Akka 2.4
> >>>>>> (not sure
> >>>>>> > > > why
> >>>>>> > > > this is the case but kerr was telling me this).
> >>>>>> > > >
> >>>>>> > > > > If people are interested in keeping it around and also opt
> in
> >>>>>> to
> >>>>>> > > > maintaining it (which in the first place means, making the
> >>>>>> tests for
> >>>>>> > > > it work *now*), than fine, but otherwise, being able to use it
> >>>>>> on
> >>>>>> > > > 1.0.x is already a big benefit.
> >>>>>> > > >
> >>>>>> > > > Since classic remoting is currently using Netty 3 which has
> >>>>>> CVE's
> >>>>>> > > > I don't think it's wise to encourage people that still happen
> >>>>>> to be
> >>>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't for
> final
> >>>>>> > > > version of netty 3 having CVE's then I would have no qualms
> >>>>>> > > > for dropping it in the 1.1.x series (in fact if there was a
> >>>>>> > > > hypothetical netty 3 without CVE's we would have already
> >>>>>> > > > updated to it in the 1.0.x series and dropped classic remoting
> >>>>>> in
> >>>>>> > > > 1.1.x without a heartbeat).
> >>>>>> > > >
> >>>>>> > > > I would propose that if we do accept keeping classic remoting
> >>>>>> > > > and updating to Netty 4, we would mark the feature to be
> >>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade that
> >>>>>> > > > part of code would be untouched unless there are critical
> >>>>>> > > > bugs/regressions.
> >>>>>> > > >
> >>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
> >>>>>> > > > johannes.rudolph@gmail.com>
> >>>>>> > > > wrote:
> >>>>>> > > >
> >>>>>> > > > > We definitely should remove it for 1.1.x. There's no
> >>>>>> technical reason
> >>>>>> > > > > to keep it because the newer artery TCP transport just
> >>>>>> supersedes it.
> >>>>>> > > > > As discussed before, the problem with keeping old features
> >>>>>> around is
> >>>>>> > > > > "death by a thousand cuts". The classic remoting backend is
> >>>>>> the prime
> >>>>>> > > > > example of it because it is basically unmaintained for years
> >>>>>> and
> >>>>>> > > > > there's basically no expert knowledge available about how to
> >>>>>> maintain
> >>>>>> > > > > it or how to fix the existing bugs. Testing remoting
> backends
> >>>>>> has
> >>>>>> > > > > shown to be very complicated and often unreliable and being
> >>>>>> able to
> >>>>>> > > > > remove this particular component from the test matrix will
> be
> >>>>>> a huge
> >>>>>> > > > > improvement.
> >>>>>> > > > >
> >>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
> >>>>>> > > > > <ma...@aiven.io.invalid> wrote:
> >>>>>> > > > > > I would prefer adding in netty 4 support for the classic
> >>>>>> transport
> >>>>>> > > > > > mechanism because it gives
> >>>>>> > > > > > people an upgrade path while allowing them to resolve the
> >>>>>> CVE issues.
> >>>>>> > > > If
> >>>>>> > > > > > Pekko 1.1.x didn't
> >>>>>> > > > > > support the classic transport mechanism then it forces
> >>>>>> Pekko users to
> >>>>>> > > > be
> >>>>>> > > > > > stuck on 1.0.x
> >>>>>> > > > > > with no easy upgrade path.
> >>>>>> > > > >
> >>>>>> > > > > We are not responsible for giving people an easy upgrade
> path
> >>>>>> when
> >>>>>> > > > > they were stuck on features like classic remoting which has
> >>>>>> basically
> >>>>>> > > > > been outdated for years. It is very important that, going
> >>>>>> forward,
> >>>>>> > > > > that we focus our attention on the most important features
> >>>>>> and make
> >>>>>> > > > > sure to not get stuck in maintaining old parts that we
> cannot
> >>>>>> (or
> >>>>>> > > > > don't want to) care for.
> >>>>>> > > > >
> >>>>>> > > > > If people are interested in keeping it around and also opt
> in
> >>>>>> to
> >>>>>> > > > > maintaining it (which in the first place means, making the
> >>>>>> tests for
> >>>>>> > > > > it work *now*), than fine, but otherwise, being able to use
> >>>>>> it on
> >>>>>> > > > > 1.0.x is already a big benefit.
> >>>>>> > > > >
> >>>>>> > > > > Apart from that the update path is pretty clear:
> >>>>>> > > > > * move to Pekko 1.0.x
> >>>>>> > > > > * move to Arterty TCP
> >>>>>> > > > > * move to Pekko 1.1.x
> >>>>>> > > > >
> >>>>>> > > > > If you manage to move to Pekko and were still using classic
> >>>>>> remoting,
> >>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you manage
> >>>>>> the update
> >>>>>> > > > > to Pekko, going to Artery TCP is a small step.
> >>>>>> > > > >
> >>>>>> > > > > Johannes
> >>>>>> > > > >
> >>>>>> > > > >
> >>>>>>
> ---------------------------------------------------------------------
> >>>>>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> >>>>>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> >>>>>> > > > >
> >>>>>> > > > >
> >>>>>> > > >
> >>>>>> > > > --
> >>>>>> > > >
> >>>>>> > > > Matthew de Detrich
> >>>>>> > > >
> >>>>>> > > > *Aiven Deutschland GmbH*
> >>>>>> > > >
> >>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
> >>>>>> > > >
> >>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
> >>>>>> > > >
> >>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >>>>>> > > >
> >>>>>> > > > *m:* +491603708037
> >>>>>> > > >
> >>>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >>>>>> > > >
> >>>>>> > >
> >>>>>> >
> >>>>>> >
> >>>>>> > --
> >>>>>> >
> >>>>>> > Matthew de Detrich
> >>>>>> >
> >>>>>> > *Aiven Deutschland GmbH*
> >>>>>> >
> >>>>>> > Immanuelkirchstraße 26, 10405 Berlin
> >>>>>> >
> >>>>>> > Amtsgericht Charlottenburg, HRB 209739 B
> >>>>>> >
> >>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >>>>>> >
> >>>>>> > *m:* +491603708037
> >>>>>> >
> >>>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >>>>>>
> >>>>>>
> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> >>>>>> For additional commands, e-mail: dev-help@pekko.apache.org
> >>>>>>
> >>>>>>
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by Matthew de Detrich <ma...@aiven.io.INVALID>.
> that this is a bug, not a feature
Also meant to say that this is a feature, not a bug
On Mon, Sep 11, 2023 at 1:34 PM Matthew de Detrich <
matthew.dedetrich@aiven.io> wrote:
> While I know that people have reservations for keeping the current
> classical transport,
> I just want to relay what I have written at
> https://github.com/apache/incubator-pekko/pull/643
> i.e.
>
> - Upgrading the classical transport from Netty 3 to Netty 4 was in context
> quite trivial
> - Dropping classical transport means that we are technically breaking
> semver (we can only do this in Pekko 2.0.x)
> - Not upgrading to Netty3 means that we are shipping code with known CVE's
>
> For these reasons I personally would prefer to merge the PR and just get
> it done with. Even
> with the PR merged, if we change our minds later nothing is stopping us
> from dropping it
> later (or even reverting the PR when testing is done but I doubt that will
> occur).
>
> There is of course a risk in that updating to Netty 4 might create some
> regressions
> and hence pekko users would complain but I would then argue that this is a
> bug,
> not a feature because we then get actual feedback as to how many people
> are using
> the classical transport which allows us to make more comprehensive
> decisions in
> the future.
>
> On Fri, Sep 8, 2023 at 5:56 PM kerr <he...@gmail.com> wrote:
>
>> And I gathered this result on reddit
>> [image: image.png]
>> 何品
>>
>>
>> kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
>>
>>> I just read
>>>
>>> To switch a full cluster restart is required and any overrides for
>>> classic remoting need to be ported to Artery configuration. Artery has a
>>> completely different protocol, which means that a rolling update is not
>>> supported.
>>> So for Flink use case is that ok?
>>> 何品
>>>
>>>
>>> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
>>>
>>>> I did a poll on reddit and get the result of
>>>> [image: image.png]
>>>> 何品
>>>>
>>>>
>>>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
>>>>
>>>>> > Is it really worth spending any time to support classic remoting?
>>>>> Users
>>>>> would be better served to switch to Artery.
>>>>>
>>>>> I'm one of the user and the China Scala User group may not talk much
>>>>> on web but most of us is using the netty transport, even Lightbend say the
>>>>> `Artery is the best`.
>>>>>
>>>>> Another online product with large scale nodes(here) is running with
>>>>> Akka 2.4.x with the classic transport, and Flink is using the netty.tpc
>>>>> transport at least for now too.
>>>>>
>>>>> I think the community can choose when they migrate the nodes to artery
>>>>> is essential, especially when otherwise your cluster need a hard reboot.
>>>>>
>>>>> AFAIK, the Artery transport is not wire compatible with the
>>>>> classic one.
>>>>>
>>>>> I suggest:
>>>>> 1. Start to send PRs to open sources projects like Flink with the help
>>>>> for migrating to the artery transport.
>>>>> 2. Upgrade to the new version of Netty for bugfix and
>>>>> better performance.
>>>>> 3. Suggest all the new community move to the Artery transport.
>>>>>
>>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
>>>>>
>>>>> Anyway, different company has different requirements, that just my 2
>>>>> cents.
>>>>>
>>>>> 何品
>>>>>
>>>>>
>>>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
>>>>>
>>>>>> This will end up in a vote.
>>>>>>
>>>>>> For me, we shipped 1.0.x. Users who don't want big changes can use
>>>>>> that version. We can agree to support 1.0.x with critical fixes.
>>>>>>
>>>>>> We can now start improving the code for the improvement release
>>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of deprecated
>>>>>> code and most of us appear to believe that classic remoting is long
>>>>>> deprecated.
>>>>>>
>>>>>>
>>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
>>>>>> <ma...@aiven.io.invalid> wrote:
>>>>>> >
>>>>>> > > Is it really worth spending any time to support classic remoting?
>>>>>> Users
>>>>>> > would be better served to switch to Artery.
>>>>>> >
>>>>>> > That depends on what you mean by "support". If agreed on, I would
>>>>>> > propose marking classic remoting as "to be removed" which would
>>>>>> > also mean that the code wouldn't be touched (i.e. no feature
>>>>>> > changes and no bug fixes unless extremely critical).
>>>>>> >
>>>>>> > I am in no way suggesting that we support remoting in a traditional
>>>>>> sense
>>>>>> > and again the only reason why I am even contemplating this is
>>>>>> because
>>>>>> > of the CVE's.
>>>>>> >
>>>>>> > > I don't think it is likely that a significant portion of Akka 2.4
>>>>>> users
>>>>>> > would update to Pekko any time soon if the couldn't update to Akka
>>>>>> 2.5/2.6.
>>>>>> >
>>>>>> > I also had this impression but it's already come up a few times that
>>>>>> > people still use Akka 2.4.x. They likely haven't bothered updating
>>>>>> > because they didn't see a need to (quite famously Akka is quite
>>>>>> stable
>>>>>> > and there are cases of companies using it for years without needing
>>>>>> to
>>>>>> > update, as stated by Lightbend's CEO) but with the license change
>>>>>> > it created a catalyst/trigger for some users to consider moving to
>>>>>> Pekko.
>>>>>> >
>>>>>> > On this note I would also be wary in making assumptions on what
>>>>>> versions
>>>>>> > of Akka people happen to be using. As a corollary in the discussion
>>>>>> on
>>>>>> > dropping JDK 8, when we suggested that we quite quickly got
>>>>>> feedback that
>>>>>> > there are people still using JDK 8. And while I know all of the
>>>>>> arguments
>>>>>> > that you really should not be using JDK 8 can be carried over to
>>>>>> Akka.
>>>>>> >
>>>>>> > Admittedly it's quite hard to get good feedback on what versions
>>>>>> people are
>>>>>> > using, just saying we should be a bit more careful in making these
>>>>>> > assumptions.
>>>>>> > After all, people are still using JDK 8 ;)
>>>>>> >
>>>>>> >
>>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <nv...@gmail.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > > Akka 2.4 has been EoL since end of 2017.
>>>>>> > > I don't think it is likely that a significant portion of Akka 2.4
>>>>>> users
>>>>>> > > would update to Pekko any time soon if the couldn't update to
>>>>>> Akka 2.5/2.6.
>>>>>> > >
>>>>>> > > Is it really worth spending any time to support classic remoting?
>>>>>> Users
>>>>>> > > would be better served to switch to Artery.
>>>>>> > >
>>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
>>>>>> > > <ma...@aiven.io.invalid> wrote:
>>>>>> > >
>>>>>> > > > > If you manage to move to Pekko and were still using classic
>>>>>> remoting,
>>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you manage the
>>>>>> update
>>>>>> > > > to Pekko, going to Artery TCP is a small step.
>>>>>> > > >
>>>>>> > > > One of the reasons why I was saying this is that I was under the
>>>>>> > > impression
>>>>>> > > > that there are a non-trivial amount of users still on Akka 2.4
>>>>>> (not sure
>>>>>> > > > why
>>>>>> > > > this is the case but kerr was telling me this).
>>>>>> > > >
>>>>>> > > > > If people are interested in keeping it around and also opt in
>>>>>> to
>>>>>> > > > maintaining it (which in the first place means, making the
>>>>>> tests for
>>>>>> > > > it work *now*), than fine, but otherwise, being able to use it
>>>>>> on
>>>>>> > > > 1.0.x is already a big benefit.
>>>>>> > > >
>>>>>> > > > Since classic remoting is currently using Netty 3 which has
>>>>>> CVE's
>>>>>> > > > I don't think it's wise to encourage people that still happen
>>>>>> to be
>>>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't for final
>>>>>> > > > version of netty 3 having CVE's then I would have no qualms
>>>>>> > > > for dropping it in the 1.1.x series (in fact if there was a
>>>>>> > > > hypothetical netty 3 without CVE's we would have already
>>>>>> > > > updated to it in the 1.0.x series and dropped classic remoting
>>>>>> in
>>>>>> > > > 1.1.x without a heartbeat).
>>>>>> > > >
>>>>>> > > > I would propose that if we do accept keeping classic remoting
>>>>>> > > > and updating to Netty 4, we would mark the feature to be
>>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade that
>>>>>> > > > part of code would be untouched unless there are critical
>>>>>> > > > bugs/regressions.
>>>>>> > > >
>>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
>>>>>> > > > johannes.rudolph@gmail.com>
>>>>>> > > > wrote:
>>>>>> > > >
>>>>>> > > > > We definitely should remove it for 1.1.x. There's no
>>>>>> technical reason
>>>>>> > > > > to keep it because the newer artery TCP transport just
>>>>>> supersedes it.
>>>>>> > > > > As discussed before, the problem with keeping old features
>>>>>> around is
>>>>>> > > > > "death by a thousand cuts". The classic remoting backend is
>>>>>> the prime
>>>>>> > > > > example of it because it is basically unmaintained for years
>>>>>> and
>>>>>> > > > > there's basically no expert knowledge available about how to
>>>>>> maintain
>>>>>> > > > > it or how to fix the existing bugs. Testing remoting backends
>>>>>> has
>>>>>> > > > > shown to be very complicated and often unreliable and being
>>>>>> able to
>>>>>> > > > > remove this particular component from the test matrix will be
>>>>>> a huge
>>>>>> > > > > improvement.
>>>>>> > > > >
>>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
>>>>>> > > > > <ma...@aiven.io.invalid> wrote:
>>>>>> > > > > > I would prefer adding in netty 4 support for the classic
>>>>>> transport
>>>>>> > > > > > mechanism because it gives
>>>>>> > > > > > people an upgrade path while allowing them to resolve the
>>>>>> CVE issues.
>>>>>> > > > If
>>>>>> > > > > > Pekko 1.1.x didn't
>>>>>> > > > > > support the classic transport mechanism then it forces
>>>>>> Pekko users to
>>>>>> > > > be
>>>>>> > > > > > stuck on 1.0.x
>>>>>> > > > > > with no easy upgrade path.
>>>>>> > > > >
>>>>>> > > > > We are not responsible for giving people an easy upgrade path
>>>>>> when
>>>>>> > > > > they were stuck on features like classic remoting which has
>>>>>> basically
>>>>>> > > > > been outdated for years. It is very important that, going
>>>>>> forward,
>>>>>> > > > > that we focus our attention on the most important features
>>>>>> and make
>>>>>> > > > > sure to not get stuck in maintaining old parts that we cannot
>>>>>> (or
>>>>>> > > > > don't want to) care for.
>>>>>> > > > >
>>>>>> > > > > If people are interested in keeping it around and also opt in
>>>>>> to
>>>>>> > > > > maintaining it (which in the first place means, making the
>>>>>> tests for
>>>>>> > > > > it work *now*), than fine, but otherwise, being able to use
>>>>>> it on
>>>>>> > > > > 1.0.x is already a big benefit.
>>>>>> > > > >
>>>>>> > > > > Apart from that the update path is pretty clear:
>>>>>> > > > > * move to Pekko 1.0.x
>>>>>> > > > > * move to Arterty TCP
>>>>>> > > > > * move to Pekko 1.1.x
>>>>>> > > > >
>>>>>> > > > > If you manage to move to Pekko and were still using classic
>>>>>> remoting,
>>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you manage
>>>>>> the update
>>>>>> > > > > to Pekko, going to Artery TCP is a small step.
>>>>>> > > > >
>>>>>> > > > > Johannes
>>>>>> > > > >
>>>>>> > > > >
>>>>>> ---------------------------------------------------------------------
>>>>>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>>>>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
>>>>>> > > > >
>>>>>> > > > >
>>>>>> > > >
>>>>>> > > > --
>>>>>> > > >
>>>>>> > > > Matthew de Detrich
>>>>>> > > >
>>>>>> > > > *Aiven Deutschland GmbH*
>>>>>> > > >
>>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
>>>>>> > > >
>>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
>>>>>> > > >
>>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>>>>> > > >
>>>>>> > > > *m:* +491603708037
>>>>>> > > >
>>>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>>>>> > > >
>>>>>> > >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> >
>>>>>> > Matthew de Detrich
>>>>>> >
>>>>>> > *Aiven Deutschland GmbH*
>>>>>> >
>>>>>> > Immanuelkirchstraße 26, 10405 Berlin
>>>>>> >
>>>>>> > Amtsgericht Charlottenburg, HRB 209739 B
>>>>>> >
>>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>>>>> >
>>>>>> > *m:* +491603708037
>>>>>> >
>>>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>>>>> For additional commands, e-mail: dev-help@pekko.apache.org
>>>>>>
>>>>>>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>
--
Matthew de Detrich
*Aiven Deutschland GmbH*
Immanuelkirchstraße 26, 10405 Berlin
Amtsgericht Charlottenburg, HRB 209739 B
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
*m:* +491603708037
*w:* aiven.io *e:* matthew.dedetrich@aiven.io
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by Matthew de Detrich <ma...@aiven.io.INVALID>.
While I know that people have reservations for keeping the current
classical transport,
I just want to relay what I have written at
https://github.com/apache/incubator-pekko/pull/643
i.e.
- Upgrading the classical transport from Netty 3 to Netty 4 was in context
quite trivial
- Dropping classical transport means that we are technically breaking
semver (we can only do this in Pekko 2.0.x)
- Not upgrading to Netty3 means that we are shipping code with known CVE's
For these reasons I personally would prefer to merge the PR and just get it
done with. Even
with the PR merged, if we change our minds later nothing is stopping us
from dropping it
later (or even reverting the PR when testing is done but I doubt that will
occur).
There is of course a risk in that updating to Netty 4 might create some
regressions
and hence pekko users would complain but I would then argue that this is a
bug,
not a feature because we then get actual feedback as to how many people are
using
the classical transport which allows us to make more comprehensive
decisions in
the future.
On Fri, Sep 8, 2023 at 5:56 PM kerr <he...@gmail.com> wrote:
> And I gathered this result on reddit
> [image: image.png]
> 何品
>
>
> kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
>
>> I just read
>>
>> To switch a full cluster restart is required and any overrides for
>> classic remoting need to be ported to Artery configuration. Artery has a
>> completely different protocol, which means that a rolling update is not
>> supported.
>> So for Flink use case is that ok?
>> 何品
>>
>>
>> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
>>
>>> I did a poll on reddit and get the result of
>>> [image: image.png]
>>> 何品
>>>
>>>
>>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
>>>
>>>> > Is it really worth spending any time to support classic remoting?
>>>> Users
>>>> would be better served to switch to Artery.
>>>>
>>>> I'm one of the user and the China Scala User group may not talk much on
>>>> web but most of us is using the netty transport, even Lightbend say the
>>>> `Artery is the best`.
>>>>
>>>> Another online product with large scale nodes(here) is running with
>>>> Akka 2.4.x with the classic transport, and Flink is using the netty.tpc
>>>> transport at least for now too.
>>>>
>>>> I think the community can choose when they migrate the nodes to artery
>>>> is essential, especially when otherwise your cluster need a hard reboot.
>>>>
>>>> AFAIK, the Artery transport is not wire compatible with the classic one.
>>>>
>>>> I suggest:
>>>> 1. Start to send PRs to open sources projects like Flink with the help
>>>> for migrating to the artery transport.
>>>> 2. Upgrade to the new version of Netty for bugfix and
>>>> better performance.
>>>> 3. Suggest all the new community move to the Artery transport.
>>>>
>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
>>>>
>>>> Anyway, different company has different requirements, that just my 2
>>>> cents.
>>>>
>>>> 何品
>>>>
>>>>
>>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
>>>>
>>>>> This will end up in a vote.
>>>>>
>>>>> For me, we shipped 1.0.x. Users who don't want big changes can use
>>>>> that version. We can agree to support 1.0.x with critical fixes.
>>>>>
>>>>> We can now start improving the code for the improvement release
>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of deprecated
>>>>> code and most of us appear to believe that classic remoting is long
>>>>> deprecated.
>>>>>
>>>>>
>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
>>>>> <ma...@aiven.io.invalid> wrote:
>>>>> >
>>>>> > > Is it really worth spending any time to support classic remoting?
>>>>> Users
>>>>> > would be better served to switch to Artery.
>>>>> >
>>>>> > That depends on what you mean by "support". If agreed on, I would
>>>>> > propose marking classic remoting as "to be removed" which would
>>>>> > also mean that the code wouldn't be touched (i.e. no feature
>>>>> > changes and no bug fixes unless extremely critical).
>>>>> >
>>>>> > I am in no way suggesting that we support remoting in a traditional
>>>>> sense
>>>>> > and again the only reason why I am even contemplating this is because
>>>>> > of the CVE's.
>>>>> >
>>>>> > > I don't think it is likely that a significant portion of Akka 2.4
>>>>> users
>>>>> > would update to Pekko any time soon if the couldn't update to Akka
>>>>> 2.5/2.6.
>>>>> >
>>>>> > I also had this impression but it's already come up a few times that
>>>>> > people still use Akka 2.4.x. They likely haven't bothered updating
>>>>> > because they didn't see a need to (quite famously Akka is quite
>>>>> stable
>>>>> > and there are cases of companies using it for years without needing
>>>>> to
>>>>> > update, as stated by Lightbend's CEO) but with the license change
>>>>> > it created a catalyst/trigger for some users to consider moving to
>>>>> Pekko.
>>>>> >
>>>>> > On this note I would also be wary in making assumptions on what
>>>>> versions
>>>>> > of Akka people happen to be using. As a corollary in the discussion
>>>>> on
>>>>> > dropping JDK 8, when we suggested that we quite quickly got feedback
>>>>> that
>>>>> > there are people still using JDK 8. And while I know all of the
>>>>> arguments
>>>>> > that you really should not be using JDK 8 can be carried over to
>>>>> Akka.
>>>>> >
>>>>> > Admittedly it's quite hard to get good feedback on what versions
>>>>> people are
>>>>> > using, just saying we should be a bit more careful in making these
>>>>> > assumptions.
>>>>> > After all, people are still using JDK 8 ;)
>>>>> >
>>>>> >
>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <nv...@gmail.com>
>>>>> wrote:
>>>>> >
>>>>> > > Akka 2.4 has been EoL since end of 2017.
>>>>> > > I don't think it is likely that a significant portion of Akka 2.4
>>>>> users
>>>>> > > would update to Pekko any time soon if the couldn't update to Akka
>>>>> 2.5/2.6.
>>>>> > >
>>>>> > > Is it really worth spending any time to support classic remoting?
>>>>> Users
>>>>> > > would be better served to switch to Artery.
>>>>> > >
>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
>>>>> > > <ma...@aiven.io.invalid> wrote:
>>>>> > >
>>>>> > > > > If you manage to move to Pekko and were still using classic
>>>>> remoting,
>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you manage the
>>>>> update
>>>>> > > > to Pekko, going to Artery TCP is a small step.
>>>>> > > >
>>>>> > > > One of the reasons why I was saying this is that I was under the
>>>>> > > impression
>>>>> > > > that there are a non-trivial amount of users still on Akka 2.4
>>>>> (not sure
>>>>> > > > why
>>>>> > > > this is the case but kerr was telling me this).
>>>>> > > >
>>>>> > > > > If people are interested in keeping it around and also opt in
>>>>> to
>>>>> > > > maintaining it (which in the first place means, making the tests
>>>>> for
>>>>> > > > it work *now*), than fine, but otherwise, being able to use it on
>>>>> > > > 1.0.x is already a big benefit.
>>>>> > > >
>>>>> > > > Since classic remoting is currently using Netty 3 which has CVE's
>>>>> > > > I don't think it's wise to encourage people that still happen to
>>>>> be
>>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't for final
>>>>> > > > version of netty 3 having CVE's then I would have no qualms
>>>>> > > > for dropping it in the 1.1.x series (in fact if there was a
>>>>> > > > hypothetical netty 3 without CVE's we would have already
>>>>> > > > updated to it in the 1.0.x series and dropped classic remoting in
>>>>> > > > 1.1.x without a heartbeat).
>>>>> > > >
>>>>> > > > I would propose that if we do accept keeping classic remoting
>>>>> > > > and updating to Netty 4, we would mark the feature to be
>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade that
>>>>> > > > part of code would be untouched unless there are critical
>>>>> > > > bugs/regressions.
>>>>> > > >
>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
>>>>> > > > johannes.rudolph@gmail.com>
>>>>> > > > wrote:
>>>>> > > >
>>>>> > > > > We definitely should remove it for 1.1.x. There's no technical
>>>>> reason
>>>>> > > > > to keep it because the newer artery TCP transport just
>>>>> supersedes it.
>>>>> > > > > As discussed before, the problem with keeping old features
>>>>> around is
>>>>> > > > > "death by a thousand cuts". The classic remoting backend is
>>>>> the prime
>>>>> > > > > example of it because it is basically unmaintained for years
>>>>> and
>>>>> > > > > there's basically no expert knowledge available about how to
>>>>> maintain
>>>>> > > > > it or how to fix the existing bugs. Testing remoting backends
>>>>> has
>>>>> > > > > shown to be very complicated and often unreliable and being
>>>>> able to
>>>>> > > > > remove this particular component from the test matrix will be
>>>>> a huge
>>>>> > > > > improvement.
>>>>> > > > >
>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
>>>>> > > > > <ma...@aiven.io.invalid> wrote:
>>>>> > > > > > I would prefer adding in netty 4 support for the classic
>>>>> transport
>>>>> > > > > > mechanism because it gives
>>>>> > > > > > people an upgrade path while allowing them to resolve the
>>>>> CVE issues.
>>>>> > > > If
>>>>> > > > > > Pekko 1.1.x didn't
>>>>> > > > > > support the classic transport mechanism then it forces Pekko
>>>>> users to
>>>>> > > > be
>>>>> > > > > > stuck on 1.0.x
>>>>> > > > > > with no easy upgrade path.
>>>>> > > > >
>>>>> > > > > We are not responsible for giving people an easy upgrade path
>>>>> when
>>>>> > > > > they were stuck on features like classic remoting which has
>>>>> basically
>>>>> > > > > been outdated for years. It is very important that, going
>>>>> forward,
>>>>> > > > > that we focus our attention on the most important features and
>>>>> make
>>>>> > > > > sure to not get stuck in maintaining old parts that we cannot
>>>>> (or
>>>>> > > > > don't want to) care for.
>>>>> > > > >
>>>>> > > > > If people are interested in keeping it around and also opt in
>>>>> to
>>>>> > > > > maintaining it (which in the first place means, making the
>>>>> tests for
>>>>> > > > > it work *now*), than fine, but otherwise, being able to use it
>>>>> on
>>>>> > > > > 1.0.x is already a big benefit.
>>>>> > > > >
>>>>> > > > > Apart from that the update path is pretty clear:
>>>>> > > > > * move to Pekko 1.0.x
>>>>> > > > > * move to Arterty TCP
>>>>> > > > > * move to Pekko 1.1.x
>>>>> > > > >
>>>>> > > > > If you manage to move to Pekko and were still using classic
>>>>> remoting,
>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you manage
>>>>> the update
>>>>> > > > > to Pekko, going to Artery TCP is a small step.
>>>>> > > > >
>>>>> > > > > Johannes
>>>>> > > > >
>>>>> > > > >
>>>>> ---------------------------------------------------------------------
>>>>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>>>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
>>>>> > > > >
>>>>> > > > >
>>>>> > > >
>>>>> > > > --
>>>>> > > >
>>>>> > > > Matthew de Detrich
>>>>> > > >
>>>>> > > > *Aiven Deutschland GmbH*
>>>>> > > >
>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
>>>>> > > >
>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
>>>>> > > >
>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>>>> > > >
>>>>> > > > *m:* +491603708037
>>>>> > > >
>>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>>>> > > >
>>>>> > >
>>>>> >
>>>>> >
>>>>> > --
>>>>> >
>>>>> > Matthew de Detrich
>>>>> >
>>>>> > *Aiven Deutschland GmbH*
>>>>> >
>>>>> > Immanuelkirchstraße 26, 10405 Berlin
>>>>> >
>>>>> > Amtsgericht Charlottenburg, HRB 209739 B
>>>>> >
>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>>>> >
>>>>> > *m:* +491603708037
>>>>> >
>>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>>>> For additional commands, e-mail: dev-help@pekko.apache.org
>>>>>
>>>>>
--
Matthew de Detrich
*Aiven Deutschland GmbH*
Immanuelkirchstraße 26, 10405 Berlin
Amtsgericht Charlottenburg, HRB 209739 B
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
*m:* +491603708037
*w:* aiven.io *e:* matthew.dedetrich@aiven.io
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by kerr <he...@gmail.com>.
And I gathered this result on reddit
[image: image.png]
何品
kerr <he...@gmail.com> 于2023年9月8日周五 23:51写道：
> I just read
>
> To switch a full cluster restart is required and any overrides for classic
> remoting need to be ported to Artery configuration. Artery has a completely
> different protocol, which means that a rolling update is not supported.
> So for Flink use case is that ok?
> 何品
>
>
> kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
>
>> I did a poll on reddit and get the result of
>> [image: image.png]
>> 何品
>>
>>
>> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
>>
>>> > Is it really worth spending any time to support classic remoting? Users
>>> would be better served to switch to Artery.
>>>
>>> I'm one of the user and the China Scala User group may not talk much on
>>> web but most of us is using the netty transport, even Lightbend say the
>>> `Artery is the best`.
>>>
>>> Another online product with large scale nodes(here) is running with Akka
>>> 2.4.x with the classic transport, and Flink is using the netty.tpc
>>> transport at least for now too.
>>>
>>> I think the community can choose when they migrate the nodes to artery
>>> is essential, especially when otherwise your cluster need a hard reboot.
>>>
>>> AFAIK, the Artery transport is not wire compatible with the classic one.
>>>
>>> I suggest:
>>> 1. Start to send PRs to open sources projects like Flink with the help
>>> for migrating to the artery transport.
>>> 2. Upgrade to the new version of Netty for bugfix and
>>> better performance.
>>> 3. Suggest all the new community move to the Artery transport.
>>>
>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
>>>
>>> Anyway, different company has different requirements, that just my 2
>>> cents.
>>>
>>> 何品
>>>
>>>
>>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
>>>
>>>> This will end up in a vote.
>>>>
>>>> For me, we shipped 1.0.x. Users who don't want big changes can use
>>>> that version. We can agree to support 1.0.x with critical fixes.
>>>>
>>>> We can now start improving the code for the improvement release
>>>> (v1.1.x or v2.0.x). Improvements include getting rid of deprecated
>>>> code and most of us appear to believe that classic remoting is long
>>>> deprecated.
>>>>
>>>>
>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
>>>> <ma...@aiven.io.invalid> wrote:
>>>> >
>>>> > > Is it really worth spending any time to support classic remoting?
>>>> Users
>>>> > would be better served to switch to Artery.
>>>> >
>>>> > That depends on what you mean by "support". If agreed on, I would
>>>> > propose marking classic remoting as "to be removed" which would
>>>> > also mean that the code wouldn't be touched (i.e. no feature
>>>> > changes and no bug fixes unless extremely critical).
>>>> >
>>>> > I am in no way suggesting that we support remoting in a traditional
>>>> sense
>>>> > and again the only reason why I am even contemplating this is because
>>>> > of the CVE's.
>>>> >
>>>> > > I don't think it is likely that a significant portion of Akka 2.4
>>>> users
>>>> > would update to Pekko any time soon if the couldn't update to Akka
>>>> 2.5/2.6.
>>>> >
>>>> > I also had this impression but it's already come up a few times that
>>>> > people still use Akka 2.4.x. They likely haven't bothered updating
>>>> > because they didn't see a need to (quite famously Akka is quite stable
>>>> > and there are cases of companies using it for years without needing to
>>>> > update, as stated by Lightbend's CEO) but with the license change
>>>> > it created a catalyst/trigger for some users to consider moving to
>>>> Pekko.
>>>> >
>>>> > On this note I would also be wary in making assumptions on what
>>>> versions
>>>> > of Akka people happen to be using. As a corollary in the discussion on
>>>> > dropping JDK 8, when we suggested that we quite quickly got feedback
>>>> that
>>>> > there are people still using JDK 8. And while I know all of the
>>>> arguments
>>>> > that you really should not be using JDK 8 can be carried over to Akka.
>>>> >
>>>> > Admittedly it's quite hard to get good feedback on what versions
>>>> people are
>>>> > using, just saying we should be a bit more careful in making these
>>>> > assumptions.
>>>> > After all, people are still using JDK 8 ;)
>>>> >
>>>> >
>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <nv...@gmail.com>
>>>> wrote:
>>>> >
>>>> > > Akka 2.4 has been EoL since end of 2017.
>>>> > > I don't think it is likely that a significant portion of Akka 2.4
>>>> users
>>>> > > would update to Pekko any time soon if the couldn't update to Akka
>>>> 2.5/2.6.
>>>> > >
>>>> > > Is it really worth spending any time to support classic remoting?
>>>> Users
>>>> > > would be better served to switch to Artery.
>>>> > >
>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
>>>> > > <ma...@aiven.io.invalid> wrote:
>>>> > >
>>>> > > > > If you manage to move to Pekko and were still using classic
>>>> remoting,
>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you manage the
>>>> update
>>>> > > > to Pekko, going to Artery TCP is a small step.
>>>> > > >
>>>> > > > One of the reasons why I was saying this is that I was under the
>>>> > > impression
>>>> > > > that there are a non-trivial amount of users still on Akka 2.4
>>>> (not sure
>>>> > > > why
>>>> > > > this is the case but kerr was telling me this).
>>>> > > >
>>>> > > > > If people are interested in keeping it around and also opt in to
>>>> > > > maintaining it (which in the first place means, making the tests
>>>> for
>>>> > > > it work *now*), than fine, but otherwise, being able to use it on
>>>> > > > 1.0.x is already a big benefit.
>>>> > > >
>>>> > > > Since classic remoting is currently using Netty 3 which has CVE's
>>>> > > > I don't think it's wise to encourage people that still happen to
>>>> be
>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't for final
>>>> > > > version of netty 3 having CVE's then I would have no qualms
>>>> > > > for dropping it in the 1.1.x series (in fact if there was a
>>>> > > > hypothetical netty 3 without CVE's we would have already
>>>> > > > updated to it in the 1.0.x series and dropped classic remoting in
>>>> > > > 1.1.x without a heartbeat).
>>>> > > >
>>>> > > > I would propose that if we do accept keeping classic remoting
>>>> > > > and updating to Netty 4, we would mark the feature to be
>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade that
>>>> > > > part of code would be untouched unless there are critical
>>>> > > > bugs/regressions.
>>>> > > >
>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
>>>> > > > johannes.rudolph@gmail.com>
>>>> > > > wrote:
>>>> > > >
>>>> > > > > We definitely should remove it for 1.1.x. There's no technical
>>>> reason
>>>> > > > > to keep it because the newer artery TCP transport just
>>>> supersedes it.
>>>> > > > > As discussed before, the problem with keeping old features
>>>> around is
>>>> > > > > "death by a thousand cuts". The classic remoting backend is the
>>>> prime
>>>> > > > > example of it because it is basically unmaintained for years and
>>>> > > > > there's basically no expert knowledge available about how to
>>>> maintain
>>>> > > > > it or how to fix the existing bugs. Testing remoting backends
>>>> has
>>>> > > > > shown to be very complicated and often unreliable and being
>>>> able to
>>>> > > > > remove this particular component from the test matrix will be a
>>>> huge
>>>> > > > > improvement.
>>>> > > > >
>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
>>>> > > > > <ma...@aiven.io.invalid> wrote:
>>>> > > > > > I would prefer adding in netty 4 support for the classic
>>>> transport
>>>> > > > > > mechanism because it gives
>>>> > > > > > people an upgrade path while allowing them to resolve the CVE
>>>> issues.
>>>> > > > If
>>>> > > > > > Pekko 1.1.x didn't
>>>> > > > > > support the classic transport mechanism then it forces Pekko
>>>> users to
>>>> > > > be
>>>> > > > > > stuck on 1.0.x
>>>> > > > > > with no easy upgrade path.
>>>> > > > >
>>>> > > > > We are not responsible for giving people an easy upgrade path
>>>> when
>>>> > > > > they were stuck on features like classic remoting which has
>>>> basically
>>>> > > > > been outdated for years. It is very important that, going
>>>> forward,
>>>> > > > > that we focus our attention on the most important features and
>>>> make
>>>> > > > > sure to not get stuck in maintaining old parts that we cannot
>>>> (or
>>>> > > > > don't want to) care for.
>>>> > > > >
>>>> > > > > If people are interested in keeping it around and also opt in to
>>>> > > > > maintaining it (which in the first place means, making the
>>>> tests for
>>>> > > > > it work *now*), than fine, but otherwise, being able to use it
>>>> on
>>>> > > > > 1.0.x is already a big benefit.
>>>> > > > >
>>>> > > > > Apart from that the update path is pretty clear:
>>>> > > > > * move to Pekko 1.0.x
>>>> > > > > * move to Arterty TCP
>>>> > > > > * move to Pekko 1.1.x
>>>> > > > >
>>>> > > > > If you manage to move to Pekko and were still using classic
>>>> remoting,
>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you manage the
>>>> update
>>>> > > > > to Pekko, going to Artery TCP is a small step.
>>>> > > > >
>>>> > > > > Johannes
>>>> > > > >
>>>> > > > >
>>>> ---------------------------------------------------------------------
>>>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
>>>> > > > >
>>>> > > > >
>>>> > > >
>>>> > > > --
>>>> > > >
>>>> > > > Matthew de Detrich
>>>> > > >
>>>> > > > *Aiven Deutschland GmbH*
>>>> > > >
>>>> > > > Immanuelkirchstraße 26, 10405 Berlin
>>>> > > >
>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
>>>> > > >
>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>>> > > >
>>>> > > > *m:* +491603708037
>>>> > > >
>>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>>> > > >
>>>> > >
>>>> >
>>>> >
>>>> > --
>>>> >
>>>> > Matthew de Detrich
>>>> >
>>>> > *Aiven Deutschland GmbH*
>>>> >
>>>> > Immanuelkirchstraße 26, 10405 Berlin
>>>> >
>>>> > Amtsgericht Charlottenburg, HRB 209739 B
>>>> >
>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>>> >
>>>> > *m:* +491603708037
>>>> >
>>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>>> For additional commands, e-mail: dev-help@pekko.apache.org
>>>>
>>>>
Re: Remove the old classical transport or just upgrade it to netty v4?
Posted by kerr <he...@gmail.com>.
I just read
To switch a full cluster restart is required and any overrides for classic
remoting need to be ported to Artery configuration. Artery has a completely
different protocol, which means that a rolling update is not supported.
So for Flink use case is that ok?
何品
kerr <he...@gmail.com> 于2023年9月8日周五 17:10写道：
> I did a poll on reddit and get the result of
> [image: image.png]
> 何品
>
>
> kerr <he...@gmail.com> 于2023年8月6日周日 00:33写道：
>
>> > Is it really worth spending any time to support classic remoting? Users
>> would be better served to switch to Artery.
>>
>> I'm one of the user and the China Scala User group may not talk much on
>> web but most of us is using the netty transport, even Lightbend say the
>> `Artery is the best`.
>>
>> Another online product with large scale nodes(here) is running with Akka
>> 2.4.x with the classic transport, and Flink is using the netty.tpc
>> transport at least for now too.
>>
>> I think the community can choose when they migrate the nodes to artery is
>> essential, especially when otherwise your cluster need a hard reboot.
>>
>> AFAIK, the Artery transport is not wire compatible with the classic one.
>>
>> I suggest:
>> 1. Start to send PRs to open sources projects like Flink with the help
>> for migrating to the artery transport.
>> 2. Upgrade to the new version of Netty for bugfix and better performance.
>> 3. Suggest all the new community move to the Artery transport.
>>
>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x.
>>
>> Anyway, different company has different requirements, that just my 2
>> cents.
>>
>> 何品
>>
>>
>> PJ Fanning <fa...@gmail.com> 于2023年8月2日周三 17:29写道：
>>
>>> This will end up in a vote.
>>>
>>> For me, we shipped 1.0.x. Users who don't want big changes can use
>>> that version. We can agree to support 1.0.x with critical fixes.
>>>
>>> We can now start improving the code for the improvement release
>>> (v1.1.x or v2.0.x). Improvements include getting rid of deprecated
>>> code and most of us appear to believe that classic remoting is long
>>> deprecated.
>>>
>>>
>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich
>>> <ma...@aiven.io.invalid> wrote:
>>> >
>>> > > Is it really worth spending any time to support classic remoting?
>>> Users
>>> > would be better served to switch to Artery.
>>> >
>>> > That depends on what you mean by "support". If agreed on, I would
>>> > propose marking classic remoting as "to be removed" which would
>>> > also mean that the code wouldn't be touched (i.e. no feature
>>> > changes and no bug fixes unless extremely critical).
>>> >
>>> > I am in no way suggesting that we support remoting in a traditional
>>> sense
>>> > and again the only reason why I am even contemplating this is because
>>> > of the CVE's.
>>> >
>>> > > I don't think it is likely that a significant portion of Akka 2.4
>>> users
>>> > would update to Pekko any time soon if the couldn't update to Akka
>>> 2.5/2.6.
>>> >
>>> > I also had this impression but it's already come up a few times that
>>> > people still use Akka 2.4.x. They likely haven't bothered updating
>>> > because they didn't see a need to (quite famously Akka is quite stable
>>> > and there are cases of companies using it for years without needing to
>>> > update, as stated by Lightbend's CEO) but with the license change
>>> > it created a catalyst/trigger for some users to consider moving to
>>> Pekko.
>>> >
>>> > On this note I would also be wary in making assumptions on what
>>> versions
>>> > of Akka people happen to be using. As a corollary in the discussion on
>>> > dropping JDK 8, when we suggested that we quite quickly got feedback
>>> that
>>> > there are people still using JDK 8. And while I know all of the
>>> arguments
>>> > that you really should not be using JDK 8 can be carried over to Akka.
>>> >
>>> > Admittedly it's quite hard to get good feedback on what versions
>>> people are
>>> > using, just saying we should be a bit more careful in making these
>>> > assumptions.
>>> > After all, people are still using JDK 8 ;)
>>> >
>>> >
>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar <nv...@gmail.com>
>>> wrote:
>>> >
>>> > > Akka 2.4 has been EoL since end of 2017.
>>> > > I don't think it is likely that a significant portion of Akka 2.4
>>> users
>>> > > would update to Pekko any time soon if the couldn't update to Akka
>>> 2.5/2.6.
>>> > >
>>> > > Is it really worth spending any time to support classic remoting?
>>> Users
>>> > > would be better served to switch to Artery.
>>> > >
>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich
>>> > > <ma...@aiven.io.invalid> wrote:
>>> > >
>>> > > > > If you manage to move to Pekko and were still using classic
>>> remoting,
>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you manage the
>>> update
>>> > > > to Pekko, going to Artery TCP is a small step.
>>> > > >
>>> > > > One of the reasons why I was saying this is that I was under the
>>> > > impression
>>> > > > that there are a non-trivial amount of users still on Akka 2.4
>>> (not sure
>>> > > > why
>>> > > > this is the case but kerr was telling me this).
>>> > > >
>>> > > > > If people are interested in keeping it around and also opt in to
>>> > > > maintaining it (which in the first place means, making the tests
>>> for
>>> > > > it work *now*), than fine, but otherwise, being able to use it on
>>> > > > 1.0.x is already a big benefit.
>>> > > >
>>> > > > Since classic remoting is currently using Netty 3 which has CVE's
>>> > > > I don't think it's wise to encourage people that still happen to be
>>> > > > using classic remoting to stay on 1.0.x. If it wasn't for final
>>> > > > version of netty 3 having CVE's then I would have no qualms
>>> > > > for dropping it in the 1.1.x series (in fact if there was a
>>> > > > hypothetical netty 3 without CVE's we would have already
>>> > > > updated to it in the 1.0.x series and dropped classic remoting in
>>> > > > 1.1.x without a heartbeat).
>>> > > >
>>> > > > I would propose that if we do accept keeping classic remoting
>>> > > > and updating to Netty 4, we would mark the feature to be
>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade that
>>> > > > part of code would be untouched unless there are critical
>>> > > > bugs/regressions.
>>> > > >
>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph <
>>> > > > johannes.rudolph@gmail.com>
>>> > > > wrote:
>>> > > >
>>> > > > > We definitely should remove it for 1.1.x. There's no technical
>>> reason
>>> > > > > to keep it because the newer artery TCP transport just
>>> supersedes it.
>>> > > > > As discussed before, the problem with keeping old features
>>> around is
>>> > > > > "death by a thousand cuts". The classic remoting backend is the
>>> prime
>>> > > > > example of it because it is basically unmaintained for years and
>>> > > > > there's basically no expert knowledge available about how to
>>> maintain
>>> > > > > it or how to fix the existing bugs. Testing remoting backends has
>>> > > > > shown to be very complicated and often unreliable and being able
>>> to
>>> > > > > remove this particular component from the test matrix will be a
>>> huge
>>> > > > > improvement.
>>> > > > >
>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich
>>> > > > > <ma...@aiven.io.invalid> wrote:
>>> > > > > > I would prefer adding in netty 4 support for the classic
>>> transport
>>> > > > > > mechanism because it gives
>>> > > > > > people an upgrade path while allowing them to resolve the CVE
>>> issues.
>>> > > > If
>>> > > > > > Pekko 1.1.x didn't
>>> > > > > > support the classic transport mechanism then it forces Pekko
>>> users to
>>> > > > be
>>> > > > > > stuck on 1.0.x
>>> > > > > > with no easy upgrade path.
>>> > > > >
>>> > > > > We are not responsible for giving people an easy upgrade path
>>> when
>>> > > > > they were stuck on features like classic remoting which has
>>> basically
>>> > > > > been outdated for years. It is very important that, going
>>> forward,
>>> > > > > that we focus our attention on the most important features and
>>> make
>>> > > > > sure to not get stuck in maintaining old parts that we cannot (or
>>> > > > > don't want to) care for.
>>> > > > >
>>> > > > > If people are interested in keeping it around and also opt in to
>>> > > > > maintaining it (which in the first place means, making the tests
>>> for
>>> > > > > it work *now*), than fine, but otherwise, being able to use it on
>>> > > > > 1.0.x is already a big benefit.
>>> > > > >
>>> > > > > Apart from that the update path is pretty clear:
>>> > > > > * move to Pekko 1.0.x
>>> > > > > * move to Arterty TCP
>>> > > > > * move to Pekko 1.1.x
>>> > > > >
>>> > > > > If you manage to move to Pekko and were still using classic
>>> remoting,
>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you manage the
>>> update
>>> > > > > to Pekko, going to Artery TCP is a small step.
>>> > > > >
>>> > > > > Johannes
>>> > > > >
>>> > > > >
>>> ---------------------------------------------------------------------
>>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
>>> > > > >
>>> > > > >
>>> > > >
>>> > > > --
>>> > > >
>>> > > > Matthew de Detrich
>>> > > >
>>> > > > *Aiven Deutschland GmbH*
>>> > > >
>>> > > > Immanuelkirchstraße 26, 10405 Berlin
>>> > > >
>>> > > > Amtsgericht Charlottenburg, HRB 209739 B
>>> > > >
>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>> > > >
>>> > > > *m:* +491603708037
>>> > > >
>>> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>> > > >
>>> > >
>>> >
>>> >
>>> > --
>>> >
>>> > Matthew de Detrich
>>> >
>>> > *Aiven Deutschland GmbH*
>>> >
>>> > Immanuelkirchstraße 26, 10405 Berlin
>>> >
>>> > Amtsgericht Charlottenburg, HRB 209739 B
>>> >
>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>> >
>>> > *m:* +491603708037
>>> >
>>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>> For additional commands, e-mail: dev-help@pekko.apache.org
>>>
>>>