Re: [DISCUSS] require Java 11 for Pekko doc builds

[Arnout Engelen <en...@apache.org>]

On Wed, Jan 31, 2024 at 8:44 AM Matthew de Detrich
<ma...@aiven.io.invalid> wrote:

> My main concern is that since we are still supporting JDK 8, building the
> docs with JDK 11 can bring about actual legitimate usability concerns in
> terms
> of consistency because we still have to build the main source code with JDK
> 8
> but with docs we have to build with JDK 11 and so if we do this change we
> have a situation where if someone starts sbt with JDK 8 (because they are
> developing normally with the source code) and then do docs/paradox they
> will
> get hit with runtime bytecode mismatch errors.
>
> Now this can be solved with the the source and target flags i.e.
>
> javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
>
> Which instructs the JVM to target JDK 1.8 but there are limitations here
> i.e.
> the fact that it doesn't work sun.misc.unsafe[1].
>

Luckily, only the main pekko repo uses sun.misc.Unsafe (pekko-http
indirectly through pekko.util.Unsafe).

The main pekko repo *already* supports building with Java 11 (and requires
it for releases) and targeting Java 8. While it is true it uses fairly
horrible hacks to achieve this while also using sun.misc.Unsafe, this is
already in place, and other components should be fine with the 'regular'
flags for targeting Java 8.


> There is also the other issue which is well, the compiled docs would be
> slightly
> misleading in the sense that they would link against the JDK 11 docs when
> in reality
> our projects support JDK 8 and these docs do actually differ (JDK8 docs
> differ
> to JDK11 docs).


I don't think this is an issue: while it is true we support Java 8, I'd say
users are more than likely to be on more recent Java versions, in which
case the links to more recent Java versions will be more accurate.


> I mentioned this somewhere (can't remember where) but it would actually be
> ideal
> if another release of sbt-paradox from the older series i.e. 0.9.x so it
> can be
> deployed to the newer repositories.
>

I think we should consider requiring Java 11 for all development: the
cost/complexity of requiring all our build tools to keep supporting Java 8
will only increase, and IMHO is unhealthy to the wider ecosystem. The
cost/complexity of building with Java 11 and targeting Java 8 is low when
using the well-understood compiler flags, except when also using
sun.misc.Unsafe, but we have already incurred that cost for that case
anyway.

The main remaining risk I see would be that we'd unintentionally accept an
update of a runtime dependency that drops support for Java 8.

While it is true the use of Java 8 sadly still seems widespread (
https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
mentions 33%), I'm sceptical the same holds for developer environments.


Kind regards,

Arnout


>
> [1]:
>
> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
>
>
> On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <fa...@apache.org> wrote:
>
> > We have a slightly untidy sbt plugin setup in our various Pekko repos,
> > where we force the use of old Paradox plugins so that we can use Java
> > 8 for doc building.
> >
> > Now that we have the 1.0.x releases done, can we consider switching to
> > the latest Paradox plugins and live with the fact that that means we
> > need to use Java 11 for doc builds?
> >
> > Matthias Kurz has been publishing some new versions of these plugins
> > because the versions that we use are not in Maven Central meaning that
> > we are dependent on the scala-sbt repository.
> >
> > Any objections to us at least starting with the Pekko repos where we
> > have 1.0.x branches? We can update the main branches in these repos to
> > use the latest Paradox plugins and update the build docs.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
> >
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Alexanderufer 3-7, 10117 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>


-- 
Arnout Engelen
ASF Security Response
Committer on Apache Pekko
Committer on NixOS
Independent Open Source consultant

Re: [DISCUSS] require Java 11 for Pekko doc builds

[Matthew de Detrich <ma...@aiven.io.INVALID>]

> I guess the lesson is that when we encounter such roadblocks, we should
create issues describing it, and link to those wherever we are held back by
it.

Agreed, I don't have time to find/remember it now but I might look at it
later if needed

> What problems/fixes do you mean here?

https://github.com/scala/scala/pull/10543

On Sat, Feb 10, 2024 at 11:43 PM Arnout Engelen <en...@apache.org> wrote:

> On Sat, Feb 10, 2024 at 12:25 PM Matthew de Detrich
> <ma...@aiven.io.invalid> wrote:
>
> > > I agree that would be nice, but is it a problem? Even if we could
> specify
> > the JDK version to link against for the docs, we should perhaps link to
> > Java 11 (or even Java 21): while we support Java 8, I think we should
> > encourage users to use newer versions.
> >
> > I don't know, I remember we tried messing around with this a year ago
> > and we ended up reverting it for one reason or another (I would have to
> go
> > back and find out now).
>
>
> I tried to find what you're referring to, but the main Pekko repo already
> uses Java 11 for publishing the docs (
>
> https://github.com/apache/incubator-pekko/blob/main/.github/workflows/publish-1.0-docs.yml
> )
> and I didn't find a relevant revert in the history. Neither for pekko-http
> (which seems to build with Java 8 from the start). Also I don't see any
> references from the docs to the JDK at all (e.g.
>
> https://pekko.apache.org/japi/pekko/current/org/apache/pekko/actor/testkit/typed/CapturedLogEvent.html
> and
>
> https://pekko.apache.org/japi/pekko-http/current/org/apache/pekko/http/javadsl/Http.html
> mention java.util.Optional but do not link to it). What problem with links
> from API docs are we talking about, exactly?
>
> I guess the lesson is that when we encounter such roadblocks, we should
> create issues describing it, and link to those wherever we are held back by
> it.
>
> A stronger counter argument is that it's easier for developers to
> > understand,
> > mixing and matching different JDK's for different tasks is actually
> > somewhat
> > confusing whereas now it's simply "build with JDK 8 for everything"
> (aside
> > from Pekko core) which is consistent and simple, even if it's outdated
> >
> (although
> > this can be solved with release flags especially with recent fixes
> > that have come in via Scala 2.11/2.12 but still waiting for 3.3.2)
> >
>
> What problems/fixes do you mean here?
>
>
> Kind regards,
>
> Arnout
>
>
> > On Sat, Feb 10, 2024 at 10:04 PM Arnout Engelen <en...@apache.org>
> > wrote:
> >
> > > On Sat, Feb 10, 2024 at 11:50 AM Matthew de Detrich
> > > <ma...@aiven.io.invalid> wrote:
> > >
> > > > > OK! The API docs would link to the Java version you're building
> with
> > I
> > > > suppose, but I don't see that as a problem: we can't predict what JDK
> > > users
> > > > will be using anyway.
> > > >
> > > > Yeah this is what the main issue is, I would be more amicable to
> > building
> > > > the docs on JDK 11 if there was some way to designate the JDK version
> > to
> > > > link against for the docs.
> > > >
> > >
> > > I agree that would be nice, but is it a problem? Even if we could
> specify
> > > the JDK version to link against for the docs, we should perhaps link to
> > > Java 11 (or even Java 21): while we support Java 8, I think we should
> > > encourage users to use newer versions.
> > >
> > >
> > > Kind regards,
> > >
> > > Arnout
> > >
> > >
> > > >
> > > > On Sat, Feb 10, 2024 at 9:17 PM Arnout Engelen <en...@apache.org>
> > > wrote:
> > > >
> > > > > On Sat, Feb 10, 2024 at 10:51 AM Matthew de Detrich
> > > > > <ma...@aiven.io.invalid> wrote:
> > > > >
> > > > > > > It's a trade-off between the risks of developing on more recent
> > > JDKs
> > > > > > while targeting Java 8, and making maintenance more painful by
> > > > requiring
> > > > > > development tooling to keep supporting Java 8. I think we should
> > take
> > > > the
> > > > > > risk.
> > > > > >
> > > > > > Sure if someone wants to go ahead then they are more than welcome
> > to
> > > > try,
> > > > > > just need to be careful that both the paradox docs and binary
> > > artifacts
> > > > > are
> > > > > > exactly the same before and after.
> > > > >
> > > > >
> > > > > OK! The API docs would link to the Java version you're building
> with
> > I
> > > > > suppose, but I don't see that as a problem: we can't predict what
> JDK
> > > > users
> > > > > will be using anyway.
> > > > >
> > > > > From my end we still need to republish paradox 0.9.x on maven just
> > due
> > > to
> > > > > > the 1.0.x branches so I will continue to get that done.
> > > > > >
> > > > >
> > > > > OK
> > > > >
> > > > >
> > > > > > I would also posit that a nicer solution to this would also be
> > > getting
> > > > > > sbt-multi-release-jar[1] to work because currently we have a more
> > > > > > recurring issue where we want to use features in newer JDK's
> (such
> > as
> > > > > > loom/virtual thread for JDK 21) but we can't while supporting
> JDK 8
> > > > > without
> > > > > > a lot of hackery (if it's even possible).
> > > > > > Unfortunately I am hitting some fundamental issues[2] on trying
> to
> > > get
> > > > it
> > > > > > to work
> > > > > >
> > > > >
> > > > > I agree that might be interesting but require a lot more r&d to
> find
> > > out
> > > > if
> > > > > it's viable.
> > > > >
> > > > >
> > > > > Kind regards,
> > > > >
> > > > > Arnout
> > > > >
> > > > >
> > > > > >
> > > > > > 1: https://github.com/sbt/sbt-multi-release-jar and
> > > > > >
> > > https://github.com/sbt/sbt/discussions/7174#discussioncomment-5288982
> > > > > > 2: https://github.com/sbt/sbt-multi-release-jar/issues/24
> > > > > >
> > > > > > On Sat, Feb 10, 2024 at 8:45 PM Arnout Engelen <
> engelen@apache.org
> > >
> > > > > wrote:
> > > > > >
> > > > > > > On Sat, Feb 10, 2024 at 8:57 AM Matthew de Detrich
> > > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > >
> > > > > > > > > I think we should consider requiring Java 11 for all
> > > development:
> > > > > the
> > > > > > > > cost/complexity of requiring all our build tools to keep
> > > supporting
> > > > > > Java
> > > > > > > 8
> > > > > > > > will only increase, and IMHO is unhealthy to the wider
> > ecosystem.
> > > > The
> > > > > > > > cost/complexity of building with Java 11 and targeting Java 8
> > is
> > > > low
> > > > > > when
> > > > > > > > using the well-understood compiler flags, except when also
> > using
> > > > > > > > sun.misc.Unsafe, but we have already incurred that cost for
> > that
> > > > case
> > > > > > > > anyway.
> > > > > > > >
> > > > > > > > the Pekko 1.0.x series is deliberately designed to
> > > > > > > > have minimal intrusive changes merged into it and so for this
> > > > reason
> > > > > > its
> > > > > > > > ideal
> > > > > > > > to have the current build setup remain the same, which means
> > both
> > > > > > > building
> > > > > > > > with JDK 8 (or JDK 11 + JDK 8 workaround for pekko core) and
> > > > paradox
> > > > > > > 0.9.x.
> > > > > > > >
> > > > > > >
> > > > > > > That makes sense, I'm fine with requiring JDK 11 only for
> > building
> > > > > Pekko
> > > > > > > 1.1.x onwards.
> > > > > > >
> > > > > > >
> > > > > > > > My personal preference is to remain having JDK 1.8 as the
> > > preferred
> > > > > one
> > > > > > > > for development as it is now and rather instead spend our
> > efforts
> > > > to
> > > > > > > > strongly push for dropping JDK 1.8 altogether for Pekko
> 2.0.x.
> > > This
> > > > > > would
> > > > > > > > allow
> > > > > > > > us to remove a lot of these hacks which would go a great deal
> > in
> > > > > > > > simplifying the build.
> > > > > > > >
> > > > > > >
> > > > > > > I'm all for dropping Java 8 support completely on Pekko 2.0.x,
> > and
> > > > > indeed
> > > > > > > it would allow us to simplify the build.
> > > > > > >
> > > > > > > I think requiring Java 11 for building Pekko 1.1.x would still
> be
> > > > good.
> > > > > > By
> > > > > > > the time Pekko 2.0.x happens, we might want to build with even
> > > newer
> > > > > > > versions of Java, so most of the work we do now to build with
> > Java
> > > 11
> > > > > and
> > > > > > > target Java 8 will continue to be valuable then as well.
> > > > > > >
> > > > > > > > While it is true the use of Java 8 sadly still seems
> > widespread (
> > > > > > > >
> > > > >
> > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > > > > > mentions 33%), I'm sceptical the same holds for developer
> > > > > environments.
> > > > > > > >
> > > > > > > > While I do broadly agree this is accurate, it's still
> > preferable
> > > > for
> > > > > > > > developers
> > > > > > > > to use the lowest common denominator JDK that the project
> uses
> > > just
> > > > > for
> > > > > > > > the sakes of lowering surprises especially when it comes to
> > > > releases,
> > > > > > > this
> > > > > > > > is what I do. There are plenty of solutions that allow you to
> > hot
> > > > > swap
> > > > > > > > JDK's
> > > > > > > > on local machines and on this note I was actually considering
> > > > adding
> > > > > > > > .java-version files like in sbt-multi-release-jar[2] which
> > forces
> > > > the
> > > > > > > > project
> > > > > > > > to be loaded with a specific JDK. This may be a bit heavy
> > handed
> > > > but
> > > > > > > > it does a good job of enforcing expectations.
> > > > > > >
> > > > > > >
> > > > > > > It's a trade-off between the risks of developing on more recent
> > > JDKs
> > > > > > while
> > > > > > > targeting Java 8, and making maintenance more painful by
> > requiring
> > > > > > > development tooling to keep supporting Java 8. I think we
> should
> > > take
> > > > > the
> > > > > > > risk.
> > > > > > >
> > > > > > >
> > > > > > > Kind regards,
> > > > > > >
> > > > > > > Arnout
> > > > > > >
> > > > > > >
> > > > > > > > On Fri, Feb 9, 2024 at 9:57 PM Arnout Engelen <
> > > engelen@apache.org>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > On Wed, Jan 31, 2024 at 8:44 AM Matthew de Detrich
> > > > > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > > > >
> > > > > > > > > > My main concern is that since we are still supporting JDK
> > 8,
> > > > > > building
> > > > > > > > the
> > > > > > > > > > docs with JDK 11 can bring about actual legitimate
> > usability
> > > > > > concerns
> > > > > > > > in
> > > > > > > > > > terms
> > > > > > > > > > of consistency because we still have to build the main
> > source
> > > > > code
> > > > > > > with
> > > > > > > > > JDK
> > > > > > > > > > 8
> > > > > > > > > > but with docs we have to build with JDK 11 and so if we
> do
> > > this
> > > > > > > change
> > > > > > > > we
> > > > > > > > > > have a situation where if someone starts sbt with JDK 8
> > > > (because
> > > > > > they
> > > > > > > > are
> > > > > > > > > > developing normally with the source code) and then do
> > > > > docs/paradox
> > > > > > > they
> > > > > > > > > > will
> > > > > > > > > > get hit with runtime bytecode mismatch errors.
> > > > > > > > > >
> > > > > > > > > > Now this can be solved with the the source and target
> flags
> > > > i.e.
> > > > > > > > > >
> > > > > > > > > > javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
> > > > > > > > > >
> > > > > > > > > > Which instructs the JVM to target JDK 1.8 but there are
> > > > > limitations
> > > > > > > > here
> > > > > > > > > > i.e.
> > > > > > > > > > the fact that it doesn't work sun.misc.unsafe[1].
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Luckily, only the main pekko repo uses sun.misc.Unsafe
> > > > (pekko-http
> > > > > > > > > indirectly through pekko.util.Unsafe).
> > > > > > > > >
> > > > > > > > > The main pekko repo *already* supports building with Java
> 11
> > > (and
> > > > > > > > requires
> > > > > > > > > it for releases) and targeting Java 8. While it is true it
> > uses
> > > > > > fairly
> > > > > > > > > horrible hacks to achieve this while also using
> > > sun.misc.Unsafe,
> > > > > this
> > > > > > > is
> > > > > > > > > already in place, and other components should be fine with
> > the
> > > > > > > 'regular'
> > > > > > > > > flags for targeting Java 8.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > There is also the other issue which is well, the compiled
> > > docs
> > > > > > would
> > > > > > > be
> > > > > > > > > > slightly
> > > > > > > > > > misleading in the sense that they would link against the
> > JDK
> > > 11
> > > > > > docs
> > > > > > > > when
> > > > > > > > > > in reality
> > > > > > > > > > our projects support JDK 8 and these docs do actually
> > differ
> > > > > (JDK8
> > > > > > > docs
> > > > > > > > > > differ
> > > > > > > > > > to JDK11 docs).
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I don't think this is an issue: while it is true we support
> > > Java
> > > > 8,
> > > > > > I'd
> > > > > > > > say
> > > > > > > > > users are more than likely to be on more recent Java
> > versions,
> > > in
> > > > > > which
> > > > > > > > > case the links to more recent Java versions will be more
> > > > accurate.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > I mentioned this somewhere (can't remember where) but it
> > > would
> > > > > > > actually
> > > > > > > > > be
> > > > > > > > > > ideal
> > > > > > > > > > if another release of sbt-paradox from the older series
> > i.e.
> > > > > 0.9.x
> > > > > > so
> > > > > > > > it
> > > > > > > > > > can be
> > > > > > > > > > deployed to the newer repositories.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I think we should consider requiring Java 11 for all
> > > development:
> > > > > the
> > > > > > > > > cost/complexity of requiring all our build tools to keep
> > > > supporting
> > > > > > > Java
> > > > > > > > 8
> > > > > > > > > will only increase, and IMHO is unhealthy to the wider
> > > ecosystem.
> > > > > The
> > > > > > > > > cost/complexity of building with Java 11 and targeting
> Java 8
> > > is
> > > > > low
> > > > > > > when
> > > > > > > > > using the well-understood compiler flags, except when also
> > > using
> > > > > > > > > sun.misc.Unsafe, but we have already incurred that cost for
> > > that
> > > > > case
> > > > > > > > > anyway.
> > > > > > > > >
> > > > > > > > > The main remaining risk I see would be that we'd
> > > unintentionally
> > > > > > accept
> > > > > > > > an
> > > > > > > > > update of a runtime dependency that drops support for Java
> 8.
> > > > > > > > >
> > > > > > > > > While it is true the use of Java 8 sadly still seems
> > > widespread (
> > > > > > > > >
> > > > > >
> > > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > > > > > > mentions 33%), I'm sceptical the same holds for developer
> > > > > > environments.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Kind regards,
> > > > > > > > >
> > > > > > > > > Arnout
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > [1]:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <
> > > > fanningpj@apache.org
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > We have a slightly untidy sbt plugin setup in our
> various
> > > > Pekko
> > > > > > > > repos,
> > > > > > > > > > > where we force the use of old Paradox plugins so that
> we
> > > can
> > > > > use
> > > > > > > Java
> > > > > > > > > > > 8 for doc building.
> > > > > > > > > > >
> > > > > > > > > > > Now that we have the 1.0.x releases done, can we
> consider
> > > > > > switching
> > > > > > > > to
> > > > > > > > > > > the latest Paradox plugins and live with the fact that
> > that
> > > > > means
> > > > > > > we
> > > > > > > > > > > need to use Java 11 for doc builds?
> > > > > > > > > > >
> > > > > > > > > > > Matthias Kurz has been publishing some new versions of
> > > these
> > > > > > > plugins
> > > > > > > > > > > because the versions that we use are not in Maven
> Central
> > > > > meaning
> > > > > > > > that
> > > > > > > > > > > we are dependent on the scala-sbt repository.
> > > > > > > > > > >
> > > > > > > > > > > Any objections to us at least starting with the Pekko
> > repos
> > > > > where
> > > > > > > we
> > > > > > > > > > > have 1.0.x branches? We can update the main branches in
> > > these
> > > > > > repos
> > > > > > > > to
> > > > > > > > > > > use the latest Paradox plugins and update the build
> docs.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> > dev-help@pekko.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > >
> > > > > > > > > > Matthew de Detrich
> > > > > > > > > >
> > > > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > > > >
> > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > > > >
> > > > > > > > > > Alexanderufer 3-7, 10117 Berlin
> > > > > > > > > >
> > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > > > >
> > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > > > >
> > > > > > > > > > *m:* +491603708037
> > > > > > > > > >
> > > > > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Arnout Engelen
> > > > > > > > > ASF Security Response
> > > > > > > > > Committer on Apache Pekko
> > > > > > > > > Committer on NixOS
> > > > > > > > > Independent Open Source consultant
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Matthew de Detrich
> > > > > > > >
> > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > >
> > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > >
> > > > > > > > Alexanderufer 3-7, 10117 Berlin
> > > > > > > >
> > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > >
> > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > >
> > > > > > > > *m:* +491603708037
> > > > > > > >
> > > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Arnout Engelen
> > > > > > > ASF Security Response
> > > > > > > Committer on Apache Pekko
> > > > > > > Committer on NixOS
> > > > > > > Independent Open Source consultant
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Matthew de Detrich
> > > > > >
> > > > > > *Aiven Deutschland GmbH*
> > > > > >
> > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > >
> > > > > > Alexanderufer 3-7, 10117 Berlin
> > > > > >
> > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > >
> > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > >
> > > > > > *m:* +491603708037
> > > > > >
> > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Arnout Engelen
> > > > > ASF Security Response
> > > > > Committer on Apache Pekko
> > > > > Committer on NixOS
> > > > > Independent Open Source consultant
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Matthew de Detrich
> > > >
> > > > *Aiven Deutschland GmbH*
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Alexanderufer 3-7, 10117 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > *m:* +491603708037
> > > >
> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >
> > >
> > >
> > > --
> > > Arnout Engelen
> > > ASF Security Response
> > > Committer on Apache Pekko
> > > Committer on NixOS
> > > Independent Open Source consultant
> > >
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Alexanderufer 3-7, 10117 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>
>
> --
> Arnout Engelen
> ASF Security Response
> Committer on Apache Pekko
> Committer on NixOS
> Independent Open Source consultant
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Alexanderufer 3-7, 10117 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] require Java 11 for Pekko doc builds

[Arnout Engelen <en...@apache.org>]

On Sat, Feb 10, 2024 at 12:25 PM Matthew de Detrich
<ma...@aiven.io.invalid> wrote:

> > I agree that would be nice, but is it a problem? Even if we could specify
> the JDK version to link against for the docs, we should perhaps link to
> Java 11 (or even Java 21): while we support Java 8, I think we should
> encourage users to use newer versions.
>
> I don't know, I remember we tried messing around with this a year ago
> and we ended up reverting it for one reason or another (I would have to go
> back and find out now).


I tried to find what you're referring to, but the main Pekko repo already
uses Java 11 for publishing the docs (
https://github.com/apache/incubator-pekko/blob/main/.github/workflows/publish-1.0-docs.yml)
and I didn't find a relevant revert in the history. Neither for pekko-http
(which seems to build with Java 8 from the start). Also I don't see any
references from the docs to the JDK at all (e.g.
https://pekko.apache.org/japi/pekko/current/org/apache/pekko/actor/testkit/typed/CapturedLogEvent.html
and
https://pekko.apache.org/japi/pekko-http/current/org/apache/pekko/http/javadsl/Http.html
mention java.util.Optional but do not link to it). What problem with links
from API docs are we talking about, exactly?

I guess the lesson is that when we encounter such roadblocks, we should
create issues describing it, and link to those wherever we are held back by
it.

A stronger counter argument is that it's easier for developers to
> understand,
> mixing and matching different JDK's for different tasks is actually
> somewhat
> confusing whereas now it's simply "build with JDK 8 for everything" (aside
> from Pekko core) which is consistent and simple, even if it's outdated
>
(although
> this can be solved with release flags especially with recent fixes
> that have come in via Scala 2.11/2.12 but still waiting for 3.3.2)
>

What problems/fixes do you mean here?


Kind regards,

Arnout


> On Sat, Feb 10, 2024 at 10:04 PM Arnout Engelen <en...@apache.org>
> wrote:
>
> > On Sat, Feb 10, 2024 at 11:50 AM Matthew de Detrich
> > <ma...@aiven.io.invalid> wrote:
> >
> > > > OK! The API docs would link to the Java version you're building with
> I
> > > suppose, but I don't see that as a problem: we can't predict what JDK
> > users
> > > will be using anyway.
> > >
> > > Yeah this is what the main issue is, I would be more amicable to
> building
> > > the docs on JDK 11 if there was some way to designate the JDK version
> to
> > > link against for the docs.
> > >
> >
> > I agree that would be nice, but is it a problem? Even if we could specify
> > the JDK version to link against for the docs, we should perhaps link to
> > Java 11 (or even Java 21): while we support Java 8, I think we should
> > encourage users to use newer versions.
> >
> >
> > Kind regards,
> >
> > Arnout
> >
> >
> > >
> > > On Sat, Feb 10, 2024 at 9:17 PM Arnout Engelen <en...@apache.org>
> > wrote:
> > >
> > > > On Sat, Feb 10, 2024 at 10:51 AM Matthew de Detrich
> > > > <ma...@aiven.io.invalid> wrote:
> > > >
> > > > > > It's a trade-off between the risks of developing on more recent
> > JDKs
> > > > > while targeting Java 8, and making maintenance more painful by
> > > requiring
> > > > > development tooling to keep supporting Java 8. I think we should
> take
> > > the
> > > > > risk.
> > > > >
> > > > > Sure if someone wants to go ahead then they are more than welcome
> to
> > > try,
> > > > > just need to be careful that both the paradox docs and binary
> > artifacts
> > > > are
> > > > > exactly the same before and after.
> > > >
> > > >
> > > > OK! The API docs would link to the Java version you're building with
> I
> > > > suppose, but I don't see that as a problem: we can't predict what JDK
> > > users
> > > > will be using anyway.
> > > >
> > > > From my end we still need to republish paradox 0.9.x on maven just
> due
> > to
> > > > > the 1.0.x branches so I will continue to get that done.
> > > > >
> > > >
> > > > OK
> > > >
> > > >
> > > > > I would also posit that a nicer solution to this would also be
> > getting
> > > > > sbt-multi-release-jar[1] to work because currently we have a more
> > > > > recurring issue where we want to use features in newer JDK's (such
> as
> > > > > loom/virtual thread for JDK 21) but we can't while supporting JDK 8
> > > > without
> > > > > a lot of hackery (if it's even possible).
> > > > > Unfortunately I am hitting some fundamental issues[2] on trying to
> > get
> > > it
> > > > > to work
> > > > >
> > > >
> > > > I agree that might be interesting but require a lot more r&d to find
> > out
> > > if
> > > > it's viable.
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Arnout
> > > >
> > > >
> > > > >
> > > > > 1: https://github.com/sbt/sbt-multi-release-jar and
> > > > >
> > https://github.com/sbt/sbt/discussions/7174#discussioncomment-5288982
> > > > > 2: https://github.com/sbt/sbt-multi-release-jar/issues/24
> > > > >
> > > > > On Sat, Feb 10, 2024 at 8:45 PM Arnout Engelen <engelen@apache.org
> >
> > > > wrote:
> > > > >
> > > > > > On Sat, Feb 10, 2024 at 8:57 AM Matthew de Detrich
> > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > >
> > > > > > > > I think we should consider requiring Java 11 for all
> > development:
> > > > the
> > > > > > > cost/complexity of requiring all our build tools to keep
> > supporting
> > > > > Java
> > > > > > 8
> > > > > > > will only increase, and IMHO is unhealthy to the wider
> ecosystem.
> > > The
> > > > > > > cost/complexity of building with Java 11 and targeting Java 8
> is
> > > low
> > > > > when
> > > > > > > using the well-understood compiler flags, except when also
> using
> > > > > > > sun.misc.Unsafe, but we have already incurred that cost for
> that
> > > case
> > > > > > > anyway.
> > > > > > >
> > > > > > > the Pekko 1.0.x series is deliberately designed to
> > > > > > > have minimal intrusive changes merged into it and so for this
> > > reason
> > > > > its
> > > > > > > ideal
> > > > > > > to have the current build setup remain the same, which means
> both
> > > > > > building
> > > > > > > with JDK 8 (or JDK 11 + JDK 8 workaround for pekko core) and
> > > paradox
> > > > > > 0.9.x.
> > > > > > >
> > > > > >
> > > > > > That makes sense, I'm fine with requiring JDK 11 only for
> building
> > > > Pekko
> > > > > > 1.1.x onwards.
> > > > > >
> > > > > >
> > > > > > > My personal preference is to remain having JDK 1.8 as the
> > preferred
> > > > one
> > > > > > > for development as it is now and rather instead spend our
> efforts
> > > to
> > > > > > > strongly push for dropping JDK 1.8 altogether for Pekko 2.0.x.
> > This
> > > > > would
> > > > > > > allow
> > > > > > > us to remove a lot of these hacks which would go a great deal
> in
> > > > > > > simplifying the build.
> > > > > > >
> > > > > >
> > > > > > I'm all for dropping Java 8 support completely on Pekko 2.0.x,
> and
> > > > indeed
> > > > > > it would allow us to simplify the build.
> > > > > >
> > > > > > I think requiring Java 11 for building Pekko 1.1.x would still be
> > > good.
> > > > > By
> > > > > > the time Pekko 2.0.x happens, we might want to build with even
> > newer
> > > > > > versions of Java, so most of the work we do now to build with
> Java
> > 11
> > > > and
> > > > > > target Java 8 will continue to be valuable then as well.
> > > > > >
> > > > > > > While it is true the use of Java 8 sadly still seems
> widespread (
> > > > > > >
> > > >
> https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > > > > mentions 33%), I'm sceptical the same holds for developer
> > > > environments.
> > > > > > >
> > > > > > > While I do broadly agree this is accurate, it's still
> preferable
> > > for
> > > > > > > developers
> > > > > > > to use the lowest common denominator JDK that the project uses
> > just
> > > > for
> > > > > > > the sakes of lowering surprises especially when it comes to
> > > releases,
> > > > > > this
> > > > > > > is what I do. There are plenty of solutions that allow you to
> hot
> > > > swap
> > > > > > > JDK's
> > > > > > > on local machines and on this note I was actually considering
> > > adding
> > > > > > > .java-version files like in sbt-multi-release-jar[2] which
> forces
> > > the
> > > > > > > project
> > > > > > > to be loaded with a specific JDK. This may be a bit heavy
> handed
> > > but
> > > > > > > it does a good job of enforcing expectations.
> > > > > >
> > > > > >
> > > > > > It's a trade-off between the risks of developing on more recent
> > JDKs
> > > > > while
> > > > > > targeting Java 8, and making maintenance more painful by
> requiring
> > > > > > development tooling to keep supporting Java 8. I think we should
> > take
> > > > the
> > > > > > risk.
> > > > > >
> > > > > >
> > > > > > Kind regards,
> > > > > >
> > > > > > Arnout
> > > > > >
> > > > > >
> > > > > > > On Fri, Feb 9, 2024 at 9:57 PM Arnout Engelen <
> > engelen@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > > > On Wed, Jan 31, 2024 at 8:44 AM Matthew de Detrich
> > > > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > > >
> > > > > > > > > My main concern is that since we are still supporting JDK
> 8,
> > > > > building
> > > > > > > the
> > > > > > > > > docs with JDK 11 can bring about actual legitimate
> usability
> > > > > concerns
> > > > > > > in
> > > > > > > > > terms
> > > > > > > > > of consistency because we still have to build the main
> source
> > > > code
> > > > > > with
> > > > > > > > JDK
> > > > > > > > > 8
> > > > > > > > > but with docs we have to build with JDK 11 and so if we do
> > this
> > > > > > change
> > > > > > > we
> > > > > > > > > have a situation where if someone starts sbt with JDK 8
> > > (because
> > > > > they
> > > > > > > are
> > > > > > > > > developing normally with the source code) and then do
> > > > docs/paradox
> > > > > > they
> > > > > > > > > will
> > > > > > > > > get hit with runtime bytecode mismatch errors.
> > > > > > > > >
> > > > > > > > > Now this can be solved with the the source and target flags
> > > i.e.
> > > > > > > > >
> > > > > > > > > javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
> > > > > > > > >
> > > > > > > > > Which instructs the JVM to target JDK 1.8 but there are
> > > > limitations
> > > > > > > here
> > > > > > > > > i.e.
> > > > > > > > > the fact that it doesn't work sun.misc.unsafe[1].
> > > > > > > > >
> > > > > > > >
> > > > > > > > Luckily, only the main pekko repo uses sun.misc.Unsafe
> > > (pekko-http
> > > > > > > > indirectly through pekko.util.Unsafe).
> > > > > > > >
> > > > > > > > The main pekko repo *already* supports building with Java 11
> > (and
> > > > > > > requires
> > > > > > > > it for releases) and targeting Java 8. While it is true it
> uses
> > > > > fairly
> > > > > > > > horrible hacks to achieve this while also using
> > sun.misc.Unsafe,
> > > > this
> > > > > > is
> > > > > > > > already in place, and other components should be fine with
> the
> > > > > > 'regular'
> > > > > > > > flags for targeting Java 8.
> > > > > > > >
> > > > > > > >
> > > > > > > > > There is also the other issue which is well, the compiled
> > docs
> > > > > would
> > > > > > be
> > > > > > > > > slightly
> > > > > > > > > misleading in the sense that they would link against the
> JDK
> > 11
> > > > > docs
> > > > > > > when
> > > > > > > > > in reality
> > > > > > > > > our projects support JDK 8 and these docs do actually
> differ
> > > > (JDK8
> > > > > > docs
> > > > > > > > > differ
> > > > > > > > > to JDK11 docs).
> > > > > > > >
> > > > > > > >
> > > > > > > > I don't think this is an issue: while it is true we support
> > Java
> > > 8,
> > > > > I'd
> > > > > > > say
> > > > > > > > users are more than likely to be on more recent Java
> versions,
> > in
> > > > > which
> > > > > > > > case the links to more recent Java versions will be more
> > > accurate.
> > > > > > > >
> > > > > > > >
> > > > > > > > > I mentioned this somewhere (can't remember where) but it
> > would
> > > > > > actually
> > > > > > > > be
> > > > > > > > > ideal
> > > > > > > > > if another release of sbt-paradox from the older series
> i.e.
> > > > 0.9.x
> > > > > so
> > > > > > > it
> > > > > > > > > can be
> > > > > > > > > deployed to the newer repositories.
> > > > > > > > >
> > > > > > > >
> > > > > > > > I think we should consider requiring Java 11 for all
> > development:
> > > > the
> > > > > > > > cost/complexity of requiring all our build tools to keep
> > > supporting
> > > > > > Java
> > > > > > > 8
> > > > > > > > will only increase, and IMHO is unhealthy to the wider
> > ecosystem.
> > > > The
> > > > > > > > cost/complexity of building with Java 11 and targeting Java 8
> > is
> > > > low
> > > > > > when
> > > > > > > > using the well-understood compiler flags, except when also
> > using
> > > > > > > > sun.misc.Unsafe, but we have already incurred that cost for
> > that
> > > > case
> > > > > > > > anyway.
> > > > > > > >
> > > > > > > > The main remaining risk I see would be that we'd
> > unintentionally
> > > > > accept
> > > > > > > an
> > > > > > > > update of a runtime dependency that drops support for Java 8.
> > > > > > > >
> > > > > > > > While it is true the use of Java 8 sadly still seems
> > widespread (
> > > > > > > >
> > > > >
> > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > > > > > mentions 33%), I'm sceptical the same holds for developer
> > > > > environments.
> > > > > > > >
> > > > > > > >
> > > > > > > > Kind regards,
> > > > > > > >
> > > > > > > > Arnout
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > [1]:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <
> > > fanningpj@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > We have a slightly untidy sbt plugin setup in our various
> > > Pekko
> > > > > > > repos,
> > > > > > > > > > where we force the use of old Paradox plugins so that we
> > can
> > > > use
> > > > > > Java
> > > > > > > > > > 8 for doc building.
> > > > > > > > > >
> > > > > > > > > > Now that we have the 1.0.x releases done, can we consider
> > > > > switching
> > > > > > > to
> > > > > > > > > > the latest Paradox plugins and live with the fact that
> that
> > > > means
> > > > > > we
> > > > > > > > > > need to use Java 11 for doc builds?
> > > > > > > > > >
> > > > > > > > > > Matthias Kurz has been publishing some new versions of
> > these
> > > > > > plugins
> > > > > > > > > > because the versions that we use are not in Maven Central
> > > > meaning
> > > > > > > that
> > > > > > > > > > we are dependent on the scala-sbt repository.
> > > > > > > > > >
> > > > > > > > > > Any objections to us at least starting with the Pekko
> repos
> > > > where
> > > > > > we
> > > > > > > > > > have 1.0.x branches? We can update the main branches in
> > these
> > > > > repos
> > > > > > > to
> > > > > > > > > > use the latest Paradox plugins and update the build docs.
> > > > > > > > > >
> > > > > > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > For additional commands, e-mail:
> dev-help@pekko.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > Matthew de Detrich
> > > > > > > > >
> > > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > > >
> > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > > >
> > > > > > > > > Alexanderufer 3-7, 10117 Berlin
> > > > > > > > >
> > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > > >
> > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > > >
> > > > > > > > > *m:* +491603708037
> > > > > > > > >
> > > > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Arnout Engelen
> > > > > > > > ASF Security Response
> > > > > > > > Committer on Apache Pekko
> > > > > > > > Committer on NixOS
> > > > > > > > Independent Open Source consultant
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Matthew de Detrich
> > > > > > >
> > > > > > > *Aiven Deutschland GmbH*
> > > > > > >
> > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > >
> > > > > > > Alexanderufer 3-7, 10117 Berlin
> > > > > > >
> > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > >
> > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > >
> > > > > > > *m:* +491603708037
> > > > > > >
> > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Arnout Engelen
> > > > > > ASF Security Response
> > > > > > Committer on Apache Pekko
> > > > > > Committer on NixOS
> > > > > > Independent Open Source consultant
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Matthew de Detrich
> > > > >
> > > > > *Aiven Deutschland GmbH*
> > > > >
> > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > >
> > > > > Alexanderufer 3-7, 10117 Berlin
> > > > >
> > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > >
> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > >
> > > > > *m:* +491603708037
> > > > >
> > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > >
> > > >
> > > >
> > > > --
> > > > Arnout Engelen
> > > > ASF Security Response
> > > > Committer on Apache Pekko
> > > > Committer on NixOS
> > > > Independent Open Source consultant
> > > >
> > >
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Alexanderufer 3-7, 10117 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
> >
> >
> > --
> > Arnout Engelen
> > ASF Security Response
> > Committer on Apache Pekko
> > Committer on NixOS
> > Independent Open Source consultant
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Alexanderufer 3-7, 10117 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>


-- 
Arnout Engelen
ASF Security Response
Committer on Apache Pekko
Committer on NixOS
Independent Open Source consultant

Re: [DISCUSS] require Java 11 for Pekko doc builds

[Matthew de Detrich <ma...@aiven.io.INVALID>]

> I agree that would be nice, but is it a problem? Even if we could specify
the JDK version to link against for the docs, we should perhaps link to
Java 11 (or even Java 21): while we support Java 8, I think we should
encourage users to use newer versions.

I don't know, I remember we tried messing around with this a year ago
and we ended up reverting it for one reason or another (I would have to go
back and find out now). For me it seems natural that the docs should target
the same JDK that is the lowest common denominator but I can see how
that is a weak argument.

A stronger counter argument is that it's easier for developers to
understand,
mixing and matching different JDK's for different tasks is actually somewhat
confusing whereas now it's simply "build with JDK 8 for everything" (aside
from Pekko core) which is consistent and simple, even if it's outdated
(although
this can be solved with release flags especially with recent fixes
that have come in via Scala 2.11/2.12 but still waiting for 3.3.2)

On Sat, Feb 10, 2024 at 10:04 PM Arnout Engelen <en...@apache.org> wrote:

> On Sat, Feb 10, 2024 at 11:50 AM Matthew de Detrich
> <ma...@aiven.io.invalid> wrote:
>
> > > OK! The API docs would link to the Java version you're building with I
> > suppose, but I don't see that as a problem: we can't predict what JDK
> users
> > will be using anyway.
> >
> > Yeah this is what the main issue is, I would be more amicable to building
> > the docs on JDK 11 if there was some way to designate the JDK version to
> > link against for the docs.
> >
>
> I agree that would be nice, but is it a problem? Even if we could specify
> the JDK version to link against for the docs, we should perhaps link to
> Java 11 (or even Java 21): while we support Java 8, I think we should
> encourage users to use newer versions.
>
>
> Kind regards,
>
> Arnout
>
>
> >
> > On Sat, Feb 10, 2024 at 9:17 PM Arnout Engelen <en...@apache.org>
> wrote:
> >
> > > On Sat, Feb 10, 2024 at 10:51 AM Matthew de Detrich
> > > <ma...@aiven.io.invalid> wrote:
> > >
> > > > > It's a trade-off between the risks of developing on more recent
> JDKs
> > > > while targeting Java 8, and making maintenance more painful by
> > requiring
> > > > development tooling to keep supporting Java 8. I think we should take
> > the
> > > > risk.
> > > >
> > > > Sure if someone wants to go ahead then they are more than welcome to
> > try,
> > > > just need to be careful that both the paradox docs and binary
> artifacts
> > > are
> > > > exactly the same before and after.
> > >
> > >
> > > OK! The API docs would link to the Java version you're building with I
> > > suppose, but I don't see that as a problem: we can't predict what JDK
> > users
> > > will be using anyway.
> > >
> > > From my end we still need to republish paradox 0.9.x on maven just due
> to
> > > > the 1.0.x branches so I will continue to get that done.
> > > >
> > >
> > > OK
> > >
> > >
> > > > I would also posit that a nicer solution to this would also be
> getting
> > > > sbt-multi-release-jar[1] to work because currently we have a more
> > > > recurring issue where we want to use features in newer JDK's (such as
> > > > loom/virtual thread for JDK 21) but we can't while supporting JDK 8
> > > without
> > > > a lot of hackery (if it's even possible).
> > > > Unfortunately I am hitting some fundamental issues[2] on trying to
> get
> > it
> > > > to work
> > > >
> > >
> > > I agree that might be interesting but require a lot more r&d to find
> out
> > if
> > > it's viable.
> > >
> > >
> > > Kind regards,
> > >
> > > Arnout
> > >
> > >
> > > >
> > > > 1: https://github.com/sbt/sbt-multi-release-jar and
> > > >
> https://github.com/sbt/sbt/discussions/7174#discussioncomment-5288982
> > > > 2: https://github.com/sbt/sbt-multi-release-jar/issues/24
> > > >
> > > > On Sat, Feb 10, 2024 at 8:45 PM Arnout Engelen <en...@apache.org>
> > > wrote:
> > > >
> > > > > On Sat, Feb 10, 2024 at 8:57 AM Matthew de Detrich
> > > > > <ma...@aiven.io.invalid> wrote:
> > > > >
> > > > > > > I think we should consider requiring Java 11 for all
> development:
> > > the
> > > > > > cost/complexity of requiring all our build tools to keep
> supporting
> > > > Java
> > > > > 8
> > > > > > will only increase, and IMHO is unhealthy to the wider ecosystem.
> > The
> > > > > > cost/complexity of building with Java 11 and targeting Java 8 is
> > low
> > > > when
> > > > > > using the well-understood compiler flags, except when also using
> > > > > > sun.misc.Unsafe, but we have already incurred that cost for that
> > case
> > > > > > anyway.
> > > > > >
> > > > > > the Pekko 1.0.x series is deliberately designed to
> > > > > > have minimal intrusive changes merged into it and so for this
> > reason
> > > > its
> > > > > > ideal
> > > > > > to have the current build setup remain the same, which means both
> > > > > building
> > > > > > with JDK 8 (or JDK 11 + JDK 8 workaround for pekko core) and
> > paradox
> > > > > 0.9.x.
> > > > > >
> > > > >
> > > > > That makes sense, I'm fine with requiring JDK 11 only for building
> > > Pekko
> > > > > 1.1.x onwards.
> > > > >
> > > > >
> > > > > > My personal preference is to remain having JDK 1.8 as the
> preferred
> > > one
> > > > > > for development as it is now and rather instead spend our efforts
> > to
> > > > > > strongly push for dropping JDK 1.8 altogether for Pekko 2.0.x.
> This
> > > > would
> > > > > > allow
> > > > > > us to remove a lot of these hacks which would go a great deal in
> > > > > > simplifying the build.
> > > > > >
> > > > >
> > > > > I'm all for dropping Java 8 support completely on Pekko 2.0.x, and
> > > indeed
> > > > > it would allow us to simplify the build.
> > > > >
> > > > > I think requiring Java 11 for building Pekko 1.1.x would still be
> > good.
> > > > By
> > > > > the time Pekko 2.0.x happens, we might want to build with even
> newer
> > > > > versions of Java, so most of the work we do now to build with Java
> 11
> > > and
> > > > > target Java 8 will continue to be valuable then as well.
> > > > >
> > > > > > While it is true the use of Java 8 sadly still seems widespread (
> > > > > >
> > > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > > > mentions 33%), I'm sceptical the same holds for developer
> > > environments.
> > > > > >
> > > > > > While I do broadly agree this is accurate, it's still preferable
> > for
> > > > > > developers
> > > > > > to use the lowest common denominator JDK that the project uses
> just
> > > for
> > > > > > the sakes of lowering surprises especially when it comes to
> > releases,
> > > > > this
> > > > > > is what I do. There are plenty of solutions that allow you to hot
> > > swap
> > > > > > JDK's
> > > > > > on local machines and on this note I was actually considering
> > adding
> > > > > > .java-version files like in sbt-multi-release-jar[2] which forces
> > the
> > > > > > project
> > > > > > to be loaded with a specific JDK. This may be a bit heavy handed
> > but
> > > > > > it does a good job of enforcing expectations.
> > > > >
> > > > >
> > > > > It's a trade-off between the risks of developing on more recent
> JDKs
> > > > while
> > > > > targeting Java 8, and making maintenance more painful by requiring
> > > > > development tooling to keep supporting Java 8. I think we should
> take
> > > the
> > > > > risk.
> > > > >
> > > > >
> > > > > Kind regards,
> > > > >
> > > > > Arnout
> > > > >
> > > > >
> > > > > > On Fri, Feb 9, 2024 at 9:57 PM Arnout Engelen <
> engelen@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > On Wed, Jan 31, 2024 at 8:44 AM Matthew de Detrich
> > > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > >
> > > > > > > > My main concern is that since we are still supporting JDK 8,
> > > > building
> > > > > > the
> > > > > > > > docs with JDK 11 can bring about actual legitimate usability
> > > > concerns
> > > > > > in
> > > > > > > > terms
> > > > > > > > of consistency because we still have to build the main source
> > > code
> > > > > with
> > > > > > > JDK
> > > > > > > > 8
> > > > > > > > but with docs we have to build with JDK 11 and so if we do
> this
> > > > > change
> > > > > > we
> > > > > > > > have a situation where if someone starts sbt with JDK 8
> > (because
> > > > they
> > > > > > are
> > > > > > > > developing normally with the source code) and then do
> > > docs/paradox
> > > > > they
> > > > > > > > will
> > > > > > > > get hit with runtime bytecode mismatch errors.
> > > > > > > >
> > > > > > > > Now this can be solved with the the source and target flags
> > i.e.
> > > > > > > >
> > > > > > > > javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
> > > > > > > >
> > > > > > > > Which instructs the JVM to target JDK 1.8 but there are
> > > limitations
> > > > > > here
> > > > > > > > i.e.
> > > > > > > > the fact that it doesn't work sun.misc.unsafe[1].
> > > > > > > >
> > > > > > >
> > > > > > > Luckily, only the main pekko repo uses sun.misc.Unsafe
> > (pekko-http
> > > > > > > indirectly through pekko.util.Unsafe).
> > > > > > >
> > > > > > > The main pekko repo *already* supports building with Java 11
> (and
> > > > > > requires
> > > > > > > it for releases) and targeting Java 8. While it is true it uses
> > > > fairly
> > > > > > > horrible hacks to achieve this while also using
> sun.misc.Unsafe,
> > > this
> > > > > is
> > > > > > > already in place, and other components should be fine with the
> > > > > 'regular'
> > > > > > > flags for targeting Java 8.
> > > > > > >
> > > > > > >
> > > > > > > > There is also the other issue which is well, the compiled
> docs
> > > > would
> > > > > be
> > > > > > > > slightly
> > > > > > > > misleading in the sense that they would link against the JDK
> 11
> > > > docs
> > > > > > when
> > > > > > > > in reality
> > > > > > > > our projects support JDK 8 and these docs do actually differ
> > > (JDK8
> > > > > docs
> > > > > > > > differ
> > > > > > > > to JDK11 docs).
> > > > > > >
> > > > > > >
> > > > > > > I don't think this is an issue: while it is true we support
> Java
> > 8,
> > > > I'd
> > > > > > say
> > > > > > > users are more than likely to be on more recent Java versions,
> in
> > > > which
> > > > > > > case the links to more recent Java versions will be more
> > accurate.
> > > > > > >
> > > > > > >
> > > > > > > > I mentioned this somewhere (can't remember where) but it
> would
> > > > > actually
> > > > > > > be
> > > > > > > > ideal
> > > > > > > > if another release of sbt-paradox from the older series i.e.
> > > 0.9.x
> > > > so
> > > > > > it
> > > > > > > > can be
> > > > > > > > deployed to the newer repositories.
> > > > > > > >
> > > > > > >
> > > > > > > I think we should consider requiring Java 11 for all
> development:
> > > the
> > > > > > > cost/complexity of requiring all our build tools to keep
> > supporting
> > > > > Java
> > > > > > 8
> > > > > > > will only increase, and IMHO is unhealthy to the wider
> ecosystem.
> > > The
> > > > > > > cost/complexity of building with Java 11 and targeting Java 8
> is
> > > low
> > > > > when
> > > > > > > using the well-understood compiler flags, except when also
> using
> > > > > > > sun.misc.Unsafe, but we have already incurred that cost for
> that
> > > case
> > > > > > > anyway.
> > > > > > >
> > > > > > > The main remaining risk I see would be that we'd
> unintentionally
> > > > accept
> > > > > > an
> > > > > > > update of a runtime dependency that drops support for Java 8.
> > > > > > >
> > > > > > > While it is true the use of Java 8 sadly still seems
> widespread (
> > > > > > >
> > > >
> https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > > > > mentions 33%), I'm sceptical the same holds for developer
> > > > environments.
> > > > > > >
> > > > > > >
> > > > > > > Kind regards,
> > > > > > >
> > > > > > > Arnout
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > [1]:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <
> > fanningpj@apache.org
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > We have a slightly untidy sbt plugin setup in our various
> > Pekko
> > > > > > repos,
> > > > > > > > > where we force the use of old Paradox plugins so that we
> can
> > > use
> > > > > Java
> > > > > > > > > 8 for doc building.
> > > > > > > > >
> > > > > > > > > Now that we have the 1.0.x releases done, can we consider
> > > > switching
> > > > > > to
> > > > > > > > > the latest Paradox plugins and live with the fact that that
> > > means
> > > > > we
> > > > > > > > > need to use Java 11 for doc builds?
> > > > > > > > >
> > > > > > > > > Matthias Kurz has been publishing some new versions of
> these
> > > > > plugins
> > > > > > > > > because the versions that we use are not in Maven Central
> > > meaning
> > > > > > that
> > > > > > > > > we are dependent on the scala-sbt repository.
> > > > > > > > >
> > > > > > > > > Any objections to us at least starting with the Pekko repos
> > > where
> > > > > we
> > > > > > > > > have 1.0.x branches? We can update the main branches in
> these
> > > > repos
> > > > > > to
> > > > > > > > > use the latest Paradox plugins and update the build docs.
> > > > > > > > >
> > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Matthew de Detrich
> > > > > > > >
> > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > >
> > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > >
> > > > > > > > Alexanderufer 3-7, 10117 Berlin
> > > > > > > >
> > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > >
> > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > >
> > > > > > > > *m:* +491603708037
> > > > > > > >
> > > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Arnout Engelen
> > > > > > > ASF Security Response
> > > > > > > Committer on Apache Pekko
> > > > > > > Committer on NixOS
> > > > > > > Independent Open Source consultant
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Matthew de Detrich
> > > > > >
> > > > > > *Aiven Deutschland GmbH*
> > > > > >
> > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > >
> > > > > > Alexanderufer 3-7, 10117 Berlin
> > > > > >
> > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > >
> > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > >
> > > > > > *m:* +491603708037
> > > > > >
> > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Arnout Engelen
> > > > > ASF Security Response
> > > > > Committer on Apache Pekko
> > > > > Committer on NixOS
> > > > > Independent Open Source consultant
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Matthew de Detrich
> > > >
> > > > *Aiven Deutschland GmbH*
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Alexanderufer 3-7, 10117 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > *m:* +491603708037
> > > >
> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >
> > >
> > >
> > > --
> > > Arnout Engelen
> > > ASF Security Response
> > > Committer on Apache Pekko
> > > Committer on NixOS
> > > Independent Open Source consultant
> > >
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Alexanderufer 3-7, 10117 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>
>
> --
> Arnout Engelen
> ASF Security Response
> Committer on Apache Pekko
> Committer on NixOS
> Independent Open Source consultant
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Alexanderufer 3-7, 10117 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] require Java 11 for Pekko doc builds

[Arnout Engelen <en...@apache.org>]

On Sat, Feb 10, 2024 at 11:50 AM Matthew de Detrich
<ma...@aiven.io.invalid> wrote:

> > OK! The API docs would link to the Java version you're building with I
> suppose, but I don't see that as a problem: we can't predict what JDK users
> will be using anyway.
>
> Yeah this is what the main issue is, I would be more amicable to building
> the docs on JDK 11 if there was some way to designate the JDK version to
> link against for the docs.
>

I agree that would be nice, but is it a problem? Even if we could specify
the JDK version to link against for the docs, we should perhaps link to
Java 11 (or even Java 21): while we support Java 8, I think we should
encourage users to use newer versions.


Kind regards,

Arnout


>
> On Sat, Feb 10, 2024 at 9:17 PM Arnout Engelen <en...@apache.org> wrote:
>
> > On Sat, Feb 10, 2024 at 10:51 AM Matthew de Detrich
> > <ma...@aiven.io.invalid> wrote:
> >
> > > > It's a trade-off between the risks of developing on more recent JDKs
> > > while targeting Java 8, and making maintenance more painful by
> requiring
> > > development tooling to keep supporting Java 8. I think we should take
> the
> > > risk.
> > >
> > > Sure if someone wants to go ahead then they are more than welcome to
> try,
> > > just need to be careful that both the paradox docs and binary artifacts
> > are
> > > exactly the same before and after.
> >
> >
> > OK! The API docs would link to the Java version you're building with I
> > suppose, but I don't see that as a problem: we can't predict what JDK
> users
> > will be using anyway.
> >
> > From my end we still need to republish paradox 0.9.x on maven just due to
> > > the 1.0.x branches so I will continue to get that done.
> > >
> >
> > OK
> >
> >
> > > I would also posit that a nicer solution to this would also be getting
> > > sbt-multi-release-jar[1] to work because currently we have a more
> > > recurring issue where we want to use features in newer JDK's (such as
> > > loom/virtual thread for JDK 21) but we can't while supporting JDK 8
> > without
> > > a lot of hackery (if it's even possible).
> > > Unfortunately I am hitting some fundamental issues[2] on trying to get
> it
> > > to work
> > >
> >
> > I agree that might be interesting but require a lot more r&d to find out
> if
> > it's viable.
> >
> >
> > Kind regards,
> >
> > Arnout
> >
> >
> > >
> > > 1: https://github.com/sbt/sbt-multi-release-jar and
> > > https://github.com/sbt/sbt/discussions/7174#discussioncomment-5288982
> > > 2: https://github.com/sbt/sbt-multi-release-jar/issues/24
> > >
> > > On Sat, Feb 10, 2024 at 8:45 PM Arnout Engelen <en...@apache.org>
> > wrote:
> > >
> > > > On Sat, Feb 10, 2024 at 8:57 AM Matthew de Detrich
> > > > <ma...@aiven.io.invalid> wrote:
> > > >
> > > > > > I think we should consider requiring Java 11 for all development:
> > the
> > > > > cost/complexity of requiring all our build tools to keep supporting
> > > Java
> > > > 8
> > > > > will only increase, and IMHO is unhealthy to the wider ecosystem.
> The
> > > > > cost/complexity of building with Java 11 and targeting Java 8 is
> low
> > > when
> > > > > using the well-understood compiler flags, except when also using
> > > > > sun.misc.Unsafe, but we have already incurred that cost for that
> case
> > > > > anyway.
> > > > >
> > > > > the Pekko 1.0.x series is deliberately designed to
> > > > > have minimal intrusive changes merged into it and so for this
> reason
> > > its
> > > > > ideal
> > > > > to have the current build setup remain the same, which means both
> > > > building
> > > > > with JDK 8 (or JDK 11 + JDK 8 workaround for pekko core) and
> paradox
> > > > 0.9.x.
> > > > >
> > > >
> > > > That makes sense, I'm fine with requiring JDK 11 only for building
> > Pekko
> > > > 1.1.x onwards.
> > > >
> > > >
> > > > > My personal preference is to remain having JDK 1.8 as the preferred
> > one
> > > > > for development as it is now and rather instead spend our efforts
> to
> > > > > strongly push for dropping JDK 1.8 altogether for Pekko 2.0.x. This
> > > would
> > > > > allow
> > > > > us to remove a lot of these hacks which would go a great deal in
> > > > > simplifying the build.
> > > > >
> > > >
> > > > I'm all for dropping Java 8 support completely on Pekko 2.0.x, and
> > indeed
> > > > it would allow us to simplify the build.
> > > >
> > > > I think requiring Java 11 for building Pekko 1.1.x would still be
> good.
> > > By
> > > > the time Pekko 2.0.x happens, we might want to build with even newer
> > > > versions of Java, so most of the work we do now to build with Java 11
> > and
> > > > target Java 8 will continue to be valuable then as well.
> > > >
> > > > > While it is true the use of Java 8 sadly still seems widespread (
> > > > >
> > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > > mentions 33%), I'm sceptical the same holds for developer
> > environments.
> > > > >
> > > > > While I do broadly agree this is accurate, it's still preferable
> for
> > > > > developers
> > > > > to use the lowest common denominator JDK that the project uses just
> > for
> > > > > the sakes of lowering surprises especially when it comes to
> releases,
> > > > this
> > > > > is what I do. There are plenty of solutions that allow you to hot
> > swap
> > > > > JDK's
> > > > > on local machines and on this note I was actually considering
> adding
> > > > > .java-version files like in sbt-multi-release-jar[2] which forces
> the
> > > > > project
> > > > > to be loaded with a specific JDK. This may be a bit heavy handed
> but
> > > > > it does a good job of enforcing expectations.
> > > >
> > > >
> > > > It's a trade-off between the risks of developing on more recent JDKs
> > > while
> > > > targeting Java 8, and making maintenance more painful by requiring
> > > > development tooling to keep supporting Java 8. I think we should take
> > the
> > > > risk.
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Arnout
> > > >
> > > >
> > > > > On Fri, Feb 9, 2024 at 9:57 PM Arnout Engelen <en...@apache.org>
> > > > wrote:
> > > > >
> > > > > > On Wed, Jan 31, 2024 at 8:44 AM Matthew de Detrich
> > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > >
> > > > > > > My main concern is that since we are still supporting JDK 8,
> > > building
> > > > > the
> > > > > > > docs with JDK 11 can bring about actual legitimate usability
> > > concerns
> > > > > in
> > > > > > > terms
> > > > > > > of consistency because we still have to build the main source
> > code
> > > > with
> > > > > > JDK
> > > > > > > 8
> > > > > > > but with docs we have to build with JDK 11 and so if we do this
> > > > change
> > > > > we
> > > > > > > have a situation where if someone starts sbt with JDK 8
> (because
> > > they
> > > > > are
> > > > > > > developing normally with the source code) and then do
> > docs/paradox
> > > > they
> > > > > > > will
> > > > > > > get hit with runtime bytecode mismatch errors.
> > > > > > >
> > > > > > > Now this can be solved with the the source and target flags
> i.e.
> > > > > > >
> > > > > > > javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
> > > > > > >
> > > > > > > Which instructs the JVM to target JDK 1.8 but there are
> > limitations
> > > > > here
> > > > > > > i.e.
> > > > > > > the fact that it doesn't work sun.misc.unsafe[1].
> > > > > > >
> > > > > >
> > > > > > Luckily, only the main pekko repo uses sun.misc.Unsafe
> (pekko-http
> > > > > > indirectly through pekko.util.Unsafe).
> > > > > >
> > > > > > The main pekko repo *already* supports building with Java 11 (and
> > > > > requires
> > > > > > it for releases) and targeting Java 8. While it is true it uses
> > > fairly
> > > > > > horrible hacks to achieve this while also using sun.misc.Unsafe,
> > this
> > > > is
> > > > > > already in place, and other components should be fine with the
> > > > 'regular'
> > > > > > flags for targeting Java 8.
> > > > > >
> > > > > >
> > > > > > > There is also the other issue which is well, the compiled docs
> > > would
> > > > be
> > > > > > > slightly
> > > > > > > misleading in the sense that they would link against the JDK 11
> > > docs
> > > > > when
> > > > > > > in reality
> > > > > > > our projects support JDK 8 and these docs do actually differ
> > (JDK8
> > > > docs
> > > > > > > differ
> > > > > > > to JDK11 docs).
> > > > > >
> > > > > >
> > > > > > I don't think this is an issue: while it is true we support Java
> 8,
> > > I'd
> > > > > say
> > > > > > users are more than likely to be on more recent Java versions, in
> > > which
> > > > > > case the links to more recent Java versions will be more
> accurate.
> > > > > >
> > > > > >
> > > > > > > I mentioned this somewhere (can't remember where) but it would
> > > > actually
> > > > > > be
> > > > > > > ideal
> > > > > > > if another release of sbt-paradox from the older series i.e.
> > 0.9.x
> > > so
> > > > > it
> > > > > > > can be
> > > > > > > deployed to the newer repositories.
> > > > > > >
> > > > > >
> > > > > > I think we should consider requiring Java 11 for all development:
> > the
> > > > > > cost/complexity of requiring all our build tools to keep
> supporting
> > > > Java
> > > > > 8
> > > > > > will only increase, and IMHO is unhealthy to the wider ecosystem.
> > The
> > > > > > cost/complexity of building with Java 11 and targeting Java 8 is
> > low
> > > > when
> > > > > > using the well-understood compiler flags, except when also using
> > > > > > sun.misc.Unsafe, but we have already incurred that cost for that
> > case
> > > > > > anyway.
> > > > > >
> > > > > > The main remaining risk I see would be that we'd unintentionally
> > > accept
> > > > > an
> > > > > > update of a runtime dependency that drops support for Java 8.
> > > > > >
> > > > > > While it is true the use of Java 8 sadly still seems widespread (
> > > > > >
> > > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > > > mentions 33%), I'm sceptical the same holds for developer
> > > environments.
> > > > > >
> > > > > >
> > > > > > Kind regards,
> > > > > >
> > > > > > Arnout
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > [1]:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <
> fanningpj@apache.org
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > We have a slightly untidy sbt plugin setup in our various
> Pekko
> > > > > repos,
> > > > > > > > where we force the use of old Paradox plugins so that we can
> > use
> > > > Java
> > > > > > > > 8 for doc building.
> > > > > > > >
> > > > > > > > Now that we have the 1.0.x releases done, can we consider
> > > switching
> > > > > to
> > > > > > > > the latest Paradox plugins and live with the fact that that
> > means
> > > > we
> > > > > > > > need to use Java 11 for doc builds?
> > > > > > > >
> > > > > > > > Matthias Kurz has been publishing some new versions of these
> > > > plugins
> > > > > > > > because the versions that we use are not in Maven Central
> > meaning
> > > > > that
> > > > > > > > we are dependent on the scala-sbt repository.
> > > > > > > >
> > > > > > > > Any objections to us at least starting with the Pekko repos
> > where
> > > > we
> > > > > > > > have 1.0.x branches? We can update the main branches in these
> > > repos
> > > > > to
> > > > > > > > use the latest Paradox plugins and update the build docs.
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Matthew de Detrich
> > > > > > >
> > > > > > > *Aiven Deutschland GmbH*
> > > > > > >
> > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > >
> > > > > > > Alexanderufer 3-7, 10117 Berlin
> > > > > > >
> > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > >
> > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > >
> > > > > > > *m:* +491603708037
> > > > > > >
> > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Arnout Engelen
> > > > > > ASF Security Response
> > > > > > Committer on Apache Pekko
> > > > > > Committer on NixOS
> > > > > > Independent Open Source consultant
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Matthew de Detrich
> > > > >
> > > > > *Aiven Deutschland GmbH*
> > > > >
> > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > >
> > > > > Alexanderufer 3-7, 10117 Berlin
> > > > >
> > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > >
> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > >
> > > > > *m:* +491603708037
> > > > >
> > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > >
> > > >
> > > >
> > > > --
> > > > Arnout Engelen
> > > > ASF Security Response
> > > > Committer on Apache Pekko
> > > > Committer on NixOS
> > > > Independent Open Source consultant
> > > >
> > >
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Alexanderufer 3-7, 10117 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
> >
> >
> > --
> > Arnout Engelen
> > ASF Security Response
> > Committer on Apache Pekko
> > Committer on NixOS
> > Independent Open Source consultant
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Alexanderufer 3-7, 10117 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>


-- 
Arnout Engelen
ASF Security Response
Committer on Apache Pekko
Committer on NixOS
Independent Open Source consultant

Re: [DISCUSS] require Java 11 for Pekko doc builds

[Matthew de Detrich <ma...@aiven.io.INVALID>]

> OK! The API docs would link to the Java version you're building with I
suppose, but I don't see that as a problem: we can't predict what JDK users
will be using anyway.

Yeah this is what the main issue is, I would be more amicable to building
the
docs on JDK 11 if there was some way to designate the JDK version to link
against for the docs.

On Sat, Feb 10, 2024 at 9:17 PM Arnout Engelen <en...@apache.org> wrote:

> On Sat, Feb 10, 2024 at 10:51 AM Matthew de Detrich
> <ma...@aiven.io.invalid> wrote:
>
> > > It's a trade-off between the risks of developing on more recent JDKs
> > while targeting Java 8, and making maintenance more painful by requiring
> > development tooling to keep supporting Java 8. I think we should take the
> > risk.
> >
> > Sure if someone wants to go ahead then they are more than welcome to try,
> > just need to be careful that both the paradox docs and binary artifacts
> are
> > exactly the same before and after.
>
>
> OK! The API docs would link to the Java version you're building with I
> suppose, but I don't see that as a problem: we can't predict what JDK users
> will be using anyway.
>
> From my end we still need to republish paradox 0.9.x on maven just due to
> > the 1.0.x branches so I will continue to get that done.
> >
>
> OK
>
>
> > I would also posit that a nicer solution to this would also be getting
> > sbt-multi-release-jar[1] to work because currently we have a more
> > recurring issue where we want to use features in newer JDK's (such as
> > loom/virtual thread for JDK 21) but we can't while supporting JDK 8
> without
> > a lot of hackery (if it's even possible).
> > Unfortunately I am hitting some fundamental issues[2] on trying to get it
> > to work
> >
>
> I agree that might be interesting but require a lot more r&d to find out if
> it's viable.
>
>
> Kind regards,
>
> Arnout
>
>
> >
> > 1: https://github.com/sbt/sbt-multi-release-jar and
> > https://github.com/sbt/sbt/discussions/7174#discussioncomment-5288982
> > 2: https://github.com/sbt/sbt-multi-release-jar/issues/24
> >
> > On Sat, Feb 10, 2024 at 8:45 PM Arnout Engelen <en...@apache.org>
> wrote:
> >
> > > On Sat, Feb 10, 2024 at 8:57 AM Matthew de Detrich
> > > <ma...@aiven.io.invalid> wrote:
> > >
> > > > > I think we should consider requiring Java 11 for all development:
> the
> > > > cost/complexity of requiring all our build tools to keep supporting
> > Java
> > > 8
> > > > will only increase, and IMHO is unhealthy to the wider ecosystem. The
> > > > cost/complexity of building with Java 11 and targeting Java 8 is low
> > when
> > > > using the well-understood compiler flags, except when also using
> > > > sun.misc.Unsafe, but we have already incurred that cost for that case
> > > > anyway.
> > > >
> > > > the Pekko 1.0.x series is deliberately designed to
> > > > have minimal intrusive changes merged into it and so for this reason
> > its
> > > > ideal
> > > > to have the current build setup remain the same, which means both
> > > building
> > > > with JDK 8 (or JDK 11 + JDK 8 workaround for pekko core) and paradox
> > > 0.9.x.
> > > >
> > >
> > > That makes sense, I'm fine with requiring JDK 11 only for building
> Pekko
> > > 1.1.x onwards.
> > >
> > >
> > > > My personal preference is to remain having JDK 1.8 as the preferred
> one
> > > > for development as it is now and rather instead spend our efforts to
> > > > strongly push for dropping JDK 1.8 altogether for Pekko 2.0.x. This
> > would
> > > > allow
> > > > us to remove a lot of these hacks which would go a great deal in
> > > > simplifying the build.
> > > >
> > >
> > > I'm all for dropping Java 8 support completely on Pekko 2.0.x, and
> indeed
> > > it would allow us to simplify the build.
> > >
> > > I think requiring Java 11 for building Pekko 1.1.x would still be good.
> > By
> > > the time Pekko 2.0.x happens, we might want to build with even newer
> > > versions of Java, so most of the work we do now to build with Java 11
> and
> > > target Java 8 will continue to be valuable then as well.
> > >
> > > > While it is true the use of Java 8 sadly still seems widespread (
> > > >
> https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > mentions 33%), I'm sceptical the same holds for developer
> environments.
> > > >
> > > > While I do broadly agree this is accurate, it's still preferable for
> > > > developers
> > > > to use the lowest common denominator JDK that the project uses just
> for
> > > > the sakes of lowering surprises especially when it comes to releases,
> > > this
> > > > is what I do. There are plenty of solutions that allow you to hot
> swap
> > > > JDK's
> > > > on local machines and on this note I was actually considering adding
> > > > .java-version files like in sbt-multi-release-jar[2] which forces the
> > > > project
> > > > to be loaded with a specific JDK. This may be a bit heavy handed but
> > > > it does a good job of enforcing expectations.
> > >
> > >
> > > It's a trade-off between the risks of developing on more recent JDKs
> > while
> > > targeting Java 8, and making maintenance more painful by requiring
> > > development tooling to keep supporting Java 8. I think we should take
> the
> > > risk.
> > >
> > >
> > > Kind regards,
> > >
> > > Arnout
> > >
> > >
> > > > On Fri, Feb 9, 2024 at 9:57 PM Arnout Engelen <en...@apache.org>
> > > wrote:
> > > >
> > > > > On Wed, Jan 31, 2024 at 8:44 AM Matthew de Detrich
> > > > > <ma...@aiven.io.invalid> wrote:
> > > > >
> > > > > > My main concern is that since we are still supporting JDK 8,
> > building
> > > > the
> > > > > > docs with JDK 11 can bring about actual legitimate usability
> > concerns
> > > > in
> > > > > > terms
> > > > > > of consistency because we still have to build the main source
> code
> > > with
> > > > > JDK
> > > > > > 8
> > > > > > but with docs we have to build with JDK 11 and so if we do this
> > > change
> > > > we
> > > > > > have a situation where if someone starts sbt with JDK 8 (because
> > they
> > > > are
> > > > > > developing normally with the source code) and then do
> docs/paradox
> > > they
> > > > > > will
> > > > > > get hit with runtime bytecode mismatch errors.
> > > > > >
> > > > > > Now this can be solved with the the source and target flags i.e.
> > > > > >
> > > > > > javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
> > > > > >
> > > > > > Which instructs the JVM to target JDK 1.8 but there are
> limitations
> > > > here
> > > > > > i.e.
> > > > > > the fact that it doesn't work sun.misc.unsafe[1].
> > > > > >
> > > > >
> > > > > Luckily, only the main pekko repo uses sun.misc.Unsafe (pekko-http
> > > > > indirectly through pekko.util.Unsafe).
> > > > >
> > > > > The main pekko repo *already* supports building with Java 11 (and
> > > > requires
> > > > > it for releases) and targeting Java 8. While it is true it uses
> > fairly
> > > > > horrible hacks to achieve this while also using sun.misc.Unsafe,
> this
> > > is
> > > > > already in place, and other components should be fine with the
> > > 'regular'
> > > > > flags for targeting Java 8.
> > > > >
> > > > >
> > > > > > There is also the other issue which is well, the compiled docs
> > would
> > > be
> > > > > > slightly
> > > > > > misleading in the sense that they would link against the JDK 11
> > docs
> > > > when
> > > > > > in reality
> > > > > > our projects support JDK 8 and these docs do actually differ
> (JDK8
> > > docs
> > > > > > differ
> > > > > > to JDK11 docs).
> > > > >
> > > > >
> > > > > I don't think this is an issue: while it is true we support Java 8,
> > I'd
> > > > say
> > > > > users are more than likely to be on more recent Java versions, in
> > which
> > > > > case the links to more recent Java versions will be more accurate.
> > > > >
> > > > >
> > > > > > I mentioned this somewhere (can't remember where) but it would
> > > actually
> > > > > be
> > > > > > ideal
> > > > > > if another release of sbt-paradox from the older series i.e.
> 0.9.x
> > so
> > > > it
> > > > > > can be
> > > > > > deployed to the newer repositories.
> > > > > >
> > > > >
> > > > > I think we should consider requiring Java 11 for all development:
> the
> > > > > cost/complexity of requiring all our build tools to keep supporting
> > > Java
> > > > 8
> > > > > will only increase, and IMHO is unhealthy to the wider ecosystem.
> The
> > > > > cost/complexity of building with Java 11 and targeting Java 8 is
> low
> > > when
> > > > > using the well-understood compiler flags, except when also using
> > > > > sun.misc.Unsafe, but we have already incurred that cost for that
> case
> > > > > anyway.
> > > > >
> > > > > The main remaining risk I see would be that we'd unintentionally
> > accept
> > > > an
> > > > > update of a runtime dependency that drops support for Java 8.
> > > > >
> > > > > While it is true the use of Java 8 sadly still seems widespread (
> > > > >
> > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > > mentions 33%), I'm sceptical the same holds for developer
> > environments.
> > > > >
> > > > >
> > > > > Kind regards,
> > > > >
> > > > > Arnout
> > > > >
> > > > >
> > > > > >
> > > > > > [1]:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
> > > > > >
> > > > > >
> > > > > > On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <fanningpj@apache.org
> >
> > > > wrote:
> > > > > >
> > > > > > > We have a slightly untidy sbt plugin setup in our various Pekko
> > > > repos,
> > > > > > > where we force the use of old Paradox plugins so that we can
> use
> > > Java
> > > > > > > 8 for doc building.
> > > > > > >
> > > > > > > Now that we have the 1.0.x releases done, can we consider
> > switching
> > > > to
> > > > > > > the latest Paradox plugins and live with the fact that that
> means
> > > we
> > > > > > > need to use Java 11 for doc builds?
> > > > > > >
> > > > > > > Matthias Kurz has been publishing some new versions of these
> > > plugins
> > > > > > > because the versions that we use are not in Maven Central
> meaning
> > > > that
> > > > > > > we are dependent on the scala-sbt repository.
> > > > > > >
> > > > > > > Any objections to us at least starting with the Pekko repos
> where
> > > we
> > > > > > > have 1.0.x branches? We can update the main branches in these
> > repos
> > > > to
> > > > > > > use the latest Paradox plugins and update the build docs.
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Matthew de Detrich
> > > > > >
> > > > > > *Aiven Deutschland GmbH*
> > > > > >
> > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > >
> > > > > > Alexanderufer 3-7, 10117 Berlin
> > > > > >
> > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > >
> > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > >
> > > > > > *m:* +491603708037
> > > > > >
> > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Arnout Engelen
> > > > > ASF Security Response
> > > > > Committer on Apache Pekko
> > > > > Committer on NixOS
> > > > > Independent Open Source consultant
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Matthew de Detrich
> > > >
> > > > *Aiven Deutschland GmbH*
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Alexanderufer 3-7, 10117 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > *m:* +491603708037
> > > >
> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >
> > >
> > >
> > > --
> > > Arnout Engelen
> > > ASF Security Response
> > > Committer on Apache Pekko
> > > Committer on NixOS
> > > Independent Open Source consultant
> > >
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Alexanderufer 3-7, 10117 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>
>
> --
> Arnout Engelen
> ASF Security Response
> Committer on Apache Pekko
> Committer on NixOS
> Independent Open Source consultant
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Alexanderufer 3-7, 10117 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] require Java 11 for Pekko doc builds

[Arnout Engelen <en...@apache.org>]

On Sat, Feb 10, 2024 at 10:51 AM Matthew de Detrich
<ma...@aiven.io.invalid> wrote:

> > It's a trade-off between the risks of developing on more recent JDKs
> while targeting Java 8, and making maintenance more painful by requiring
> development tooling to keep supporting Java 8. I think we should take the
> risk.
>
> Sure if someone wants to go ahead then they are more than welcome to try,
> just need to be careful that both the paradox docs and binary artifacts are
> exactly the same before and after.


OK! The API docs would link to the Java version you're building with I
suppose, but I don't see that as a problem: we can't predict what JDK users
will be using anyway.

From my end we still need to republish paradox 0.9.x on maven just due to
> the 1.0.x branches so I will continue to get that done.
>

OK


> I would also posit that a nicer solution to this would also be getting
> sbt-multi-release-jar[1] to work because currently we have a more
> recurring issue where we want to use features in newer JDK's (such as
> loom/virtual thread for JDK 21) but we can't while supporting JDK 8 without
> a lot of hackery (if it's even possible).
> Unfortunately I am hitting some fundamental issues[2] on trying to get it
> to work
>

I agree that might be interesting but require a lot more r&d to find out if
it's viable.


Kind regards,

Arnout


>
> 1: https://github.com/sbt/sbt-multi-release-jar and
> https://github.com/sbt/sbt/discussions/7174#discussioncomment-5288982
> 2: https://github.com/sbt/sbt-multi-release-jar/issues/24
>
> On Sat, Feb 10, 2024 at 8:45 PM Arnout Engelen <en...@apache.org> wrote:
>
> > On Sat, Feb 10, 2024 at 8:57 AM Matthew de Detrich
> > <ma...@aiven.io.invalid> wrote:
> >
> > > > I think we should consider requiring Java 11 for all development: the
> > > cost/complexity of requiring all our build tools to keep supporting
> Java
> > 8
> > > will only increase, and IMHO is unhealthy to the wider ecosystem. The
> > > cost/complexity of building with Java 11 and targeting Java 8 is low
> when
> > > using the well-understood compiler flags, except when also using
> > > sun.misc.Unsafe, but we have already incurred that cost for that case
> > > anyway.
> > >
> > > the Pekko 1.0.x series is deliberately designed to
> > > have minimal intrusive changes merged into it and so for this reason
> its
> > > ideal
> > > to have the current build setup remain the same, which means both
> > building
> > > with JDK 8 (or JDK 11 + JDK 8 workaround for pekko core) and paradox
> > 0.9.x.
> > >
> >
> > That makes sense, I'm fine with requiring JDK 11 only for building Pekko
> > 1.1.x onwards.
> >
> >
> > > My personal preference is to remain having JDK 1.8 as the preferred one
> > > for development as it is now and rather instead spend our efforts to
> > > strongly push for dropping JDK 1.8 altogether for Pekko 2.0.x. This
> would
> > > allow
> > > us to remove a lot of these hacks which would go a great deal in
> > > simplifying the build.
> > >
> >
> > I'm all for dropping Java 8 support completely on Pekko 2.0.x, and indeed
> > it would allow us to simplify the build.
> >
> > I think requiring Java 11 for building Pekko 1.1.x would still be good.
> By
> > the time Pekko 2.0.x happens, we might want to build with even newer
> > versions of Java, so most of the work we do now to build with Java 11 and
> > target Java 8 will continue to be valuable then as well.
> >
> > > While it is true the use of Java 8 sadly still seems widespread (
> > > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > mentions 33%), I'm sceptical the same holds for developer environments.
> > >
> > > While I do broadly agree this is accurate, it's still preferable for
> > > developers
> > > to use the lowest common denominator JDK that the project uses just for
> > > the sakes of lowering surprises especially when it comes to releases,
> > this
> > > is what I do. There are plenty of solutions that allow you to hot swap
> > > JDK's
> > > on local machines and on this note I was actually considering adding
> > > .java-version files like in sbt-multi-release-jar[2] which forces the
> > > project
> > > to be loaded with a specific JDK. This may be a bit heavy handed but
> > > it does a good job of enforcing expectations.
> >
> >
> > It's a trade-off between the risks of developing on more recent JDKs
> while
> > targeting Java 8, and making maintenance more painful by requiring
> > development tooling to keep supporting Java 8. I think we should take the
> > risk.
> >
> >
> > Kind regards,
> >
> > Arnout
> >
> >
> > > On Fri, Feb 9, 2024 at 9:57 PM Arnout Engelen <en...@apache.org>
> > wrote:
> > >
> > > > On Wed, Jan 31, 2024 at 8:44 AM Matthew de Detrich
> > > > <ma...@aiven.io.invalid> wrote:
> > > >
> > > > > My main concern is that since we are still supporting JDK 8,
> building
> > > the
> > > > > docs with JDK 11 can bring about actual legitimate usability
> concerns
> > > in
> > > > > terms
> > > > > of consistency because we still have to build the main source code
> > with
> > > > JDK
> > > > > 8
> > > > > but with docs we have to build with JDK 11 and so if we do this
> > change
> > > we
> > > > > have a situation where if someone starts sbt with JDK 8 (because
> they
> > > are
> > > > > developing normally with the source code) and then do docs/paradox
> > they
> > > > > will
> > > > > get hit with runtime bytecode mismatch errors.
> > > > >
> > > > > Now this can be solved with the the source and target flags i.e.
> > > > >
> > > > > javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
> > > > >
> > > > > Which instructs the JVM to target JDK 1.8 but there are limitations
> > > here
> > > > > i.e.
> > > > > the fact that it doesn't work sun.misc.unsafe[1].
> > > > >
> > > >
> > > > Luckily, only the main pekko repo uses sun.misc.Unsafe (pekko-http
> > > > indirectly through pekko.util.Unsafe).
> > > >
> > > > The main pekko repo *already* supports building with Java 11 (and
> > > requires
> > > > it for releases) and targeting Java 8. While it is true it uses
> fairly
> > > > horrible hacks to achieve this while also using sun.misc.Unsafe, this
> > is
> > > > already in place, and other components should be fine with the
> > 'regular'
> > > > flags for targeting Java 8.
> > > >
> > > >
> > > > > There is also the other issue which is well, the compiled docs
> would
> > be
> > > > > slightly
> > > > > misleading in the sense that they would link against the JDK 11
> docs
> > > when
> > > > > in reality
> > > > > our projects support JDK 8 and these docs do actually differ (JDK8
> > docs
> > > > > differ
> > > > > to JDK11 docs).
> > > >
> > > >
> > > > I don't think this is an issue: while it is true we support Java 8,
> I'd
> > > say
> > > > users are more than likely to be on more recent Java versions, in
> which
> > > > case the links to more recent Java versions will be more accurate.
> > > >
> > > >
> > > > > I mentioned this somewhere (can't remember where) but it would
> > actually
> > > > be
> > > > > ideal
> > > > > if another release of sbt-paradox from the older series i.e. 0.9.x
> so
> > > it
> > > > > can be
> > > > > deployed to the newer repositories.
> > > > >
> > > >
> > > > I think we should consider requiring Java 11 for all development: the
> > > > cost/complexity of requiring all our build tools to keep supporting
> > Java
> > > 8
> > > > will only increase, and IMHO is unhealthy to the wider ecosystem. The
> > > > cost/complexity of building with Java 11 and targeting Java 8 is low
> > when
> > > > using the well-understood compiler flags, except when also using
> > > > sun.misc.Unsafe, but we have already incurred that cost for that case
> > > > anyway.
> > > >
> > > > The main remaining risk I see would be that we'd unintentionally
> accept
> > > an
> > > > update of a runtime dependency that drops support for Java 8.
> > > >
> > > > While it is true the use of Java 8 sadly still seems widespread (
> > > >
> https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > > mentions 33%), I'm sceptical the same holds for developer
> environments.
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Arnout
> > > >
> > > >
> > > > >
> > > > > [1]:
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
> > > > >
> > > > >
> > > > > On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <fa...@apache.org>
> > > wrote:
> > > > >
> > > > > > We have a slightly untidy sbt plugin setup in our various Pekko
> > > repos,
> > > > > > where we force the use of old Paradox plugins so that we can use
> > Java
> > > > > > 8 for doc building.
> > > > > >
> > > > > > Now that we have the 1.0.x releases done, can we consider
> switching
> > > to
> > > > > > the latest Paradox plugins and live with the fact that that means
> > we
> > > > > > need to use Java 11 for doc builds?
> > > > > >
> > > > > > Matthias Kurz has been publishing some new versions of these
> > plugins
> > > > > > because the versions that we use are not in Maven Central meaning
> > > that
> > > > > > we are dependent on the scala-sbt repository.
> > > > > >
> > > > > > Any objections to us at least starting with the Pekko repos where
> > we
> > > > > > have 1.0.x branches? We can update the main branches in these
> repos
> > > to
> > > > > > use the latest Paradox plugins and update the build docs.
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Matthew de Detrich
> > > > >
> > > > > *Aiven Deutschland GmbH*
> > > > >
> > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > >
> > > > > Alexanderufer 3-7, 10117 Berlin
> > > > >
> > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > >
> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > >
> > > > > *m:* +491603708037
> > > > >
> > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > >
> > > >
> > > >
> > > > --
> > > > Arnout Engelen
> > > > ASF Security Response
> > > > Committer on Apache Pekko
> > > > Committer on NixOS
> > > > Independent Open Source consultant
> > > >
> > >
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Alexanderufer 3-7, 10117 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
> >
> >
> > --
> > Arnout Engelen
> > ASF Security Response
> > Committer on Apache Pekko
> > Committer on NixOS
> > Independent Open Source consultant
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Alexanderufer 3-7, 10117 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>


-- 
Arnout Engelen
ASF Security Response
Committer on Apache Pekko
Committer on NixOS
Independent Open Source consultant

Re: [DISCUSS] require Java 11 for Pekko doc builds

[Matthew de Detrich <ma...@aiven.io.INVALID>]

> It's a trade-off between the risks of developing on more recent JDKs while
targeting Java 8, and making maintenance more painful by requiring
development tooling to keep supporting Java 8. I think we should take the
risk.

Sure if someone wants to go ahead then they are more than welcome to try,
just need to be careful that both the paradox docs and binary artifacts are
exactly the same
before and after. From my end we still need to republish paradox 0.9.x on
maven just due to the 1.0.x branches so I will continue to get that done.

I would also posit that a nicer solution to this would also be getting
sbt-multi-release-jar[1]
to work because currently we have a more recurring issue where we wan't
to use features in newer JDK's (such as loom/virtual thread for JDK 21) but
we
can't while supporting JDK 8 without a lot of hackery (if it's even
possible).
Unfortunately I am hitting some fundamental issues[2] on trying to get it
to work

1: https://github.com/sbt/sbt-multi-release-jar and
https://github.com/sbt/sbt/discussions/7174#discussioncomment-5288982
2: https://github.com/sbt/sbt-multi-release-jar/issues/24

On Sat, Feb 10, 2024 at 8:45 PM Arnout Engelen <en...@apache.org> wrote:

> On Sat, Feb 10, 2024 at 8:57 AM Matthew de Detrich
> <ma...@aiven.io.invalid> wrote:
>
> > > I think we should consider requiring Java 11 for all development: the
> > cost/complexity of requiring all our build tools to keep supporting Java
> 8
> > will only increase, and IMHO is unhealthy to the wider ecosystem. The
> > cost/complexity of building with Java 11 and targeting Java 8 is low when
> > using the well-understood compiler flags, except when also using
> > sun.misc.Unsafe, but we have already incurred that cost for that case
> > anyway.
> >
> > the Pekko 1.0.x series is deliberately designed to
> > have minimal intrusive changes merged into it and so for this reason its
> > ideal
> > to have the current build setup remain the same, which means both
> building
> > with JDK 8 (or JDK 11 + JDK 8 workaround for pekko core) and paradox
> 0.9.x.
> >
>
> That makes sense, I'm fine with requiring JDK 11 only for building Pekko
> 1.1.x onwards.
>
>
> > My personal preference is to remain having JDK 1.8 as the preferred one
> > for development as it is now and rather instead spend our efforts to
> > strongly push for dropping JDK 1.8 altogether for Pekko 2.0.x. This would
> > allow
> > us to remove a lot of these hacks which would go a great deal in
> > simplifying the build.
> >
>
> I'm all for dropping Java 8 support completely on Pekko 2.0.x, and indeed
> it would allow us to simplify the build.
>
> I think requiring Java 11 for building Pekko 1.1.x would still be good. By
> the time Pekko 2.0.x happens, we might want to build with even newer
> versions of Java, so most of the work we do now to build with Java 11 and
> target Java 8 will continue to be valuable then as well.
>
> > While it is true the use of Java 8 sadly still seems widespread (
> > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > mentions 33%), I'm sceptical the same holds for developer environments.
> >
> > While I do broadly agree this is accurate, it's still preferable for
> > developers
> > to use the lowest common denominator JDK that the project uses just for
> > the sakes of lowering surprises especially when it comes to releases,
> this
> > is what I do. There are plenty of solutions that allow you to hot swap
> > JDK's
> > on local machines and on this note I was actually considering adding
> > .java-version files like in sbt-multi-release-jar[2] which forces the
> > project
> > to be loaded with a specific JDK. This may be a bit heavy handed but
> > it does a good job of enforcing expectations.
>
>
> It's a trade-off between the risks of developing on more recent JDKs while
> targeting Java 8, and making maintenance more painful by requiring
> development tooling to keep supporting Java 8. I think we should take the
> risk.
>
>
> Kind regards,
>
> Arnout
>
>
> > On Fri, Feb 9, 2024 at 9:57 PM Arnout Engelen <en...@apache.org>
> wrote:
> >
> > > On Wed, Jan 31, 2024 at 8:44 AM Matthew de Detrich
> > > <ma...@aiven.io.invalid> wrote:
> > >
> > > > My main concern is that since we are still supporting JDK 8, building
> > the
> > > > docs with JDK 11 can bring about actual legitimate usability concerns
> > in
> > > > terms
> > > > of consistency because we still have to build the main source code
> with
> > > JDK
> > > > 8
> > > > but with docs we have to build with JDK 11 and so if we do this
> change
> > we
> > > > have a situation where if someone starts sbt with JDK 8 (because they
> > are
> > > > developing normally with the source code) and then do docs/paradox
> they
> > > > will
> > > > get hit with runtime bytecode mismatch errors.
> > > >
> > > > Now this can be solved with the the source and target flags i.e.
> > > >
> > > > javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
> > > >
> > > > Which instructs the JVM to target JDK 1.8 but there are limitations
> > here
> > > > i.e.
> > > > the fact that it doesn't work sun.misc.unsafe[1].
> > > >
> > >
> > > Luckily, only the main pekko repo uses sun.misc.Unsafe (pekko-http
> > > indirectly through pekko.util.Unsafe).
> > >
> > > The main pekko repo *already* supports building with Java 11 (and
> > requires
> > > it for releases) and targeting Java 8. While it is true it uses fairly
> > > horrible hacks to achieve this while also using sun.misc.Unsafe, this
> is
> > > already in place, and other components should be fine with the
> 'regular'
> > > flags for targeting Java 8.
> > >
> > >
> > > > There is also the other issue which is well, the compiled docs would
> be
> > > > slightly
> > > > misleading in the sense that they would link against the JDK 11 docs
> > when
> > > > in reality
> > > > our projects support JDK 8 and these docs do actually differ (JDK8
> docs
> > > > differ
> > > > to JDK11 docs).
> > >
> > >
> > > I don't think this is an issue: while it is true we support Java 8, I'd
> > say
> > > users are more than likely to be on more recent Java versions, in which
> > > case the links to more recent Java versions will be more accurate.
> > >
> > >
> > > > I mentioned this somewhere (can't remember where) but it would
> actually
> > > be
> > > > ideal
> > > > if another release of sbt-paradox from the older series i.e. 0.9.x so
> > it
> > > > can be
> > > > deployed to the newer repositories.
> > > >
> > >
> > > I think we should consider requiring Java 11 for all development: the
> > > cost/complexity of requiring all our build tools to keep supporting
> Java
> > 8
> > > will only increase, and IMHO is unhealthy to the wider ecosystem. The
> > > cost/complexity of building with Java 11 and targeting Java 8 is low
> when
> > > using the well-understood compiler flags, except when also using
> > > sun.misc.Unsafe, but we have already incurred that cost for that case
> > > anyway.
> > >
> > > The main remaining risk I see would be that we'd unintentionally accept
> > an
> > > update of a runtime dependency that drops support for Java 8.
> > >
> > > While it is true the use of Java 8 sadly still seems widespread (
> > > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > > mentions 33%), I'm sceptical the same holds for developer environments.
> > >
> > >
> > > Kind regards,
> > >
> > > Arnout
> > >
> > >
> > > >
> > > > [1]:
> > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
> > > >
> > > >
> > > > On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <fa...@apache.org>
> > wrote:
> > > >
> > > > > We have a slightly untidy sbt plugin setup in our various Pekko
> > repos,
> > > > > where we force the use of old Paradox plugins so that we can use
> Java
> > > > > 8 for doc building.
> > > > >
> > > > > Now that we have the 1.0.x releases done, can we consider switching
> > to
> > > > > the latest Paradox plugins and live with the fact that that means
> we
> > > > > need to use Java 11 for doc builds?
> > > > >
> > > > > Matthias Kurz has been publishing some new versions of these
> plugins
> > > > > because the versions that we use are not in Maven Central meaning
> > that
> > > > > we are dependent on the scala-sbt repository.
> > > > >
> > > > > Any objections to us at least starting with the Pekko repos where
> we
> > > > > have 1.0.x branches? We can update the main branches in these repos
> > to
> > > > > use the latest Paradox plugins and update the build docs.
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Matthew de Detrich
> > > >
> > > > *Aiven Deutschland GmbH*
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Alexanderufer 3-7, 10117 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > *m:* +491603708037
> > > >
> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >
> > >
> > >
> > > --
> > > Arnout Engelen
> > > ASF Security Response
> > > Committer on Apache Pekko
> > > Committer on NixOS
> > > Independent Open Source consultant
> > >
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Alexanderufer 3-7, 10117 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>
>
> --
> Arnout Engelen
> ASF Security Response
> Committer on Apache Pekko
> Committer on NixOS
> Independent Open Source consultant
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Alexanderufer 3-7, 10117 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] require Java 11 for Pekko doc builds

[Arnout Engelen <en...@apache.org>]

On Sat, Feb 10, 2024 at 8:57 AM Matthew de Detrich
<ma...@aiven.io.invalid> wrote:

> > I think we should consider requiring Java 11 for all development: the
> cost/complexity of requiring all our build tools to keep supporting Java 8
> will only increase, and IMHO is unhealthy to the wider ecosystem. The
> cost/complexity of building with Java 11 and targeting Java 8 is low when
> using the well-understood compiler flags, except when also using
> sun.misc.Unsafe, but we have already incurred that cost for that case
> anyway.
>
> the Pekko 1.0.x series is deliberately designed to
> have minimal intrusive changes merged into it and so for this reason its
> ideal
> to have the current build setup remain the same, which means both building
> with JDK 8 (or JDK 11 + JDK 8 workaround for pekko core) and paradox 0.9.x.
>

That makes sense, I'm fine with requiring JDK 11 only for building Pekko
1.1.x onwards.


> My personal preference is to remain having JDK 1.8 as the preferred one
> for development as it is now and rather instead spend our efforts to
> strongly push for dropping JDK 1.8 altogether for Pekko 2.0.x. This would
> allow
> us to remove a lot of these hacks which would go a great deal in
> simplifying the build.
>

I'm all for dropping Java 8 support completely on Pekko 2.0.x, and indeed
it would allow us to simplify the build.

I think requiring Java 11 for building Pekko 1.1.x would still be good. By
the time Pekko 2.0.x happens, we might want to build with even newer
versions of Java, so most of the work we do now to build with Java 11 and
target Java 8 will continue to be valuable then as well.

> While it is true the use of Java 8 sadly still seems widespread (
> https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> mentions 33%), I'm sceptical the same holds for developer environments.
>
> While I do broadly agree this is accurate, it's still preferable for
> developers
> to use the lowest common denominator JDK that the project uses just for
> the sakes of lowering surprises especially when it comes to releases, this
> is what I do. There are plenty of solutions that allow you to hot swap
> JDK's
> on local machines and on this note I was actually considering adding
> .java-version files like in sbt-multi-release-jar[2] which forces the
> project
> to be loaded with a specific JDK. This may be a bit heavy handed but
> it does a good job of enforcing expectations.


It's a trade-off between the risks of developing on more recent JDKs while
targeting Java 8, and making maintenance more painful by requiring
development tooling to keep supporting Java 8. I think we should take the
risk.


Kind regards,

Arnout


> On Fri, Feb 9, 2024 at 9:57 PM Arnout Engelen <en...@apache.org> wrote:
>
> > On Wed, Jan 31, 2024 at 8:44 AM Matthew de Detrich
> > <ma...@aiven.io.invalid> wrote:
> >
> > > My main concern is that since we are still supporting JDK 8, building
> the
> > > docs with JDK 11 can bring about actual legitimate usability concerns
> in
> > > terms
> > > of consistency because we still have to build the main source code with
> > JDK
> > > 8
> > > but with docs we have to build with JDK 11 and so if we do this change
> we
> > > have a situation where if someone starts sbt with JDK 8 (because they
> are
> > > developing normally with the source code) and then do docs/paradox they
> > > will
> > > get hit with runtime bytecode mismatch errors.
> > >
> > > Now this can be solved with the the source and target flags i.e.
> > >
> > > javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
> > >
> > > Which instructs the JVM to target JDK 1.8 but there are limitations
> here
> > > i.e.
> > > the fact that it doesn't work sun.misc.unsafe[1].
> > >
> >
> > Luckily, only the main pekko repo uses sun.misc.Unsafe (pekko-http
> > indirectly through pekko.util.Unsafe).
> >
> > The main pekko repo *already* supports building with Java 11 (and
> requires
> > it for releases) and targeting Java 8. While it is true it uses fairly
> > horrible hacks to achieve this while also using sun.misc.Unsafe, this is
> > already in place, and other components should be fine with the 'regular'
> > flags for targeting Java 8.
> >
> >
> > > There is also the other issue which is well, the compiled docs would be
> > > slightly
> > > misleading in the sense that they would link against the JDK 11 docs
> when
> > > in reality
> > > our projects support JDK 8 and these docs do actually differ (JDK8 docs
> > > differ
> > > to JDK11 docs).
> >
> >
> > I don't think this is an issue: while it is true we support Java 8, I'd
> say
> > users are more than likely to be on more recent Java versions, in which
> > case the links to more recent Java versions will be more accurate.
> >
> >
> > > I mentioned this somewhere (can't remember where) but it would actually
> > be
> > > ideal
> > > if another release of sbt-paradox from the older series i.e. 0.9.x so
> it
> > > can be
> > > deployed to the newer repositories.
> > >
> >
> > I think we should consider requiring Java 11 for all development: the
> > cost/complexity of requiring all our build tools to keep supporting Java
> 8
> > will only increase, and IMHO is unhealthy to the wider ecosystem. The
> > cost/complexity of building with Java 11 and targeting Java 8 is low when
> > using the well-understood compiler flags, except when also using
> > sun.misc.Unsafe, but we have already incurred that cost for that case
> > anyway.
> >
> > The main remaining risk I see would be that we'd unintentionally accept
> an
> > update of a runtime dependency that drops support for Java 8.
> >
> > While it is true the use of Java 8 sadly still seems widespread (
> > https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> > mentions 33%), I'm sceptical the same holds for developer environments.
> >
> >
> > Kind regards,
> >
> > Arnout
> >
> >
> > >
> > > [1]:
> > >
> > >
> >
> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
> > >
> > >
> > > On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <fa...@apache.org>
> wrote:
> > >
> > > > We have a slightly untidy sbt plugin setup in our various Pekko
> repos,
> > > > where we force the use of old Paradox plugins so that we can use Java
> > > > 8 for doc building.
> > > >
> > > > Now that we have the 1.0.x releases done, can we consider switching
> to
> > > > the latest Paradox plugins and live with the fact that that means we
> > > > need to use Java 11 for doc builds?
> > > >
> > > > Matthias Kurz has been publishing some new versions of these plugins
> > > > because the versions that we use are not in Maven Central meaning
> that
> > > > we are dependent on the scala-sbt repository.
> > > >
> > > > Any objections to us at least starting with the Pekko repos where we
> > > > have 1.0.x branches? We can update the main branches in these repos
> to
> > > > use the latest Paradox plugins and update the build docs.
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > >
> > > >
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Alexanderufer 3-7, 10117 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
> >
> >
> > --
> > Arnout Engelen
> > ASF Security Response
> > Committer on Apache Pekko
> > Committer on NixOS
> > Independent Open Source consultant
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Alexanderufer 3-7, 10117 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>


-- 
Arnout Engelen
ASF Security Response
Committer on Apache Pekko
Committer on NixOS
Independent Open Source consultant

Re: [DISCUSS] require Java 11 for Pekko doc builds

[Matthew de Detrich <ma...@aiven.io.INVALID>]

> I don't think this is an issue: while it is true we support Java 8, I'd
say
users are more than likely to be on more recent Java versions, in which
case the links to more recent Java versions will be more accurate.

This was actually brought up before when we asked to drop support for JDK
1.8
and what we learnt is that people seriously under-estimate how many
companies/users are stuck on JDK 1.8, these even include companies where
current frequent Pekko contributors work at.

Unfortunately it's not really the developers who have any control here,
otherwise
we would probably be using JDK 17/21 already but it is what it is.

> I think we should consider requiring Java 11 for all development: the
cost/complexity of requiring all our build tools to keep supporting Java 8
will only increase, and IMHO is unhealthy to the wider ecosystem. The
cost/complexity of building with Java 11 and targeting Java 8 is low when
using the well-understood compiler flags, except when also using
sun.misc.Unsafe, but we have already incurred that cost for that case
anyway.

In the grand scheme of things supporting JDK 1.8 hasn't been that much
of an issue, there are for more pressing things that are creating a much
bigger maintenance burden. The recent issue we are dealing with regarding
moving away from Lightbend/Typesafe repo to maven is not even directly
related to JDK 1.8, it's just that we have this transition period where we
need to redeploy artifacts on maven central which I have
been helping out with directly. The easiest solution to this specific
problem
is to redeploy the paradox 0.9.x series onto maven[1].

I don't want to stop others from helping out building the project with JDK
11+
but we have tried doing this in the past and realized the it is very
brittle,
we ended up breaking the doc generation (or the build) in weird and subtle
ways.
The other point is that the Pekko 1.0.x series is deliberately designed to
have
minimal intrusive changes merged into it and so for this reason its ideal
to have
the current build setup remain the same, which means both building with
JDK 8 (or JDK 11 + JDK 8 workaround for pekko core) and paradox 0.9.x.

My personal preference is to remain having JDK 1.8 as the preferred one
for development as it is now and rather instead spend our efforts to
strongly
push for dropping JDK 1.8 altogether for Pekko 2.0.x. This would allow
us to remove a lot of these hacks which would go a great deal in
simplifying the build.

> While it is true the use of Java 8 sadly still seems widespread (
https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
mentions 33%), I'm sceptical the same holds for developer environments.

While I do broadly agree this is accurate, it's still preferable for
developers
to use the lowest common denominator JDK that the project uses just for
the sakes of lowering surprises especially when it comes to releases, this
is what I do. There are plenty of solutions that allow you to hot swap JDK's
on local machines and on this note I was actually considering adding
.java-version files like in sbt-multi-release-jar[2] which forces the
project
to be loaded with a specific JDK. This may be a bit heavy handed but
it does a good job of enforcing expectations.

1: https://github.com/lightbend/paradox/issues/601
2: https://github.com/sbt/sbt-multi-release-jar/blob/main/.java-version

On Fri, Feb 9, 2024 at 9:57 PM Arnout Engelen <en...@apache.org> wrote:

> On Wed, Jan 31, 2024 at 8:44 AM Matthew de Detrich
> <ma...@aiven.io.invalid> wrote:
>
> > My main concern is that since we are still supporting JDK 8, building the
> > docs with JDK 11 can bring about actual legitimate usability concerns in
> > terms
> > of consistency because we still have to build the main source code with
> JDK
> > 8
> > but with docs we have to build with JDK 11 and so if we do this change we
> > have a situation where if someone starts sbt with JDK 8 (because they are
> > developing normally with the source code) and then do docs/paradox they
> > will
> > get hit with runtime bytecode mismatch errors.
> >
> > Now this can be solved with the the source and target flags i.e.
> >
> > javacOptions ++= Seq("-source", "1.8", "-target", "1.8")
> >
> > Which instructs the JVM to target JDK 1.8 but there are limitations here
> > i.e.
> > the fact that it doesn't work sun.misc.unsafe[1].
> >
>
> Luckily, only the main pekko repo uses sun.misc.Unsafe (pekko-http
> indirectly through pekko.util.Unsafe).
>
> The main pekko repo *already* supports building with Java 11 (and requires
> it for releases) and targeting Java 8. While it is true it uses fairly
> horrible hacks to achieve this while also using sun.misc.Unsafe, this is
> already in place, and other components should be fine with the 'regular'
> flags for targeting Java 8.
>
>
> > There is also the other issue which is well, the compiled docs would be
> > slightly
> > misleading in the sense that they would link against the JDK 11 docs when
> > in reality
> > our projects support JDK 8 and these docs do actually differ (JDK8 docs
> > differ
> > to JDK11 docs).
>
>
> I don't think this is an issue: while it is true we support Java 8, I'd say
> users are more than likely to be on more recent Java versions, in which
> case the links to more recent Java versions will be more accurate.
>
>
> > I mentioned this somewhere (can't remember where) but it would actually
> be
> > ideal
> > if another release of sbt-paradox from the older series i.e. 0.9.x so it
> > can be
> > deployed to the newer repositories.
> >
>
> I think we should consider requiring Java 11 for all development: the
> cost/complexity of requiring all our build tools to keep supporting Java 8
> will only increase, and IMHO is unhealthy to the wider ecosystem. The
> cost/complexity of building with Java 11 and targeting Java 8 is low when
> using the well-understood compiler flags, except when also using
> sun.misc.Unsafe, but we have already incurred that cost for that case
> anyway.
>
> The main remaining risk I see would be that we'd unintentionally accept an
> update of a runtime dependency that drops support for Java 8.
>
> While it is true the use of Java 8 sadly still seems widespread (
> https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> mentions 33%), I'm sceptical the same holds for developer environments.
>
>
> Kind regards,
>
> Arnout
>
>
> >
> > [1]:
> >
> >
> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
> >
> >
> > On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <fa...@apache.org> wrote:
> >
> > > We have a slightly untidy sbt plugin setup in our various Pekko repos,
> > > where we force the use of old Paradox plugins so that we can use Java
> > > 8 for doc building.
> > >
> > > Now that we have the 1.0.x releases done, can we consider switching to
> > > the latest Paradox plugins and live with the fact that that means we
> > > need to use Java 11 for doc builds?
> > >
> > > Matthias Kurz has been publishing some new versions of these plugins
> > > because the versions that we use are not in Maven Central meaning that
> > > we are dependent on the scala-sbt repository.
> > >
> > > Any objections to us at least starting with the Pekko repos where we
> > > have 1.0.x branches? We can update the main branches in these repos to
> > > use the latest Paradox plugins and update the build docs.
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> > >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Alexanderufer 3-7, 10117 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>
>
> --
> Arnout Engelen
> ASF Security Response
> Committer on Apache Pekko
> Committer on NixOS
> Independent Open Source consultant
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Alexanderufer 3-7, 10117 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] require Java 11 for Pekko doc builds

[Arnout Engelen <en...@apache.org>]

On Fri, Feb 9, 2024 at 11:56 AM Arnout Engelen <en...@apache.org> wrote:

> I think we should consider requiring Java 11 for all development (...)
>
> The main remaining risk I see would be that we'd unintentionally accept an
> update of a runtime dependency that drops support for Java 8.
>

I think this is now the main roadblock - if anyone has good ideas on how to
achieve this I'm all ears.

I created https://github.com/apache/incubator-pekko/issues/1118 to track
the tasks/opportunities that are unlocked once we make the decision to
require Java 11 for building Pekko.


Kind regards,

Arnout


>
> While it is true the use of Java 8 sadly still seems widespread (
> https://newrelic.com/resources/report/2023-state-of-the-java-ecosystem
> mentions 33%), I'm sceptical the same holds for developer environments.
>
>
> Kind regards,
>
> Arnout
>
>
>>
>> [1]:
>>
>> https://github.com/apache/incubator-pekko/blob/bcec7c0fa0cd9f68c4858d168f0dfadcb3ba4602/project/JdkOptions.scala#L71-L82
>>
>>
>> On Wed, Jan 31, 2024 at 2:48 AM PJ Fanning <fa...@apache.org> wrote:
>>
>> > We have a slightly untidy sbt plugin setup in our various Pekko repos,
>> > where we force the use of old Paradox plugins so that we can use Java
>> > 8 for doc building.
>> >
>> > Now that we have the 1.0.x releases done, can we consider switching to
>> > the latest Paradox plugins and live with the fact that that means we
>> > need to use Java 11 for doc builds?
>> >
>> > Matthias Kurz has been publishing some new versions of these plugins
>> > because the versions that we use are not in Maven Central meaning that
>> > we are dependent on the scala-sbt repository.
>> >
>> > Any objections to us at least starting with the Pekko repos where we
>> > have 1.0.x branches? We can update the main branches in these repos to
>> > use the latest Paradox plugins and update the build docs.
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>> > For additional commands, e-mail: dev-help@pekko.apache.org
>> >
>> >
>>
>> --
>>
>> Matthew de Detrich
>>
>> *Aiven Deutschland GmbH*
>>
>> Immanuelkirchstraße 26, 10405 Berlin
>>
>> Alexanderufer 3-7, 10117 Berlin
>>
>> Amtsgericht Charlottenburg, HRB 209739 B
>>
>> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>
>> *m:* +491603708037
>>
>> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>>
>
>
> --
> Arnout Engelen
> ASF Security Response
> Committer on Apache Pekko
> Committer on NixOS
> Independent Open Source consultant
>


-- 
Arnout Engelen
ASF Security Response
Committer on Apache Pekko
Committer on NixOS
Independent Open Source consultant