You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pekko.apache.org by "Claude Warren, Jr" <cl...@aiven.io.INVALID> on 2023/05/23 10:38:30 UTC

[DISCUSS] RC1 build

Greetings,


I think that it is time to start to take a hard look at a release
candidate.  I do not expect the release candidate to succeed, however,
there is a lot to be learned in the failure.

There is a milestone [1] currently open.  Let's take that and work toward
the release candidate.

I think we should start by determining if there are items that are not
required for the release and items that are missing.  I am opening two
issues today that I will add to the milestone.  They are the addition of
the LICENSE and NOTICE files to the snapshots, under the assumption that
doing so will also add them to the release.  Let's clean the list and then
concentrate on only those items that will get us to RC1.  So let's hold a
vote on the items in the current milestone and see if they all need to be
in release 1.0

In this process we will need to ensure that everyone is communicating what
they are doing.  So, I think we should do the following:
  1. do not work on a change without a ticket.
  2. make sure the ticket is associated with the milestone.
  3. when you are working on a ticket, assign it to yourself.  If you can
not assign it to yourself, post a note on the dev mailing list with a
subject prefix of [ACCESS] noting the ticket that you do not have access to
and then add a comment to the ticket indicating that you are working on it.
  4. any change that is not a simple break/fix should be discussed in the
dev mail list with the subject prefix of [DISUCSS].

We should also be looking on the dev list for offers of assistance and
point them to the milestone first.  We need to develop standard operating
procedures that are welcoming to new contributors.

[1] https://github.com/apache/incubator-pekko/milestone/1

Re: [DISCUSS] Building Binary Package

Posted by Arnout Engelen <en...@apache.org>.
On Tue, May 23, 2023 at 3:39 PM Matthew Benedict de Detrich
<ma...@aiven.io.invalid> wrote:
> The Pekko SBT build is currently configured to detect Java 8 installation via
> the JAVA_8_HOME environment variable (see
> https://github.com/apache/incubator-pekko/blob/85c2a469098abd064738dbf9bb50f3cc271dc73b/project/JdkOptions.scala#L94
> ).
>
> The reason why the ubuntu docker images/github actions CI can build Pekko
> without any additional JDK configuration/work is
> that the ubuntu image is already preinstalled with various JDK's installed
> with the relevant JAVA_<VERSION>_HOME
> environment variables exposed.

This might be drifting off-topic/too far down the rabbithole, but:
when JAVA_8_HOME is not set the build will fall back to auto-detecting
common JDK 8 installation directories using (ultimately) the logic at
https://github.com/sbt/sbt/blob/71adb62d86b976e0e08cbffbe66a1b56031d36fd/main/src/main/scala/sbt/internal/CrossJava.scala#L485
. I'm fairly sure Ubuntu doesn't set JAVA_8_HOME, but that logic
correctly discovers JDKs installed from the Ubuntu packages, so all is
well in the end :)


Kind regards,

Arnout

> On Tue, May 23, 2023 at 3:28 PM Claude Warren, Jr
> <cl...@aiven.io.invalid> wrote:
>
> > I have seen other packages that required the rt.jar from java8 be installed
> > in a specific directory for the build.  Something like the /lib directory.
> > I am trying to find that project to see if I can figure out what they did
> > and how it might work for Pekko.
> >
> >
> > On Tue, May 23, 2023 at 2:09 PM Matthew Benedict de Detrich
> > <ma...@aiven.io.invalid> wrote:
> >
> > > > But more to the point here, is there a Dockerfile to build the Docker
> > > image?
> > >
> > > There is an in progress PR at
> > > https://github.com/apache/incubator-pekko/pull/188
> > >
> > > >  Can we spell out how to configure a standard linux system to build
> > > Pekko without the docker image?
> > >
> > > Not entirely sure what you mean by "standard linux system to build".
> > > Linux's have
> > > different distributions/package managers and most developers install
> > things
> > > like
> > > OpenJDK via a package manager. While that in of itself is not
> > problematic,
> > > the issue
> > > here is the "multiple installs" part (or to be more accurate you need
> > > OpenJDK 11
> > > to be installed and you have to download a distribution of OpenJDK 8 to
> > get
> > > the
> > > rt.jar so that Pekko can compile sources that refer to sun.misc.unsafe
> > and
> > > feed that
> > > into sbt). There are other solutions like sdkman which can handle
> > multiple
> > > JDK
> > > installs, but the obvious issue here is the explosion of installation
> > > instructions (
> > > also this presumes that developers primary machines are Linux).
> > >
> > > The one big advantage that the Docker build has (aside from it just
> > working
> > > on any major OS) is that it uses the same docker image that github
> > actions
> > > does
> > > (at least this appears to be the case from a cursory glance).
> > >
> > > I will try and spend some time getting
> > > https://github.com/apache/incubator-pekko/pull/188
> > > finished and merged this week.
> > >
> > > On Tue, May 23, 2023 at 3:00 PM Claude Warren, Jr
> > > <cl...@aiven.io.invalid> wrote:
> > >
> > > > >
> > > > > Also I would like to point out/remind that even though making a
> > source
> > > > > package release is important
> > > > > for Apache, it's useless for Pekko's users. It's been stated many
> > times
> > > > on
> > > > > the mailing list that the people
> > > > > actually using Pekko ultimately only care about the maven binary jars
> > > > (i.e.
> > > > > 1.0.0 release) and that a lot
> > > > > of potential contributors are waiting for that 1.0.0 maven jar binary
> > > > > release. The ironic thing here is that
> > > > > we already have a method of creating proper artifacts, it's just done
> > > by
> > > > > github actions CI which doesn't
> > > > > adhere to Apache's processes for a release and that is an example of
> > > > what's
> > > > > making it more complicated
> > > > > (otherwise we could have been pushing RC's weeks/months ago).
> > > > >
> > > > >
> > > > I want to point out that most of the projects I work on are libraries,
> > > and
> > > > as you say most users just want the binary packages.
> > > >
> > > > But more to the point here, is there a Dockerfile to build the Docker
> > > > image?   Can we spell out how to configure a standard linux system to
> > > build
> > > > Pekko without the docker image?
> > > >
> > > > Claude

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
> I have seen other packages that required the rt.jar from java8 be
installed
in a specific directory for the build.  Something like the /lib directory.
I am trying to find that project to see if I can figure out what they did
and how it might work for Pekko.

TPekko SBT build is currently configured to detect Java 8 installation via
the JAVA_8_HOME environment variable (see
https://github.com/apache/incubator-pekko/blob/85c2a469098abd064738dbf9bb50f3cc271dc73b/project/JdkOptions.scala#L94
).

The reason why the ubuntu docker images/github actions CI can build Pekko
without any additional JDK configuration/work is
that the ubuntu image is already preinstalled with various JDK's installed
with the relevant JAVA_<VERSION>_HOME
environment variables exposed.

On Tue, May 23, 2023 at 3:28 PM Claude Warren, Jr
<cl...@aiven.io.invalid> wrote:

> I have seen other packages that required the rt.jar from java8 be installed
> in a specific directory for the build.  Something like the /lib directory.
> I am trying to find that project to see if I can figure out what they did
> and how it might work for Pekko.
>
>
> On Tue, May 23, 2023 at 2:09 PM Matthew Benedict de Detrich
> <ma...@aiven.io.invalid> wrote:
>
> > > But more to the point here, is there a Dockerfile to build the Docker
> > image?
> >
> > There is an in progress PR at
> > https://github.com/apache/incubator-pekko/pull/188
> >
> > >  Can we spell out how to configure a standard linux system to build
> > Pekko without the docker image?
> >
> > Not entirely sure what you mean by "standard linux system to build".
> > Linux's have
> > different distributions/package managers and most developers install
> things
> > like
> > OpenJDK via a package manager. While that in of itself is not
> problematic,
> > the issue
> > here is the "multiple installs" part (or to be more accurate you need
> > OpenJDK 11
> > to be installed and you have to download a distribution of OpenJDK 8 to
> get
> > the
> > rt.jar so that Pekko can compile sources that refer to sun.misc.unsafe
> and
> > feed that
> > into sbt). There are other solutions like sdkman which can handle
> multiple
> > JDK
> > installs, but the obvious issue here is the explosion of installation
> > instructions (
> > also this presumes that developers primary machines are Linux).
> >
> > The one big advantage that the Docker build has (aside from it just
> working
> > on any major OS) is that it uses the same docker image that github
> actions
> > does
> > (at least this appears to be the case from a cursory glance).
> >
> > I will try and spend some time getting
> > https://github.com/apache/incubator-pekko/pull/188
> > finished and merged this week.
> >
> > On Tue, May 23, 2023 at 3:00 PM Claude Warren, Jr
> > <cl...@aiven.io.invalid> wrote:
> >
> > > >
> > > > Also I would like to point out/remind that even though making a
> source
> > > > package release is important
> > > > for Apache, it's useless for Pekko's users. It's been stated many
> times
> > > on
> > > > the mailing list that the people
> > > > actually using Pekko ultimately only care about the maven binary jars
> > > (i.e.
> > > > 1.0.0 release) and that a lot
> > > > of potential contributors are waiting for that 1.0.0 maven jar binary
> > > > release. The ironic thing here is that
> > > > we already have a method of creating proper artifacts, it's just done
> > by
> > > > github actions CI which doesn't
> > > > adhere to Apache's processes for a release and that is an example of
> > > what's
> > > > making it more complicated
> > > > (otherwise we could have been pushing RC's weeks/months ago).
> > > >
> > > >
> > > I want to point out that most of the projects I work on are libraries,
> > and
> > > as you say most users just want the binary packages.
> > >
> > > But more to the point here, is there a Dockerfile to build the Docker
> > > image?   Can we spell out how to configure a standard linux system to
> > build
> > > Pekko without the docker image?
> > >
> > > Claude
> > >
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] Building Binary Package

Posted by "Claude Warren, Jr" <cl...@aiven.io.INVALID>.
I have seen other packages that required the rt.jar from java8 be installed
in a specific directory for the build.  Something like the /lib directory.
I am trying to find that project to see if I can figure out what they did
and how it might work for Pekko.


On Tue, May 23, 2023 at 2:09 PM Matthew Benedict de Detrich
<ma...@aiven.io.invalid> wrote:

> > But more to the point here, is there a Dockerfile to build the Docker
> image?
>
> There is an in progress PR at
> https://github.com/apache/incubator-pekko/pull/188
>
> >  Can we spell out how to configure a standard linux system to build
> Pekko without the docker image?
>
> Not entirely sure what you mean by "standard linux system to build".
> Linux's have
> different distributions/package managers and most developers install things
> like
> OpenJDK via a package manager. While that in of itself is not problematic,
> the issue
> here is the "multiple installs" part (or to be more accurate you need
> OpenJDK 11
> to be installed and you have to download a distribution of OpenJDK 8 to get
> the
> rt.jar so that Pekko can compile sources that refer to sun.misc.unsafe and
> feed that
> into sbt). There are other solutions like sdkman which can handle multiple
> JDK
> installs, but the obvious issue here is the explosion of installation
> instructions (
> also this presumes that developers primary machines are Linux).
>
> The one big advantage that the Docker build has (aside from it just working
> on any major OS) is that it uses the same docker image that github actions
> does
> (at least this appears to be the case from a cursory glance).
>
> I will try and spend some time getting
> https://github.com/apache/incubator-pekko/pull/188
> finished and merged this week.
>
> On Tue, May 23, 2023 at 3:00 PM Claude Warren, Jr
> <cl...@aiven.io.invalid> wrote:
>
> > >
> > > Also I would like to point out/remind that even though making a source
> > > package release is important
> > > for Apache, it's useless for Pekko's users. It's been stated many times
> > on
> > > the mailing list that the people
> > > actually using Pekko ultimately only care about the maven binary jars
> > (i.e.
> > > 1.0.0 release) and that a lot
> > > of potential contributors are waiting for that 1.0.0 maven jar binary
> > > release. The ironic thing here is that
> > > we already have a method of creating proper artifacts, it's just done
> by
> > > github actions CI which doesn't
> > > adhere to Apache's processes for a release and that is an example of
> > what's
> > > making it more complicated
> > > (otherwise we could have been pushing RC's weeks/months ago).
> > >
> > >
> > I want to point out that most of the projects I work on are libraries,
> and
> > as you say most users just want the binary packages.
> >
> > But more to the point here, is there a Dockerfile to build the Docker
> > image?   Can we spell out how to configure a standard linux system to
> build
> > Pekko without the docker image?
> >
> > Claude
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>

Re: [DISCUSS] Building Binary Package

Posted by PJ Fanning <fa...@gmail.com>.
Mathew - that's a reasonable point about the latest JDKs (18+). We
don't actually build with these in CI builds. There are changes in
these JDKs to try to discourage the use of some old classes. In our
build docs, it might be simpler for everyone if we recommended Java 11
and Java 8 specifically instead of leaving some leeway for users to
choose the JDK versions to use.

On Thu, 25 May 2023 at 15:51, Matthew Benedict de Detrich
<ma...@aiven.io.invalid> wrote:
>
> > Have we seen anyone run into problems
> (re)building from source?
>
> The only issue I have found is if you try to build Pekko with even newer
> JDK's, such as JDK 18. Claude was trying this out and he came across this
> error
>
> [error]
> /home/claude/apache/incubator-pekko/actor/src/main/scala/org/apache/pekko/actor/LightArrayRevolverScheduler.scala:350:41:
> method objectFieldOffset in class Unsafe is deprecated
> [error]   private[this] val taskOffset =
> unsafe.objectFieldOffset(classOf[TaskHolder].getDeclaredField("task"))
> [error]                                         ^
> [error] one error found
>
> Which is probably due to JDK 18 deprecating even more methods from
> sun.misc.unsafe. My immediate low hanging fruit for this is to detect if
> sbt the JDK being used is 18 or newer and just exit out with a helpful
> error (in the same way that it exits out right now if use JDK 11 without
> detecting an JDK 8 rt.jar)
>
> > Why not build 2 binary packages:  Java 8 and Java 11?  Use the java 8 JDK
> to build the java 8 version and Java 11 JDK to build java 11 version.
>
> I think that for all intents and purposes having multiple artifacts for
> different JDK's is out of the question, both for Scala ecosystem reasons (I
> can't think of any other OS Scala project that does this) or for user
> experience (I don't see how you can correctly handle transitive
> dependencies unless you want to add yet another axis to Scala's build
> matrix).
>
> Apparently there is a way to generate multi release jars (see
> https://openjdk.org/jeps/238) however sbt doesn't support that yet although
> it was raised for sbt 2.0.x (see
> https://github.com/sbt/sbt/discussions/7174#discussioncomment-5288982).
>
> This is however detracting a bit, as others said the most rational course
> right now is to just document and help
>
> On Thu, May 25, 2023 at 4:35 PM Arnout Engelen <en...@apache.org> wrote:
>
> > On Thu, May 25, 2023 at 4:29 PM PJ Fanning <fa...@gmail.com> wrote:
> > > Building from source is not as hard as some people are assuming it is.
> > We can document it. We can
> > > answer questions for people who are having issues.
> >
> > This is my impression as well. Have we seen anyone run into problems
> > (re)building from source? I know we're planning further improvements
> > but https://github.com/apache/incubator-pekko/#building-from-source
> > already looks fairly good to me, I'd be interested to learn
> > how/where/why people would get 'stuck'.
> >
> >
> > Kind regards,
> >
> > Arnout
> >
> > > On Thu, 25 May 2023 at 15:18, Claude Warren, Jr
> > > <cl...@aiven.io.invalid> wrote:
> > > >
> > > > Why not build 2 binary packages:  Java 8 and Java 11?  Use the java 8
> > JDK
> > > > to build the java 8 version and Java 11 JDK to build java 11 version.
> > > >
> > > > Then no need to copy rt.jar
> > > >
> > > > On Thu, May 25, 2023 at 2:35 PM Arnout Engelen <en...@apache.org>
> > wrote:
> > > >
> > > > > On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr
> > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > Assume Java 11 is installed and no Java 8 is installed on the
> > system
> > > > > where
> > > > > > Pekko is being deployed.
> > > > > > Assume Java 11 compilation with rt.jar present for the build of
> > the Pekko
> > > > > > package being deployed. (My understanding that even with Java 11
> > the
> > > > > rt.jar
> > > > > > is required for compilation)
> > > > > >
> > > > > > When the code is executed on the deployed system and the class
> > from the
> > > > > > rt.jar is required, what happens?
> > > > > > I would expect that a NoClassDefFoundError to be thrown.   If the
> > error
> > > > > is
> > > > > > not thrown I can only see two possibilities:
> > > > > >
> > > > > >    1. the class(es) from the rt.jar are defined somewhere else in
> > which
> > > > > >    case why can that not be used during the build?
> > > > > >    2. the class(es) are not used and the dependency on the rt.jar
> > can be
> > > > > >    removed.
> > > > > >
> > > > > > So what happens and why?
> > > > >
> > > > > No NoClassDefFoundError will be thrown, because the runtime for Java
> > > > > 11 is backwards compatible with the runtime for Java 8, so there
> > > > > should be no classes/methods that exist in the jdk8 rt.jar that are
> > > > > not also available in the jdk11 runtime.
> > > > >
> > > > > From your two possibilities, '1' is true: they are defined both in
> > the
> > > > > Java 8 and in the Java 11 runtimes. Indeed if we removed the rt.jar
> > > > > everything would still work on Java 11. Just no longer on Java 8.
> > This
> > > > > is because the runtimes are not *forwards* compatible: there are
> > > > > things in the Java 11 runtime that are not in the Java 8 one.
> > > > >
> > > > > To illustrate and make this more tangible, one specific example is
> > > > > ByteBuffer.clear(): on Java 8, this method returned a Buffer. On Java
> > > > > 11, there is an implementation that returns a ByteBuffer. If you
> > > > > compile a class that calls this method against the Java 8 runtime and
> > > > > then run it against the Java 11 one, this is fine, because Java 11
> > has
> > > > > *both* a clear() method that returns a ByteBuffer and (for backwards
> > > > > compatibility) one that returns a Buffer. If you compile the same
> > code
> > > > > against the Java 11 runtime and then run it against Java 8, it will
> > > > > pick the one that returns a ByteBuffer at compile time, and you will
> > > > > get a NoSuchMethod exception at runtime because this new method does
> > > > > not exist in the Java 8 runtime.
> > > > >
> > > > > Of course we could change the code to make sure this *particular*
> > > > > problem does not occur in Pekko, but that is risky as we don't know
> > > > > how many similar situations there are.
> > > > >
> > > > > Is there a better solution to this than compiling with two JVM's
> > > > > installed? Yes! Invoking the JDK11 compiler with '--release 8' will
> > > > > also generate the correct reference. However, '--release 8' does not
> > > > > allow access to sun.misc.Unsafe API's, which unfortunately Pekko
> > still
> > > > > relies on - so we can't do that just yet.
> > > > >
> > > > >
> > > > > Kind regards,
> > > > >
> > > > > Arnout
> > > > >
> > > > > > On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich
> > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > >
> > > > > > > > If you build Pekko with both Java 11 and Java 8 installed, the
> > > > > classes
> > > > > > > we will produce in our jars will be in Java 8 class format.
> > > > > > >
> > > > > > > More specifically, normally if you compile Java/Scala source
> > with JDK
> > > > > 11
> > > > > > > (which is what Pekko does, JDK 8 is only necessary for its
> > rt.jar) it
> > > > > will
> > > > > > > produce
> > > > > > > classfiles compatible with JDK 11 and above. You can however
> > provide a
> > > > > flag
> > > > > > > to
> > > > > > > javac/scalac so that it generates class files with a specific
> > bytecode
> > > > > > > format, i.e.
> > > > > > >
> > > > > > >
> > > > >
> > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60
> > > > > > > and
> > > > > > >
> > > > > > >
> > > > >
> > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73
> > > > > > > does.
> > > > > > >
> > > > > > > In short, Pekko is built using JDK 11 with Xtarget/release flags
> > to
> > > > > force
> > > > > > > JDK 8
> > > > > > > compatible bytecode generation in addition to providing the JDK8
> > > > > rt.jar via
> > > > > > > the
> > > > > > > -bootclasspath javac flag.
> > > > > > >
> > > > > > > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <fa...@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > > Q: does the build package the rt.jar in the final binary
> > package?
> > > > > > > >
> > > > > > > > No. We only produce our own jars that we push to Maven - and
> > these do
> > > > > > > > not embed any rt.jar classes (shaded or otherwise). We have no
> > other
> > > > > > > > type of 'binary' package.
> > > > > > > >
> > > > > > > > > Q: if it does not, what happens if you try to run using Java
> > 11 on
> > > > > a
> > > > > > > > system
> > > > > > > > > that does not have Java 8 nor the rt.jar installed?
> > > > > > > >
> > > > > > > > Java compiler produces class files. These have a Java version
> > > > > > > > associated with them and you can't use a class that is in Java
> > 11
> > > > > > > > format in a Java 8 runtime.
> > > > > > > > If you build Pekko with both Java 11 and Java 8 installed, the
> > > > > classes
> > > > > > > > we will produce in our jars will be in Java 8 class format.
> > > > > > > >
> > > > > > > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich
> > > > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > > > >
> > > > > > > > > afaik Pekko will just fail to build at all if you use JDK 11
> > > > > without
> > > > > > > > > pointing to JDK 8 rt.jar.
> > > > > > > > >
> > > > > > > > > This is because Pekko uses sun.misc.unsafe which is only
> > shipping
> > > > > with
> > > > > > > > JDK
> > > > > > > > > 8 and the
> > > > > > > > > reason why we have to build on JDK 11 is because there are
> > features
> > > > > > > > > shipping in JDK 11
> > > > > > > > > which pekko uses.
> > > > > > > > >
> > > > > > > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
> > > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > >
> > > > > > > > > > I have a couple of questions:
> > > > > > > > > >
> > > > > > > > > > Q: does the build package the rt.jar in the final binary
> > package?
> > > > > > > > > >
> > > > > > > > > > Q: if it does not, what happens if you try to run using
> > Java 11
> > > > > on a
> > > > > > > > system
> > > > > > > > > > that does not have Java 8 nor the rt.jar installed?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> > > > > > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > > > > >
> > > > > > > > > > > > We could provide a Docker image with the right sbt and
> > Java
> > > > > setup
> > > > > > > > for
> > > > > > > > > > > convenience but I think it is fair not to require users
> > to use
> > > > > > > > Docker.
> > > > > > > > > > >
> > > > > > > > > > > Considering how much implicit behaviour there is in
> > making a
> > > > > full
> > > > > > > > build
> > > > > > > > > > > (as Arnout mentioned earlier, you can compile Pekko fine
> > with
> > > > > just
> > > > > > > > JDK 8
> > > > > > > > > > > just fine however it will be missing post java 8 optional
> > > > > > > features) I
> > > > > > > > > > would
> > > > > > > > > > > prefer
> > > > > > > > > > > to strongly encourage people to use Docker to do a proper
> > > > > build.
> > > > > > > As a
> > > > > > > > > > bare
> > > > > > > > > > > minimum we can also print a warning in sbt telling users
> > if
> > > > > they
> > > > > > > are
> > > > > > > > not
> > > > > > > > > > > going
> > > > > > > > > > > to be making a complete Pekko build when building on JDK
> > 8 (as
> > > > > > > > stated it
> > > > > > > > > > > already errors out if you build on JDK 11 without
> > pointing to
> > > > > a JDK
> > > > > > > > > > > rt.jar).
> > > > > > > > > > >
> > > > > > > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <
> > > > > fanningpj@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Thanks Arnout.
> > > > > > > > > > > >
> > > > > > > > > > > > I think if we clearly document what you need to set up
> > to do
> > > > > a
> > > > > > > > build,
> > > > > > > > > > > > that should be enough. I think it is fair to assume
> > that
> > > > > users
> > > > > > > can
> > > > > > > > > > > > install Java and sbt - and that they should be able to
> > > > > install 2
> > > > > > > > > > > > versions of Java if we are explicit about how to do
> > this and
> > > > > why
> > > > > > > > this
> > > > > > > > > > > > is the best setup for building Pekko.
> > > > > > > > > > > >
> > > > > > > > > > > > We will need to expand what we have in
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > > > > > > > > > > >
> > > > > > > > > > > > This README file will appear in the source releases.
> > > > > > > > > > > >
> > > > > > > > > > > > We could provide a Docker image with the right sbt and
> > Java
> > > > > setup
> > > > > > > > for
> > > > > > > > > > > > convenience but I think it is fair not to require
> > users to
> > > > > use
> > > > > > > > Docker.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <
> > > > > engelen@apache.org
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <
> > > > > > > fanningpj@gmail.com
> > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > > Ok. It seems the nightly job to publish the
> > snapshot jars
> > > > > > > uses
> > > > > > > > Java
> > > > > > > > > > > > > > 11. It does not explicitly install Java 8 but
> > maybe, it's
> > > > > > > > possible
> > > > > > > > > > > > > > that Java 8 is installed silently,
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > > > > > > > > > > suggests so, yes.
> > > > > > > > > > > > >
> > > > > > > > > > > > > > The nightly build does produce jars that can be
> > used by
> > > > > Java
> > > > > > > 8.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > So if this nightly build relies on the silent
> > > > > installation of
> > > > > > > > Java
> > > > > > > > > > 8,
> > > > > > > > > > > > > > should we add a check in our sbt script that will
> > fail if
> > > > > > > Java
> > > > > > > > 8 is
> > > > > > > > > > > > > > not found? This could be useful for users who are
> > > > > building on
> > > > > > > > their
> > > > > > > > > > > > > > own machines.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This check already exists: when I build with jdk11
> > without
> > > > > > > having
> > > > > > > > > > jdk8
> > > > > > > > > > > > > available the build fails with:
> > > > > > > > > > > > >
> > > > > > > > > > > > > [error] A JDK 8 installation was not found, but is
> > > > > required to
> > > > > > > > build
> > > > > > > > > > > > > Apache Pekko. To manually specify a JDK 8
> > installation,
> > > > > use the
> > > > > > > > "set
> > > > > > > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If
> > you
> > > > > have no
> > > > > > > > JDK 8
> > > > > > > > > > > > > installation, target your system JDK with the "set
> > every
> > > > > > > > > > > > > targetSystemJdk := true" sbt command, but beware
> > resulting
> > > > > > > > artifacts
> > > > > > > > > > > > > will not work on JDK 8
> > > > > > > > > > > > >
> > > > > > > > > > > > > It would be good to also document the JAVA_8_HOME
> > > > > environment
> > > > > > > > > > variable
> > > > > > > > > > > > > in this error message. Created
> > > > > > > > > > > > > https://github.com/apache/incubator-pekko/pull/333
> > for
> > > > > that.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Arnout
> > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <
> > > > > > > > engelen@apache.org>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <
> > > > > > > > fanningpj@gmail.com
> > > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > Only building the website docs needs Java 11
> > (and
> > > > > up).
> > > > > > > > > > Everything
> > > > > > > > > > > > else
> > > > > > > > > > > > > > > > builds with Java 8 (and up).
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Yes, though builds created with only Java 8 will
> > miss
> > > > > some
> > > > > > > > > > optional
> > > > > > > > > > > > > > > post-Java-8 features/implementations for the
> > > > > "actor-typed",
> > > > > > > > > > > "stream",
> > > > > > > > > > > > > > > "remote" and "cluster-sharding" modules - so for
> > a
> > > > > faithful
> > > > > > > > test
> > > > > > > > > > > > you'd
> > > > > > > > > > > > > > > ideally build with JDK 9-or-later while having
> > the
> > > > > Java 8
> > > > > > > > runtime
> > > > > > > > > > > > > > > available.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > (TBH I don't think this is _too_ different from
> > other
> > > > > > > > projects
> > > > > > > > > > > where
> > > > > > > > > > > > > > > you have to have some compile-time dependencies
> > > > > installed
> > > > > > > to
> > > > > > > > be
> > > > > > > > > > > able
> > > > > > > > > > > > > > > to build the project)
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Arnout
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > > > > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Justin,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > There is an issue in the code base that is
> > > > > reasonably
> > > > > > > > common
> > > > > > > > > > > > among systems
> > > > > > > > > > > > > > > > > that operate at low levels and are migrating
> > from
> > > > > Java
> > > > > > > 8
> > > > > > > > > > > > implementation
> > > > > > > > > > > > > > > > > wherein they need the rt.jar from java 8 for
> > > > > > > compilation.
> > > > > > > > > > > >  There are
> > > > > > > > > > > > > > > > > several ways to solve this problem, Pekko
> > (and I
> > > > > > > suppose
> > > > > > > > > > Akka)
> > > > > > > > > > > > require that
> > > > > > > > > > > > > > > > > Java 8 be installed so that the build
> > process can
> > > > > find
> > > > > > > > the
> > > > > > > > > > Java
> > > > > > > > > > > > 8 rt.jar.
> > > > > > > > > > > > > > > > > I suspect that all that is actually required
> > to
> > > > > get the
> > > > > > > > > > current
> > > > > > > > > > > > code to
> > > > > > > > > > > > > > > > > compile is setting JAVA_8_HOME to point to
> > the
> > > > > > > directory
> > > > > > > > with
> > > > > > > > > > > > the rt.jar in
> > > > > > > > > > > > > > > > > it.  However, the Java 8 rt.jar is still
> > > > > required.  I
> > > > > > > > believe
> > > > > > > > > > > > there is a
> > > > > > > > > > > > > > > > > ticket to resolve this issue but I may be
> > mistaken
> > > > > > > about
> > > > > > > > > > this.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Claude
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin
> > Mclean <
> > > > > > > > > > > > justin@classsoftware.com>
> > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Part of the requirement of a release is
> > that
> > > > > someone
> > > > > > > > can
> > > > > > > > > > > build
> > > > > > > > > > > > it on their
> > > > > > > > > > > > > > > > > > system, and that ideally should be easy to
> > do. I
> > > > > > > would
> > > > > > > > > > assume
> > > > > > > > > > > > by their
> > > > > > > > > > > > > > > > > > system than means a single version of the
> > JDK
> > > > > which
> > > > > > > > they
> > > > > > > > > > use
> > > > > > > > > > > > and is
> > > > > > > > > > > > > > > > > > supported. I’m not sure there is be a need
> > for
> > > > > > > someone
> > > > > > > > > > > testing
> > > > > > > > > > > > the release
> > > > > > > > > > > > > > > > > > to compile against multiple JDKs?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Kind Regards,
> > > > > > > > > > > > > > > > > > Justin
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > > > > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > Arnout Engelen
> > > > > > > > > > > > > > > ASF Security Response
> > > > > > > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > > > > > > Committer on NixOS
> > > > > > > > > > > > > > > Independent Open Source consultant
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > > To unsubscribe, e-mail:
> > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > Arnout Engelen
> > > > > > > > > > > > > ASF Security Response
> > > > > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > > > > Committer on NixOS
> > > > > > > > > > > > > Independent Open Source consultant
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > > > > To unsubscribe, e-mail:
> > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > For additional commands, e-mail:
> > dev-help@pekko.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > > > To unsubscribe, e-mail:
> > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > For additional commands, e-mail:
> > dev-help@pekko.apache.org
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > >
> > > > > > > > > > > Matthew de Detrich
> > > > > > > > > > >
> > > > > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > > > > >
> > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > > > > >
> > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > > > > >
> > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > > > > >
> > > > > > > > > > > *m:* +491603708037
> > > > > > > > > > >
> > > > > > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > Matthew de Detrich
> > > > > > > > >
> > > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > > >
> > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > > >
> > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > > >
> > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > > >
> > > > > > > > > *m:* +491603708037
> > > > > > > > >
> > > > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > > >
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Matthew de Detrich
> > > > > > >
> > > > > > > *Aiven Deutschland GmbH*
> > > > > > >
> > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > >
> > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > >
> > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > >
> > > > > > > *m:* +491603708037
> > > > > > >
> > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> >
> >
> > --
> > Arnout Engelen
> > ASF Security Response
> > Committer on Apache Pekko
> > Committer on NixOS
> > Independent Open Source consultant
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
> >
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
> Have we seen anyone run into problems
(re)building from source?

The only issue I have found is if you try to build Pekko with even newer
JDK's, such as JDK 18. Claude was trying this out and he came across this
error

[error]
/home/claude/apache/incubator-pekko/actor/src/main/scala/org/apache/pekko/actor/LightArrayRevolverScheduler.scala:350:41:
method objectFieldOffset in class Unsafe is deprecated
[error]   private[this] val taskOffset =
unsafe.objectFieldOffset(classOf[TaskHolder].getDeclaredField("task"))
[error]                                         ^
[error] one error found

Which is probably due to JDK 18 deprecating even more methods from
sun.misc.unsafe. My immediate low hanging fruit for this is to detect if
sbt the JDK being used is 18 or newer and just exit out with a helpful
error (in the same way that it exits out right now if use JDK 11 without
detecting an JDK 8 rt.jar)

> Why not build 2 binary packages:  Java 8 and Java 11?  Use the java 8 JDK
to build the java 8 version and Java 11 JDK to build java 11 version.

I think that for all intents and purposes having multiple artifacts for
different JDK's is out of the question, both for Scala ecosystem reasons (I
can't think of any other OS Scala project that does this) or for user
experience (I don't see how you can correctly handle transitive
dependencies unless you want to add yet another axis to Scala's build
matrix).

Apparently there is a way to generate multi release jars (see
https://openjdk.org/jeps/238) however sbt doesn't support that yet although
it was raised for sbt 2.0.x (see
https://github.com/sbt/sbt/discussions/7174#discussioncomment-5288982).

This is however detracting a bit, as others said the most rational course
right now is to just document and help

On Thu, May 25, 2023 at 4:35 PM Arnout Engelen <en...@apache.org> wrote:

> On Thu, May 25, 2023 at 4:29 PM PJ Fanning <fa...@gmail.com> wrote:
> > Building from source is not as hard as some people are assuming it is.
> We can document it. We can
> > answer questions for people who are having issues.
>
> This is my impression as well. Have we seen anyone run into problems
> (re)building from source? I know we're planning further improvements
> but https://github.com/apache/incubator-pekko/#building-from-source
> already looks fairly good to me, I'd be interested to learn
> how/where/why people would get 'stuck'.
>
>
> Kind regards,
>
> Arnout
>
> > On Thu, 25 May 2023 at 15:18, Claude Warren, Jr
> > <cl...@aiven.io.invalid> wrote:
> > >
> > > Why not build 2 binary packages:  Java 8 and Java 11?  Use the java 8
> JDK
> > > to build the java 8 version and Java 11 JDK to build java 11 version.
> > >
> > > Then no need to copy rt.jar
> > >
> > > On Thu, May 25, 2023 at 2:35 PM Arnout Engelen <en...@apache.org>
> wrote:
> > >
> > > > On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr
> > > > <cl...@aiven.io.invalid> wrote:
> > > > > Assume Java 11 is installed and no Java 8 is installed on the
> system
> > > > where
> > > > > Pekko is being deployed.
> > > > > Assume Java 11 compilation with rt.jar present for the build of
> the Pekko
> > > > > package being deployed. (My understanding that even with Java 11
> the
> > > > rt.jar
> > > > > is required for compilation)
> > > > >
> > > > > When the code is executed on the deployed system and the class
> from the
> > > > > rt.jar is required, what happens?
> > > > > I would expect that a NoClassDefFoundError to be thrown.   If the
> error
> > > > is
> > > > > not thrown I can only see two possibilities:
> > > > >
> > > > >    1. the class(es) from the rt.jar are defined somewhere else in
> which
> > > > >    case why can that not be used during the build?
> > > > >    2. the class(es) are not used and the dependency on the rt.jar
> can be
> > > > >    removed.
> > > > >
> > > > > So what happens and why?
> > > >
> > > > No NoClassDefFoundError will be thrown, because the runtime for Java
> > > > 11 is backwards compatible with the runtime for Java 8, so there
> > > > should be no classes/methods that exist in the jdk8 rt.jar that are
> > > > not also available in the jdk11 runtime.
> > > >
> > > > From your two possibilities, '1' is true: they are defined both in
> the
> > > > Java 8 and in the Java 11 runtimes. Indeed if we removed the rt.jar
> > > > everything would still work on Java 11. Just no longer on Java 8.
> This
> > > > is because the runtimes are not *forwards* compatible: there are
> > > > things in the Java 11 runtime that are not in the Java 8 one.
> > > >
> > > > To illustrate and make this more tangible, one specific example is
> > > > ByteBuffer.clear(): on Java 8, this method returned a Buffer. On Java
> > > > 11, there is an implementation that returns a ByteBuffer. If you
> > > > compile a class that calls this method against the Java 8 runtime and
> > > > then run it against the Java 11 one, this is fine, because Java 11
> has
> > > > *both* a clear() method that returns a ByteBuffer and (for backwards
> > > > compatibility) one that returns a Buffer. If you compile the same
> code
> > > > against the Java 11 runtime and then run it against Java 8, it will
> > > > pick the one that returns a ByteBuffer at compile time, and you will
> > > > get a NoSuchMethod exception at runtime because this new method does
> > > > not exist in the Java 8 runtime.
> > > >
> > > > Of course we could change the code to make sure this *particular*
> > > > problem does not occur in Pekko, but that is risky as we don't know
> > > > how many similar situations there are.
> > > >
> > > > Is there a better solution to this than compiling with two JVM's
> > > > installed? Yes! Invoking the JDK11 compiler with '--release 8' will
> > > > also generate the correct reference. However, '--release 8' does not
> > > > allow access to sun.misc.Unsafe API's, which unfortunately Pekko
> still
> > > > relies on - so we can't do that just yet.
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Arnout
> > > >
> > > > > On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich
> > > > > <ma...@aiven.io.invalid> wrote:
> > > > >
> > > > > > > If you build Pekko with both Java 11 and Java 8 installed, the
> > > > classes
> > > > > > we will produce in our jars will be in Java 8 class format.
> > > > > >
> > > > > > More specifically, normally if you compile Java/Scala source
> with JDK
> > > > 11
> > > > > > (which is what Pekko does, JDK 8 is only necessary for its
> rt.jar) it
> > > > will
> > > > > > produce
> > > > > > classfiles compatible with JDK 11 and above. You can however
> provide a
> > > > flag
> > > > > > to
> > > > > > javac/scalac so that it generates class files with a specific
> bytecode
> > > > > > format, i.e.
> > > > > >
> > > > > >
> > > >
> https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60
> > > > > > and
> > > > > >
> > > > > >
> > > >
> https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73
> > > > > > does.
> > > > > >
> > > > > > In short, Pekko is built using JDK 11 with Xtarget/release flags
> to
> > > > force
> > > > > > JDK 8
> > > > > > compatible bytecode generation in addition to providing the JDK8
> > > > rt.jar via
> > > > > > the
> > > > > > -bootclasspath javac flag.
> > > > > >
> > > > > > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <fa...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > > Q: does the build package the rt.jar in the final binary
> package?
> > > > > > >
> > > > > > > No. We only produce our own jars that we push to Maven - and
> these do
> > > > > > > not embed any rt.jar classes (shaded or otherwise). We have no
> other
> > > > > > > type of 'binary' package.
> > > > > > >
> > > > > > > > Q: if it does not, what happens if you try to run using Java
> 11 on
> > > > a
> > > > > > > system
> > > > > > > > that does not have Java 8 nor the rt.jar installed?
> > > > > > >
> > > > > > > Java compiler produces class files. These have a Java version
> > > > > > > associated with them and you can't use a class that is in Java
> 11
> > > > > > > format in a Java 8 runtime.
> > > > > > > If you build Pekko with both Java 11 and Java 8 installed, the
> > > > classes
> > > > > > > we will produce in our jars will be in Java 8 class format.
> > > > > > >
> > > > > > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich
> > > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > > >
> > > > > > > > afaik Pekko will just fail to build at all if you use JDK 11
> > > > without
> > > > > > > > pointing to JDK 8 rt.jar.
> > > > > > > >
> > > > > > > > This is because Pekko uses sun.misc.unsafe which is only
> shipping
> > > > with
> > > > > > > JDK
> > > > > > > > 8 and the
> > > > > > > > reason why we have to build on JDK 11 is because there are
> features
> > > > > > > > shipping in JDK 11
> > > > > > > > which pekko uses.
> > > > > > > >
> > > > > > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
> > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > >
> > > > > > > > > I have a couple of questions:
> > > > > > > > >
> > > > > > > > > Q: does the build package the rt.jar in the final binary
> package?
> > > > > > > > >
> > > > > > > > > Q: if it does not, what happens if you try to run using
> Java 11
> > > > on a
> > > > > > > system
> > > > > > > > > that does not have Java 8 nor the rt.jar installed?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> > > > > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > > > >
> > > > > > > > > > > We could provide a Docker image with the right sbt and
> Java
> > > > setup
> > > > > > > for
> > > > > > > > > > convenience but I think it is fair not to require users
> to use
> > > > > > > Docker.
> > > > > > > > > >
> > > > > > > > > > Considering how much implicit behaviour there is in
> making a
> > > > full
> > > > > > > build
> > > > > > > > > > (as Arnout mentioned earlier, you can compile Pekko fine
> with
> > > > just
> > > > > > > JDK 8
> > > > > > > > > > just fine however it will be missing post java 8 optional
> > > > > > features) I
> > > > > > > > > would
> > > > > > > > > > prefer
> > > > > > > > > > to strongly encourage people to use Docker to do a proper
> > > > build.
> > > > > > As a
> > > > > > > > > bare
> > > > > > > > > > minimum we can also print a warning in sbt telling users
> if
> > > > they
> > > > > > are
> > > > > > > not
> > > > > > > > > > going
> > > > > > > > > > to be making a complete Pekko build when building on JDK
> 8 (as
> > > > > > > stated it
> > > > > > > > > > already errors out if you build on JDK 11 without
> pointing to
> > > > a JDK
> > > > > > > > > > rt.jar).
> > > > > > > > > >
> > > > > > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <
> > > > fanningpj@gmail.com>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Thanks Arnout.
> > > > > > > > > > >
> > > > > > > > > > > I think if we clearly document what you need to set up
> to do
> > > > a
> > > > > > > build,
> > > > > > > > > > > that should be enough. I think it is fair to assume
> that
> > > > users
> > > > > > can
> > > > > > > > > > > install Java and sbt - and that they should be able to
> > > > install 2
> > > > > > > > > > > versions of Java if we are explicit about how to do
> this and
> > > > why
> > > > > > > this
> > > > > > > > > > > is the best setup for building Pekko.
> > > > > > > > > > >
> > > > > > > > > > > We will need to expand what we have in
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > >
> https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > > > > > > > > > >
> > > > > > > > > > > This README file will appear in the source releases.
> > > > > > > > > > >
> > > > > > > > > > > We could provide a Docker image with the right sbt and
> Java
> > > > setup
> > > > > > > for
> > > > > > > > > > > convenience but I think it is fair not to require
> users to
> > > > use
> > > > > > > Docker.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <
> > > > engelen@apache.org
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <
> > > > > > fanningpj@gmail.com
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > > > Ok. It seems the nightly job to publish the
> snapshot jars
> > > > > > uses
> > > > > > > Java
> > > > > > > > > > > > > 11. It does not explicitly install Java 8 but
> maybe, it's
> > > > > > > possible
> > > > > > > > > > > > > that Java 8 is installed silently,
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > >
> https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > > > > > > > > > suggests so, yes.
> > > > > > > > > > > >
> > > > > > > > > > > > > The nightly build does produce jars that can be
> used by
> > > > Java
> > > > > > 8.
> > > > > > > > > > > > >
> > > > > > > > > > > > > So if this nightly build relies on the silent
> > > > installation of
> > > > > > > Java
> > > > > > > > > 8,
> > > > > > > > > > > > > should we add a check in our sbt script that will
> fail if
> > > > > > Java
> > > > > > > 8 is
> > > > > > > > > > > > > not found? This could be useful for users who are
> > > > building on
> > > > > > > their
> > > > > > > > > > > > > own machines.
> > > > > > > > > > > >
> > > > > > > > > > > > This check already exists: when I build with jdk11
> without
> > > > > > having
> > > > > > > > > jdk8
> > > > > > > > > > > > available the build fails with:
> > > > > > > > > > > >
> > > > > > > > > > > > [error] A JDK 8 installation was not found, but is
> > > > required to
> > > > > > > build
> > > > > > > > > > > > Apache Pekko. To manually specify a JDK 8
> installation,
> > > > use the
> > > > > > > "set
> > > > > > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If
> you
> > > > have no
> > > > > > > JDK 8
> > > > > > > > > > > > installation, target your system JDK with the "set
> every
> > > > > > > > > > > > targetSystemJdk := true" sbt command, but beware
> resulting
> > > > > > > artifacts
> > > > > > > > > > > > will not work on JDK 8
> > > > > > > > > > > >
> > > > > > > > > > > > It would be good to also document the JAVA_8_HOME
> > > > environment
> > > > > > > > > variable
> > > > > > > > > > > > in this error message. Created
> > > > > > > > > > > > https://github.com/apache/incubator-pekko/pull/333
> for
> > > > that.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Kind regards,
> > > > > > > > > > > >
> > > > > > > > > > > > Arnout
> > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <
> > > > > > > engelen@apache.org>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <
> > > > > > > fanningpj@gmail.com
> > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > Only building the website docs needs Java 11
> (and
> > > > up).
> > > > > > > > > Everything
> > > > > > > > > > > else
> > > > > > > > > > > > > > > builds with Java 8 (and up).
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Yes, though builds created with only Java 8 will
> miss
> > > > some
> > > > > > > > > optional
> > > > > > > > > > > > > > post-Java-8 features/implementations for the
> > > > "actor-typed",
> > > > > > > > > > "stream",
> > > > > > > > > > > > > > "remote" and "cluster-sharding" modules - so for
> a
> > > > faithful
> > > > > > > test
> > > > > > > > > > > you'd
> > > > > > > > > > > > > > ideally build with JDK 9-or-later while having
> the
> > > > Java 8
> > > > > > > runtime
> > > > > > > > > > > > > > available.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > (TBH I don't think this is _too_ different from
> other
> > > > > > > projects
> > > > > > > > > > where
> > > > > > > > > > > > > > you have to have some compile-time dependencies
> > > > installed
> > > > > > to
> > > > > > > be
> > > > > > > > > > able
> > > > > > > > > > > > > > to build the project)
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Arnout
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > > > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Justin,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > There is an issue in the code base that is
> > > > reasonably
> > > > > > > common
> > > > > > > > > > > among systems
> > > > > > > > > > > > > > > > that operate at low levels and are migrating
> from
> > > > Java
> > > > > > 8
> > > > > > > > > > > implementation
> > > > > > > > > > > > > > > > wherein they need the rt.jar from java 8 for
> > > > > > compilation.
> > > > > > > > > > >  There are
> > > > > > > > > > > > > > > > several ways to solve this problem, Pekko
> (and I
> > > > > > suppose
> > > > > > > > > Akka)
> > > > > > > > > > > require that
> > > > > > > > > > > > > > > > Java 8 be installed so that the build
> process can
> > > > find
> > > > > > > the
> > > > > > > > > Java
> > > > > > > > > > > 8 rt.jar.
> > > > > > > > > > > > > > > > I suspect that all that is actually required
> to
> > > > get the
> > > > > > > > > current
> > > > > > > > > > > code to
> > > > > > > > > > > > > > > > compile is setting JAVA_8_HOME to point to
> the
> > > > > > directory
> > > > > > > with
> > > > > > > > > > > the rt.jar in
> > > > > > > > > > > > > > > > it.  However, the Java 8 rt.jar is still
> > > > required.  I
> > > > > > > believe
> > > > > > > > > > > there is a
> > > > > > > > > > > > > > > > ticket to resolve this issue but I may be
> mistaken
> > > > > > about
> > > > > > > > > this.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Claude
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin
> Mclean <
> > > > > > > > > > > justin@classsoftware.com>
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Part of the requirement of a release is
> that
> > > > someone
> > > > > > > can
> > > > > > > > > > build
> > > > > > > > > > > it on their
> > > > > > > > > > > > > > > > > system, and that ideally should be easy to
> do. I
> > > > > > would
> > > > > > > > > assume
> > > > > > > > > > > by their
> > > > > > > > > > > > > > > > > system than means a single version of the
> JDK
> > > > which
> > > > > > > they
> > > > > > > > > use
> > > > > > > > > > > and is
> > > > > > > > > > > > > > > > > supported. I’m not sure there is be a need
> for
> > > > > > someone
> > > > > > > > > > testing
> > > > > > > > > > > the release
> > > > > > > > > > > > > > > > > to compile against multiple JDKs?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Kind Regards,
> > > > > > > > > > > > > > > > > Justin
> > > > > > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > > > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > Arnout Engelen
> > > > > > > > > > > > > > ASF Security Response
> > > > > > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > > > > > Committer on NixOS
> > > > > > > > > > > > > > Independent Open Source consultant
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > For additional commands, e-mail:
> > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > For additional commands, e-mail:
> > > > dev-help@pekko.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Arnout Engelen
> > > > > > > > > > > > ASF Security Response
> > > > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > > > Committer on NixOS
> > > > > > > > > > > > Independent Open Source consultant
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > For additional commands, e-mail:
> dev-help@pekko.apache.org
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> dev-help@pekko.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > >
> > > > > > > > > > Matthew de Detrich
> > > > > > > > > >
> > > > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > > > >
> > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > > > >
> > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > > > >
> > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > > > >
> > > > > > > > > > *m:* +491603708037
> > > > > > > > > >
> > > > > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Matthew de Detrich
> > > > > > > >
> > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > >
> > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > >
> > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > >
> > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > >
> > > > > > > > *m:* +491603708037
> > > > > > > >
> > > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > >
> > > > > > >
> ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Matthew de Detrich
> > > > > >
> > > > > > *Aiven Deutschland GmbH*
> > > > > >
> > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > >
> > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > >
> > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > >
> > > > > > *m:* +491603708037
> > > > > >
> > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
>
>
> --
> Arnout Engelen
> ASF Security Response
> Committer on Apache Pekko
> Committer on NixOS
> Independent Open Source consultant
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>
>

-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] Building Binary Package

Posted by Arnout Engelen <en...@apache.org>.
On Thu, May 25, 2023 at 4:29 PM PJ Fanning <fa...@gmail.com> wrote:
> Building from source is not as hard as some people are assuming it is. We can document it. We can
> answer questions for people who are having issues.

This is my impression as well. Have we seen anyone run into problems
(re)building from source? I know we're planning further improvements
but https://github.com/apache/incubator-pekko/#building-from-source
already looks fairly good to me, I'd be interested to learn
how/where/why people would get 'stuck'.


Kind regards,

Arnout

> On Thu, 25 May 2023 at 15:18, Claude Warren, Jr
> <cl...@aiven.io.invalid> wrote:
> >
> > Why not build 2 binary packages:  Java 8 and Java 11?  Use the java 8 JDK
> > to build the java 8 version and Java 11 JDK to build java 11 version.
> >
> > Then no need to copy rt.jar
> >
> > On Thu, May 25, 2023 at 2:35 PM Arnout Engelen <en...@apache.org> wrote:
> >
> > > On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr
> > > <cl...@aiven.io.invalid> wrote:
> > > > Assume Java 11 is installed and no Java 8 is installed on the system
> > > where
> > > > Pekko is being deployed.
> > > > Assume Java 11 compilation with rt.jar present for the build of the Pekko
> > > > package being deployed. (My understanding that even with Java 11 the
> > > rt.jar
> > > > is required for compilation)
> > > >
> > > > When the code is executed on the deployed system and the class from the
> > > > rt.jar is required, what happens?
> > > > I would expect that a NoClassDefFoundError to be thrown.   If the error
> > > is
> > > > not thrown I can only see two possibilities:
> > > >
> > > >    1. the class(es) from the rt.jar are defined somewhere else in which
> > > >    case why can that not be used during the build?
> > > >    2. the class(es) are not used and the dependency on the rt.jar can be
> > > >    removed.
> > > >
> > > > So what happens and why?
> > >
> > > No NoClassDefFoundError will be thrown, because the runtime for Java
> > > 11 is backwards compatible with the runtime for Java 8, so there
> > > should be no classes/methods that exist in the jdk8 rt.jar that are
> > > not also available in the jdk11 runtime.
> > >
> > > From your two possibilities, '1' is true: they are defined both in the
> > > Java 8 and in the Java 11 runtimes. Indeed if we removed the rt.jar
> > > everything would still work on Java 11. Just no longer on Java 8. This
> > > is because the runtimes are not *forwards* compatible: there are
> > > things in the Java 11 runtime that are not in the Java 8 one.
> > >
> > > To illustrate and make this more tangible, one specific example is
> > > ByteBuffer.clear(): on Java 8, this method returned a Buffer. On Java
> > > 11, there is an implementation that returns a ByteBuffer. If you
> > > compile a class that calls this method against the Java 8 runtime and
> > > then run it against the Java 11 one, this is fine, because Java 11 has
> > > *both* a clear() method that returns a ByteBuffer and (for backwards
> > > compatibility) one that returns a Buffer. If you compile the same code
> > > against the Java 11 runtime and then run it against Java 8, it will
> > > pick the one that returns a ByteBuffer at compile time, and you will
> > > get a NoSuchMethod exception at runtime because this new method does
> > > not exist in the Java 8 runtime.
> > >
> > > Of course we could change the code to make sure this *particular*
> > > problem does not occur in Pekko, but that is risky as we don't know
> > > how many similar situations there are.
> > >
> > > Is there a better solution to this than compiling with two JVM's
> > > installed? Yes! Invoking the JDK11 compiler with '--release 8' will
> > > also generate the correct reference. However, '--release 8' does not
> > > allow access to sun.misc.Unsafe API's, which unfortunately Pekko still
> > > relies on - so we can't do that just yet.
> > >
> > >
> > > Kind regards,
> > >
> > > Arnout
> > >
> > > > On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich
> > > > <ma...@aiven.io.invalid> wrote:
> > > >
> > > > > > If you build Pekko with both Java 11 and Java 8 installed, the
> > > classes
> > > > > we will produce in our jars will be in Java 8 class format.
> > > > >
> > > > > More specifically, normally if you compile Java/Scala source with JDK
> > > 11
> > > > > (which is what Pekko does, JDK 8 is only necessary for its rt.jar) it
> > > will
> > > > > produce
> > > > > classfiles compatible with JDK 11 and above. You can however provide a
> > > flag
> > > > > to
> > > > > javac/scalac so that it generates class files with a specific bytecode
> > > > > format, i.e.
> > > > >
> > > > >
> > > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60
> > > > > and
> > > > >
> > > > >
> > > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73
> > > > > does.
> > > > >
> > > > > In short, Pekko is built using JDK 11 with Xtarget/release flags to
> > > force
> > > > > JDK 8
> > > > > compatible bytecode generation in addition to providing the JDK8
> > > rt.jar via
> > > > > the
> > > > > -bootclasspath javac flag.
> > > > >
> > > > > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <fa...@gmail.com>
> > > wrote:
> > > > >
> > > > > > > Q: does the build package the rt.jar in the final binary package?
> > > > > >
> > > > > > No. We only produce our own jars that we push to Maven - and these do
> > > > > > not embed any rt.jar classes (shaded or otherwise). We have no other
> > > > > > type of 'binary' package.
> > > > > >
> > > > > > > Q: if it does not, what happens if you try to run using Java 11 on
> > > a
> > > > > > system
> > > > > > > that does not have Java 8 nor the rt.jar installed?
> > > > > >
> > > > > > Java compiler produces class files. These have a Java version
> > > > > > associated with them and you can't use a class that is in Java 11
> > > > > > format in a Java 8 runtime.
> > > > > > If you build Pekko with both Java 11 and Java 8 installed, the
> > > classes
> > > > > > we will produce in our jars will be in Java 8 class format.
> > > > > >
> > > > > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich
> > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > >
> > > > > > > afaik Pekko will just fail to build at all if you use JDK 11
> > > without
> > > > > > > pointing to JDK 8 rt.jar.
> > > > > > >
> > > > > > > This is because Pekko uses sun.misc.unsafe which is only shipping
> > > with
> > > > > > JDK
> > > > > > > 8 and the
> > > > > > > reason why we have to build on JDK 11 is because there are features
> > > > > > > shipping in JDK 11
> > > > > > > which pekko uses.
> > > > > > >
> > > > > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
> > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > >
> > > > > > > > I have a couple of questions:
> > > > > > > >
> > > > > > > > Q: does the build package the rt.jar in the final binary package?
> > > > > > > >
> > > > > > > > Q: if it does not, what happens if you try to run using Java 11
> > > on a
> > > > > > system
> > > > > > > > that does not have Java 8 nor the rt.jar installed?
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> > > > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > > >
> > > > > > > > > > We could provide a Docker image with the right sbt and Java
> > > setup
> > > > > > for
> > > > > > > > > convenience but I think it is fair not to require users to use
> > > > > > Docker.
> > > > > > > > >
> > > > > > > > > Considering how much implicit behaviour there is in making a
> > > full
> > > > > > build
> > > > > > > > > (as Arnout mentioned earlier, you can compile Pekko fine with
> > > just
> > > > > > JDK 8
> > > > > > > > > just fine however it will be missing post java 8 optional
> > > > > features) I
> > > > > > > > would
> > > > > > > > > prefer
> > > > > > > > > to strongly encourage people to use Docker to do a proper
> > > build.
> > > > > As a
> > > > > > > > bare
> > > > > > > > > minimum we can also print a warning in sbt telling users if
> > > they
> > > > > are
> > > > > > not
> > > > > > > > > going
> > > > > > > > > to be making a complete Pekko build when building on JDK 8 (as
> > > > > > stated it
> > > > > > > > > already errors out if you build on JDK 11 without pointing to
> > > a JDK
> > > > > > > > > rt.jar).
> > > > > > > > >
> > > > > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <
> > > fanningpj@gmail.com>
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Thanks Arnout.
> > > > > > > > > >
> > > > > > > > > > I think if we clearly document what you need to set up to do
> > > a
> > > > > > build,
> > > > > > > > > > that should be enough. I think it is fair to assume that
> > > users
> > > > > can
> > > > > > > > > > install Java and sbt - and that they should be able to
> > > install 2
> > > > > > > > > > versions of Java if we are explicit about how to do this and
> > > why
> > > > > > this
> > > > > > > > > > is the best setup for building Pekko.
> > > > > > > > > >
> > > > > > > > > > We will need to expand what we have in
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > > > > > > > > >
> > > > > > > > > > This README file will appear in the source releases.
> > > > > > > > > >
> > > > > > > > > > We could provide a Docker image with the right sbt and Java
> > > setup
> > > > > > for
> > > > > > > > > > convenience but I think it is fair not to require users to
> > > use
> > > > > > Docker.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <
> > > engelen@apache.org
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <
> > > > > fanningpj@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > > > Ok. It seems the nightly job to publish the snapshot jars
> > > > > uses
> > > > > > Java
> > > > > > > > > > > > 11. It does not explicitly install Java 8 but maybe, it's
> > > > > > possible
> > > > > > > > > > > > that Java 8 is installed silently,
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > > > > > > > > suggests so, yes.
> > > > > > > > > > >
> > > > > > > > > > > > The nightly build does produce jars that can be used by
> > > Java
> > > > > 8.
> > > > > > > > > > > >
> > > > > > > > > > > > So if this nightly build relies on the silent
> > > installation of
> > > > > > Java
> > > > > > > > 8,
> > > > > > > > > > > > should we add a check in our sbt script that will fail if
> > > > > Java
> > > > > > 8 is
> > > > > > > > > > > > not found? This could be useful for users who are
> > > building on
> > > > > > their
> > > > > > > > > > > > own machines.
> > > > > > > > > > >
> > > > > > > > > > > This check already exists: when I build with jdk11 without
> > > > > having
> > > > > > > > jdk8
> > > > > > > > > > > available the build fails with:
> > > > > > > > > > >
> > > > > > > > > > > [error] A JDK 8 installation was not found, but is
> > > required to
> > > > > > build
> > > > > > > > > > > Apache Pekko. To manually specify a JDK 8 installation,
> > > use the
> > > > > > "set
> > > > > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you
> > > have no
> > > > > > JDK 8
> > > > > > > > > > > installation, target your system JDK with the "set every
> > > > > > > > > > > targetSystemJdk := true" sbt command, but beware resulting
> > > > > > artifacts
> > > > > > > > > > > will not work on JDK 8
> > > > > > > > > > >
> > > > > > > > > > > It would be good to also document the JAVA_8_HOME
> > > environment
> > > > > > > > variable
> > > > > > > > > > > in this error message. Created
> > > > > > > > > > > https://github.com/apache/incubator-pekko/pull/333 for
> > > that.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Kind regards,
> > > > > > > > > > >
> > > > > > > > > > > Arnout
> > > > > > > > > > >
> > > > > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <
> > > > > > engelen@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <
> > > > > > fanningpj@gmail.com
> > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > > > > Only building the website docs needs Java 11 (and
> > > up).
> > > > > > > > Everything
> > > > > > > > > > else
> > > > > > > > > > > > > > builds with Java 8 (and up).
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yes, though builds created with only Java 8 will miss
> > > some
> > > > > > > > optional
> > > > > > > > > > > > > post-Java-8 features/implementations for the
> > > "actor-typed",
> > > > > > > > > "stream",
> > > > > > > > > > > > > "remote" and "cluster-sharding" modules - so for a
> > > faithful
> > > > > > test
> > > > > > > > > > you'd
> > > > > > > > > > > > > ideally build with JDK 9-or-later while having the
> > > Java 8
> > > > > > runtime
> > > > > > > > > > > > > available.
> > > > > > > > > > > > >
> > > > > > > > > > > > > (TBH I don't think this is _too_ different from other
> > > > > > projects
> > > > > > > > > where
> > > > > > > > > > > > > you have to have some compile-time dependencies
> > > installed
> > > > > to
> > > > > > be
> > > > > > > > > able
> > > > > > > > > > > > > to build the project)
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Arnout
> > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Justin,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > There is an issue in the code base that is
> > > reasonably
> > > > > > common
> > > > > > > > > > among systems
> > > > > > > > > > > > > > > that operate at low levels and are migrating from
> > > Java
> > > > > 8
> > > > > > > > > > implementation
> > > > > > > > > > > > > > > wherein they need the rt.jar from java 8 for
> > > > > compilation.
> > > > > > > > > >  There are
> > > > > > > > > > > > > > > several ways to solve this problem, Pekko (and I
> > > > > suppose
> > > > > > > > Akka)
> > > > > > > > > > require that
> > > > > > > > > > > > > > > Java 8 be installed so that the build process can
> > > find
> > > > > > the
> > > > > > > > Java
> > > > > > > > > > 8 rt.jar.
> > > > > > > > > > > > > > > I suspect that all that is actually required to
> > > get the
> > > > > > > > current
> > > > > > > > > > code to
> > > > > > > > > > > > > > > compile is setting JAVA_8_HOME to point to the
> > > > > directory
> > > > > > with
> > > > > > > > > > the rt.jar in
> > > > > > > > > > > > > > > it.  However, the Java 8 rt.jar is still
> > > required.  I
> > > > > > believe
> > > > > > > > > > there is a
> > > > > > > > > > > > > > > ticket to resolve this issue but I may be mistaken
> > > > > about
> > > > > > > > this.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Claude
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> > > > > > > > > > justin@classsoftware.com>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Part of the requirement of a release is that
> > > someone
> > > > > > can
> > > > > > > > > build
> > > > > > > > > > it on their
> > > > > > > > > > > > > > > > system, and that ideally should be easy to do. I
> > > > > would
> > > > > > > > assume
> > > > > > > > > > by their
> > > > > > > > > > > > > > > > system than means a single version of the JDK
> > > which
> > > > > > they
> > > > > > > > use
> > > > > > > > > > and is
> > > > > > > > > > > > > > > > supported. I’m not sure there is be a need for
> > > > > someone
> > > > > > > > > testing
> > > > > > > > > > the release
> > > > > > > > > > > > > > > > to compile against multiple JDKs?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Kind Regards,
> > > > > > > > > > > > > > > > Justin
> > > > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > Arnout Engelen
> > > > > > > > > > > > > ASF Security Response
> > > > > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > > > > Committer on NixOS
> > > > > > > > > > > > > Independent Open Source consultant
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > >
> > > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > To unsubscribe, e-mail:
> > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > For additional commands, e-mail:
> > > dev-help@pekko.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > For additional commands, e-mail:
> > > dev-help@pekko.apache.org
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Arnout Engelen
> > > > > > > > > > > ASF Security Response
> > > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > > Committer on NixOS
> > > > > > > > > > > Independent Open Source consultant
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > > Matthew de Detrich
> > > > > > > > >
> > > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > > >
> > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > > >
> > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > > >
> > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > > >
> > > > > > > > > *m:* +491603708037
> > > > > > > > >
> > > > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Matthew de Detrich
> > > > > > >
> > > > > > > *Aiven Deutschland GmbH*
> > > > > > >
> > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > >
> > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > >
> > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > >
> > > > > > > *m:* +491603708037
> > > > > > >
> > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Matthew de Detrich
> > > > >
> > > > > *Aiven Deutschland GmbH*
> > > > >
> > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > >
> > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > >
> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > >
> > > > > *m:* +491603708037
> > > > >
> > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>


-- 
Arnout Engelen
ASF Security Response
Committer on Apache Pekko
Committer on NixOS
Independent Open Source consultant

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by PJ Fanning <fa...@gmail.com>.
Can we just document this? In the end of the day, we produce jars that
can be used in Java 8 and up.

We intend to keep this for the 1.0.0 release.

We might consider dropping Java 8 support at some stage.

I work on a number of Open Source projects and many of them have
similar requirements to use multiple JDKs to build the jars.

The vast majority of users will use the jars that we publish to Maven Central.

A small minority will build from source. Building from source is not
as hard as some people are assuming it is. We can document it. We can
answer questions for people who are having issues.


On Thu, 25 May 2023 at 15:18, Claude Warren, Jr
<cl...@aiven.io.invalid> wrote:
>
> Why not build 2 binary packages:  Java 8 and Java 11?  Use the java 8 JDK
> to build the java 8 version and Java 11 JDK to build java 11 version.
>
> Then no need to copy rt.jar
>
> On Thu, May 25, 2023 at 2:35 PM Arnout Engelen <en...@apache.org> wrote:
>
> > On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr
> > <cl...@aiven.io.invalid> wrote:
> > > Assume Java 11 is installed and no Java 8 is installed on the system
> > where
> > > Pekko is being deployed.
> > > Assume Java 11 compilation with rt.jar present for the build of the Pekko
> > > package being deployed. (My understanding that even with Java 11 the
> > rt.jar
> > > is required for compilation)
> > >
> > > When the code is executed on the deployed system and the class from the
> > > rt.jar is required, what happens?
> > > I would expect that a NoClassDefFoundError to be thrown.   If the error
> > is
> > > not thrown I can only see two possibilities:
> > >
> > >    1. the class(es) from the rt.jar are defined somewhere else in which
> > >    case why can that not be used during the build?
> > >    2. the class(es) are not used and the dependency on the rt.jar can be
> > >    removed.
> > >
> > > So what happens and why?
> >
> > No NoClassDefFoundError will be thrown, because the runtime for Java
> > 11 is backwards compatible with the runtime for Java 8, so there
> > should be no classes/methods that exist in the jdk8 rt.jar that are
> > not also available in the jdk11 runtime.
> >
> > From your two possibilities, '1' is true: they are defined both in the
> > Java 8 and in the Java 11 runtimes. Indeed if we removed the rt.jar
> > everything would still work on Java 11. Just no longer on Java 8. This
> > is because the runtimes are not *forwards* compatible: there are
> > things in the Java 11 runtime that are not in the Java 8 one.
> >
> > To illustrate and make this more tangible, one specific example is
> > ByteBuffer.clear(): on Java 8, this method returned a Buffer. On Java
> > 11, there is an implementation that returns a ByteBuffer. If you
> > compile a class that calls this method against the Java 8 runtime and
> > then run it against the Java 11 one, this is fine, because Java 11 has
> > *both* a clear() method that returns a ByteBuffer and (for backwards
> > compatibility) one that returns a Buffer. If you compile the same code
> > against the Java 11 runtime and then run it against Java 8, it will
> > pick the one that returns a ByteBuffer at compile time, and you will
> > get a NoSuchMethod exception at runtime because this new method does
> > not exist in the Java 8 runtime.
> >
> > Of course we could change the code to make sure this *particular*
> > problem does not occur in Pekko, but that is risky as we don't know
> > how many similar situations there are.
> >
> > Is there a better solution to this than compiling with two JVM's
> > installed? Yes! Invoking the JDK11 compiler with '--release 8' will
> > also generate the correct reference. However, '--release 8' does not
> > allow access to sun.misc.Unsafe API's, which unfortunately Pekko still
> > relies on - so we can't do that just yet.
> >
> >
> > Kind regards,
> >
> > Arnout
> >
> > > On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich
> > > <ma...@aiven.io.invalid> wrote:
> > >
> > > > > If you build Pekko with both Java 11 and Java 8 installed, the
> > classes
> > > > we will produce in our jars will be in Java 8 class format.
> > > >
> > > > More specifically, normally if you compile Java/Scala source with JDK
> > 11
> > > > (which is what Pekko does, JDK 8 is only necessary for its rt.jar) it
> > will
> > > > produce
> > > > classfiles compatible with JDK 11 and above. You can however provide a
> > flag
> > > > to
> > > > javac/scalac so that it generates class files with a specific bytecode
> > > > format, i.e.
> > > >
> > > >
> > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60
> > > > and
> > > >
> > > >
> > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73
> > > > does.
> > > >
> > > > In short, Pekko is built using JDK 11 with Xtarget/release flags to
> > force
> > > > JDK 8
> > > > compatible bytecode generation in addition to providing the JDK8
> > rt.jar via
> > > > the
> > > > -bootclasspath javac flag.
> > > >
> > > > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <fa...@gmail.com>
> > wrote:
> > > >
> > > > > > Q: does the build package the rt.jar in the final binary package?
> > > > >
> > > > > No. We only produce our own jars that we push to Maven - and these do
> > > > > not embed any rt.jar classes (shaded or otherwise). We have no other
> > > > > type of 'binary' package.
> > > > >
> > > > > > Q: if it does not, what happens if you try to run using Java 11 on
> > a
> > > > > system
> > > > > > that does not have Java 8 nor the rt.jar installed?
> > > > >
> > > > > Java compiler produces class files. These have a Java version
> > > > > associated with them and you can't use a class that is in Java 11
> > > > > format in a Java 8 runtime.
> > > > > If you build Pekko with both Java 11 and Java 8 installed, the
> > classes
> > > > > we will produce in our jars will be in Java 8 class format.
> > > > >
> > > > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich
> > > > > <ma...@aiven.io.invalid> wrote:
> > > > > >
> > > > > > afaik Pekko will just fail to build at all if you use JDK 11
> > without
> > > > > > pointing to JDK 8 rt.jar.
> > > > > >
> > > > > > This is because Pekko uses sun.misc.unsafe which is only shipping
> > with
> > > > > JDK
> > > > > > 8 and the
> > > > > > reason why we have to build on JDK 11 is because there are features
> > > > > > shipping in JDK 11
> > > > > > which pekko uses.
> > > > > >
> > > > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
> > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > >
> > > > > > > I have a couple of questions:
> > > > > > >
> > > > > > > Q: does the build package the rt.jar in the final binary package?
> > > > > > >
> > > > > > > Q: if it does not, what happens if you try to run using Java 11
> > on a
> > > > > system
> > > > > > > that does not have Java 8 nor the rt.jar installed?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> > > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > > >
> > > > > > > > > We could provide a Docker image with the right sbt and Java
> > setup
> > > > > for
> > > > > > > > convenience but I think it is fair not to require users to use
> > > > > Docker.
> > > > > > > >
> > > > > > > > Considering how much implicit behaviour there is in making a
> > full
> > > > > build
> > > > > > > > (as Arnout mentioned earlier, you can compile Pekko fine with
> > just
> > > > > JDK 8
> > > > > > > > just fine however it will be missing post java 8 optional
> > > > features) I
> > > > > > > would
> > > > > > > > prefer
> > > > > > > > to strongly encourage people to use Docker to do a proper
> > build.
> > > > As a
> > > > > > > bare
> > > > > > > > minimum we can also print a warning in sbt telling users if
> > they
> > > > are
> > > > > not
> > > > > > > > going
> > > > > > > > to be making a complete Pekko build when building on JDK 8 (as
> > > > > stated it
> > > > > > > > already errors out if you build on JDK 11 without pointing to
> > a JDK
> > > > > > > > rt.jar).
> > > > > > > >
> > > > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <
> > fanningpj@gmail.com>
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Thanks Arnout.
> > > > > > > > >
> > > > > > > > > I think if we clearly document what you need to set up to do
> > a
> > > > > build,
> > > > > > > > > that should be enough. I think it is fair to assume that
> > users
> > > > can
> > > > > > > > > install Java and sbt - and that they should be able to
> > install 2
> > > > > > > > > versions of Java if we are explicit about how to do this and
> > why
> > > > > this
> > > > > > > > > is the best setup for building Pekko.
> > > > > > > > >
> > > > > > > > > We will need to expand what we have in
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > > > > > > > >
> > > > > > > > > This README file will appear in the source releases.
> > > > > > > > >
> > > > > > > > > We could provide a Docker image with the right sbt and Java
> > setup
> > > > > for
> > > > > > > > > convenience but I think it is fair not to require users to
> > use
> > > > > Docker.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <
> > engelen@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <
> > > > fanningpj@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > > > Ok. It seems the nightly job to publish the snapshot jars
> > > > uses
> > > > > Java
> > > > > > > > > > > 11. It does not explicitly install Java 8 but maybe, it's
> > > > > possible
> > > > > > > > > > > that Java 8 is installed silently,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > > > > > > > suggests so, yes.
> > > > > > > > > >
> > > > > > > > > > > The nightly build does produce jars that can be used by
> > Java
> > > > 8.
> > > > > > > > > > >
> > > > > > > > > > > So if this nightly build relies on the silent
> > installation of
> > > > > Java
> > > > > > > 8,
> > > > > > > > > > > should we add a check in our sbt script that will fail if
> > > > Java
> > > > > 8 is
> > > > > > > > > > > not found? This could be useful for users who are
> > building on
> > > > > their
> > > > > > > > > > > own machines.
> > > > > > > > > >
> > > > > > > > > > This check already exists: when I build with jdk11 without
> > > > having
> > > > > > > jdk8
> > > > > > > > > > available the build fails with:
> > > > > > > > > >
> > > > > > > > > > [error] A JDK 8 installation was not found, but is
> > required to
> > > > > build
> > > > > > > > > > Apache Pekko. To manually specify a JDK 8 installation,
> > use the
> > > > > "set
> > > > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you
> > have no
> > > > > JDK 8
> > > > > > > > > > installation, target your system JDK with the "set every
> > > > > > > > > > targetSystemJdk := true" sbt command, but beware resulting
> > > > > artifacts
> > > > > > > > > > will not work on JDK 8
> > > > > > > > > >
> > > > > > > > > > It would be good to also document the JAVA_8_HOME
> > environment
> > > > > > > variable
> > > > > > > > > > in this error message. Created
> > > > > > > > > > https://github.com/apache/incubator-pekko/pull/333 for
> > that.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Kind regards,
> > > > > > > > > >
> > > > > > > > > > Arnout
> > > > > > > > > >
> > > > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <
> > > > > engelen@apache.org>
> > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <
> > > > > fanningpj@gmail.com
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > > > > Only building the website docs needs Java 11 (and
> > up).
> > > > > > > Everything
> > > > > > > > > else
> > > > > > > > > > > > > builds with Java 8 (and up).
> > > > > > > > > > > >
> > > > > > > > > > > > Yes, though builds created with only Java 8 will miss
> > some
> > > > > > > optional
> > > > > > > > > > > > post-Java-8 features/implementations for the
> > "actor-typed",
> > > > > > > > "stream",
> > > > > > > > > > > > "remote" and "cluster-sharding" modules - so for a
> > faithful
> > > > > test
> > > > > > > > > you'd
> > > > > > > > > > > > ideally build with JDK 9-or-later while having the
> > Java 8
> > > > > runtime
> > > > > > > > > > > > available.
> > > > > > > > > > > >
> > > > > > > > > > > > (TBH I don't think this is _too_ different from other
> > > > > projects
> > > > > > > > where
> > > > > > > > > > > > you have to have some compile-time dependencies
> > installed
> > > > to
> > > > > be
> > > > > > > > able
> > > > > > > > > > > > to build the project)
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Kind regards,
> > > > > > > > > > > >
> > > > > > > > > > > > Arnout
> > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Justin,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > There is an issue in the code base that is
> > reasonably
> > > > > common
> > > > > > > > > among systems
> > > > > > > > > > > > > > that operate at low levels and are migrating from
> > Java
> > > > 8
> > > > > > > > > implementation
> > > > > > > > > > > > > > wherein they need the rt.jar from java 8 for
> > > > compilation.
> > > > > > > > >  There are
> > > > > > > > > > > > > > several ways to solve this problem, Pekko (and I
> > > > suppose
> > > > > > > Akka)
> > > > > > > > > require that
> > > > > > > > > > > > > > Java 8 be installed so that the build process can
> > find
> > > > > the
> > > > > > > Java
> > > > > > > > > 8 rt.jar.
> > > > > > > > > > > > > > I suspect that all that is actually required to
> > get the
> > > > > > > current
> > > > > > > > > code to
> > > > > > > > > > > > > > compile is setting JAVA_8_HOME to point to the
> > > > directory
> > > > > with
> > > > > > > > > the rt.jar in
> > > > > > > > > > > > > > it.  However, the Java 8 rt.jar is still
> > required.  I
> > > > > believe
> > > > > > > > > there is a
> > > > > > > > > > > > > > ticket to resolve this issue but I may be mistaken
> > > > about
> > > > > > > this.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Claude
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> > > > > > > > > justin@classsoftware.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Part of the requirement of a release is that
> > someone
> > > > > can
> > > > > > > > build
> > > > > > > > > it on their
> > > > > > > > > > > > > > > system, and that ideally should be easy to do. I
> > > > would
> > > > > > > assume
> > > > > > > > > by their
> > > > > > > > > > > > > > > system than means a single version of the JDK
> > which
> > > > > they
> > > > > > > use
> > > > > > > > > and is
> > > > > > > > > > > > > > > supported. I’m not sure there is be a need for
> > > > someone
> > > > > > > > testing
> > > > > > > > > the release
> > > > > > > > > > > > > > > to compile against multiple JDKs?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Kind Regards,
> > > > > > > > > > > > > > > Justin
> > > > > > > > > > > > > > >
> > > > > > > > >
> > > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > >
> > > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > To unsubscribe, e-mail:
> > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > For additional commands, e-mail:
> > > > dev-help@pekko.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Arnout Engelen
> > > > > > > > > > > > ASF Security Response
> > > > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > > > Committer on NixOS
> > > > > > > > > > > > Independent Open Source consultant
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > > > > > To unsubscribe, e-mail:
> > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > For additional commands, e-mail:
> > dev-help@pekko.apache.org
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> > dev-help@pekko.apache.org
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Arnout Engelen
> > > > > > > > > > ASF Security Response
> > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > Committer on NixOS
> > > > > > > > > > Independent Open Source consultant
> > > > > > > > > >
> > > > > > > > > >
> > > > > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Matthew de Detrich
> > > > > > > >
> > > > > > > > *Aiven Deutschland GmbH*
> > > > > > > >
> > > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > > >
> > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > > >
> > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > > >
> > > > > > > > *m:* +491603708037
> > > > > > > >
> > > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Matthew de Detrich
> > > > > >
> > > > > > *Aiven Deutschland GmbH*
> > > > > >
> > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > >
> > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > >
> > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > >
> > > > > > *m:* +491603708037
> > > > > >
> > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Matthew de Detrich
> > > >
> > > > *Aiven Deutschland GmbH*
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > *m:* +491603708037
> > > >
> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by "Claude Warren, Jr" <cl...@aiven.io.INVALID>.
Why not build 2 binary packages:  Java 8 and Java 11?  Use the java 8 JDK
to build the java 8 version and Java 11 JDK to build java 11 version.

Then no need to copy rt.jar

On Thu, May 25, 2023 at 2:35 PM Arnout Engelen <en...@apache.org> wrote:

> On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr
> <cl...@aiven.io.invalid> wrote:
> > Assume Java 11 is installed and no Java 8 is installed on the system
> where
> > Pekko is being deployed.
> > Assume Java 11 compilation with rt.jar present for the build of the Pekko
> > package being deployed. (My understanding that even with Java 11 the
> rt.jar
> > is required for compilation)
> >
> > When the code is executed on the deployed system and the class from the
> > rt.jar is required, what happens?
> > I would expect that a NoClassDefFoundError to be thrown.   If the error
> is
> > not thrown I can only see two possibilities:
> >
> >    1. the class(es) from the rt.jar are defined somewhere else in which
> >    case why can that not be used during the build?
> >    2. the class(es) are not used and the dependency on the rt.jar can be
> >    removed.
> >
> > So what happens and why?
>
> No NoClassDefFoundError will be thrown, because the runtime for Java
> 11 is backwards compatible with the runtime for Java 8, so there
> should be no classes/methods that exist in the jdk8 rt.jar that are
> not also available in the jdk11 runtime.
>
> From your two possibilities, '1' is true: they are defined both in the
> Java 8 and in the Java 11 runtimes. Indeed if we removed the rt.jar
> everything would still work on Java 11. Just no longer on Java 8. This
> is because the runtimes are not *forwards* compatible: there are
> things in the Java 11 runtime that are not in the Java 8 one.
>
> To illustrate and make this more tangible, one specific example is
> ByteBuffer.clear(): on Java 8, this method returned a Buffer. On Java
> 11, there is an implementation that returns a ByteBuffer. If you
> compile a class that calls this method against the Java 8 runtime and
> then run it against the Java 11 one, this is fine, because Java 11 has
> *both* a clear() method that returns a ByteBuffer and (for backwards
> compatibility) one that returns a Buffer. If you compile the same code
> against the Java 11 runtime and then run it against Java 8, it will
> pick the one that returns a ByteBuffer at compile time, and you will
> get a NoSuchMethod exception at runtime because this new method does
> not exist in the Java 8 runtime.
>
> Of course we could change the code to make sure this *particular*
> problem does not occur in Pekko, but that is risky as we don't know
> how many similar situations there are.
>
> Is there a better solution to this than compiling with two JVM's
> installed? Yes! Invoking the JDK11 compiler with '--release 8' will
> also generate the correct reference. However, '--release 8' does not
> allow access to sun.misc.Unsafe API's, which unfortunately Pekko still
> relies on - so we can't do that just yet.
>
>
> Kind regards,
>
> Arnout
>
> > On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich
> > <ma...@aiven.io.invalid> wrote:
> >
> > > > If you build Pekko with both Java 11 and Java 8 installed, the
> classes
> > > we will produce in our jars will be in Java 8 class format.
> > >
> > > More specifically, normally if you compile Java/Scala source with JDK
> 11
> > > (which is what Pekko does, JDK 8 is only necessary for its rt.jar) it
> will
> > > produce
> > > classfiles compatible with JDK 11 and above. You can however provide a
> flag
> > > to
> > > javac/scalac so that it generates class files with a specific bytecode
> > > format, i.e.
> > >
> > >
> https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60
> > > and
> > >
> > >
> https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73
> > > does.
> > >
> > > In short, Pekko is built using JDK 11 with Xtarget/release flags to
> force
> > > JDK 8
> > > compatible bytecode generation in addition to providing the JDK8
> rt.jar via
> > > the
> > > -bootclasspath javac flag.
> > >
> > > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <fa...@gmail.com>
> wrote:
> > >
> > > > > Q: does the build package the rt.jar in the final binary package?
> > > >
> > > > No. We only produce our own jars that we push to Maven - and these do
> > > > not embed any rt.jar classes (shaded or otherwise). We have no other
> > > > type of 'binary' package.
> > > >
> > > > > Q: if it does not, what happens if you try to run using Java 11 on
> a
> > > > system
> > > > > that does not have Java 8 nor the rt.jar installed?
> > > >
> > > > Java compiler produces class files. These have a Java version
> > > > associated with them and you can't use a class that is in Java 11
> > > > format in a Java 8 runtime.
> > > > If you build Pekko with both Java 11 and Java 8 installed, the
> classes
> > > > we will produce in our jars will be in Java 8 class format.
> > > >
> > > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich
> > > > <ma...@aiven.io.invalid> wrote:
> > > > >
> > > > > afaik Pekko will just fail to build at all if you use JDK 11
> without
> > > > > pointing to JDK 8 rt.jar.
> > > > >
> > > > > This is because Pekko uses sun.misc.unsafe which is only shipping
> with
> > > > JDK
> > > > > 8 and the
> > > > > reason why we have to build on JDK 11 is because there are features
> > > > > shipping in JDK 11
> > > > > which pekko uses.
> > > > >
> > > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
> > > > > <cl...@aiven.io.invalid> wrote:
> > > > >
> > > > > > I have a couple of questions:
> > > > > >
> > > > > > Q: does the build package the rt.jar in the final binary package?
> > > > > >
> > > > > > Q: if it does not, what happens if you try to run using Java 11
> on a
> > > > system
> > > > > > that does not have Java 8 nor the rt.jar installed?
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> > > > > > <ma...@aiven.io.invalid> wrote:
> > > > > >
> > > > > > > > We could provide a Docker image with the right sbt and Java
> setup
> > > > for
> > > > > > > convenience but I think it is fair not to require users to use
> > > > Docker.
> > > > > > >
> > > > > > > Considering how much implicit behaviour there is in making a
> full
> > > > build
> > > > > > > (as Arnout mentioned earlier, you can compile Pekko fine with
> just
> > > > JDK 8
> > > > > > > just fine however it will be missing post java 8 optional
> > > features) I
> > > > > > would
> > > > > > > prefer
> > > > > > > to strongly encourage people to use Docker to do a proper
> build.
> > > As a
> > > > > > bare
> > > > > > > minimum we can also print a warning in sbt telling users if
> they
> > > are
> > > > not
> > > > > > > going
> > > > > > > to be making a complete Pekko build when building on JDK 8 (as
> > > > stated it
> > > > > > > already errors out if you build on JDK 11 without pointing to
> a JDK
> > > > > > > rt.jar).
> > > > > > >
> > > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <
> fanningpj@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > > Thanks Arnout.
> > > > > > > >
> > > > > > > > I think if we clearly document what you need to set up to do
> a
> > > > build,
> > > > > > > > that should be enough. I think it is fair to assume that
> users
> > > can
> > > > > > > > install Java and sbt - and that they should be able to
> install 2
> > > > > > > > versions of Java if we are explicit about how to do this and
> why
> > > > this
> > > > > > > > is the best setup for building Pekko.
> > > > > > > >
> > > > > > > > We will need to expand what we have in
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > > > > > > >
> > > > > > > > This README file will appear in the source releases.
> > > > > > > >
> > > > > > > > We could provide a Docker image with the right sbt and Java
> setup
> > > > for
> > > > > > > > convenience but I think it is fair not to require users to
> use
> > > > Docker.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <
> engelen@apache.org
> > > >
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <
> > > fanningpj@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > > > > Ok. It seems the nightly job to publish the snapshot jars
> > > uses
> > > > Java
> > > > > > > > > > 11. It does not explicitly install Java 8 but maybe, it's
> > > > possible
> > > > > > > > > > that Java 8 is installed silently,
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > > > > > > suggests so, yes.
> > > > > > > > >
> > > > > > > > > > The nightly build does produce jars that can be used by
> Java
> > > 8.
> > > > > > > > > >
> > > > > > > > > > So if this nightly build relies on the silent
> installation of
> > > > Java
> > > > > > 8,
> > > > > > > > > > should we add a check in our sbt script that will fail if
> > > Java
> > > > 8 is
> > > > > > > > > > not found? This could be useful for users who are
> building on
> > > > their
> > > > > > > > > > own machines.
> > > > > > > > >
> > > > > > > > > This check already exists: when I build with jdk11 without
> > > having
> > > > > > jdk8
> > > > > > > > > available the build fails with:
> > > > > > > > >
> > > > > > > > > [error] A JDK 8 installation was not found, but is
> required to
> > > > build
> > > > > > > > > Apache Pekko. To manually specify a JDK 8 installation,
> use the
> > > > "set
> > > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you
> have no
> > > > JDK 8
> > > > > > > > > installation, target your system JDK with the "set every
> > > > > > > > > targetSystemJdk := true" sbt command, but beware resulting
> > > > artifacts
> > > > > > > > > will not work on JDK 8
> > > > > > > > >
> > > > > > > > > It would be good to also document the JAVA_8_HOME
> environment
> > > > > > variable
> > > > > > > > > in this error message. Created
> > > > > > > > > https://github.com/apache/incubator-pekko/pull/333 for
> that.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Kind regards,
> > > > > > > > >
> > > > > > > > > Arnout
> > > > > > > > >
> > > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <
> > > > engelen@apache.org>
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <
> > > > fanningpj@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > > > > > > Only building the website docs needs Java 11 (and
> up).
> > > > > > Everything
> > > > > > > > else
> > > > > > > > > > > > builds with Java 8 (and up).
> > > > > > > > > > >
> > > > > > > > > > > Yes, though builds created with only Java 8 will miss
> some
> > > > > > optional
> > > > > > > > > > > post-Java-8 features/implementations for the
> "actor-typed",
> > > > > > > "stream",
> > > > > > > > > > > "remote" and "cluster-sharding" modules - so for a
> faithful
> > > > test
> > > > > > > > you'd
> > > > > > > > > > > ideally build with JDK 9-or-later while having the
> Java 8
> > > > runtime
> > > > > > > > > > > available.
> > > > > > > > > > >
> > > > > > > > > > > (TBH I don't think this is _too_ different from other
> > > > projects
> > > > > > > where
> > > > > > > > > > > you have to have some compile-time dependencies
> installed
> > > to
> > > > be
> > > > > > > able
> > > > > > > > > > > to build the project)
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Kind regards,
> > > > > > > > > > >
> > > > > > > > > > > Arnout
> > > > > > > > > > >
> > > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Justin,
> > > > > > > > > > > > >
> > > > > > > > > > > > > There is an issue in the code base that is
> reasonably
> > > > common
> > > > > > > > among systems
> > > > > > > > > > > > > that operate at low levels and are migrating from
> Java
> > > 8
> > > > > > > > implementation
> > > > > > > > > > > > > wherein they need the rt.jar from java 8 for
> > > compilation.
> > > > > > > >  There are
> > > > > > > > > > > > > several ways to solve this problem, Pekko (and I
> > > suppose
> > > > > > Akka)
> > > > > > > > require that
> > > > > > > > > > > > > Java 8 be installed so that the build process can
> find
> > > > the
> > > > > > Java
> > > > > > > > 8 rt.jar.
> > > > > > > > > > > > > I suspect that all that is actually required to
> get the
> > > > > > current
> > > > > > > > code to
> > > > > > > > > > > > > compile is setting JAVA_8_HOME to point to the
> > > directory
> > > > with
> > > > > > > > the rt.jar in
> > > > > > > > > > > > > it.  However, the Java 8 rt.jar is still
> required.  I
> > > > believe
> > > > > > > > there is a
> > > > > > > > > > > > > ticket to resolve this issue but I may be mistaken
> > > about
> > > > > > this.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Claude
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> > > > > > > > justin@classsoftware.com>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Part of the requirement of a release is that
> someone
> > > > can
> > > > > > > build
> > > > > > > > it on their
> > > > > > > > > > > > > > system, and that ideally should be easy to do. I
> > > would
> > > > > > assume
> > > > > > > > by their
> > > > > > > > > > > > > > system than means a single version of the JDK
> which
> > > > they
> > > > > > use
> > > > > > > > and is
> > > > > > > > > > > > > > supported. I’m not sure there is be a need for
> > > someone
> > > > > > > testing
> > > > > > > > the release
> > > > > > > > > > > > > > to compile against multiple JDKs?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Kind Regards,
> > > > > > > > > > > > > > Justin
> > > > > > > > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > > For additional commands, e-mail:
> > > > dev-help@pekko.apache.org
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > For additional commands, e-mail:
> > > dev-help@pekko.apache.org
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Arnout Engelen
> > > > > > > > > > > ASF Security Response
> > > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > > Committer on NixOS
> > > > > > > > > > > Independent Open Source consultant
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> dev-help@pekko.apache.org
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > For additional commands, e-mail:
> dev-help@pekko.apache.org
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Arnout Engelen
> > > > > > > > > ASF Security Response
> > > > > > > > > Committer on Apache Pekko
> > > > > > > > > Committer on NixOS
> > > > > > > > > Independent Open Source consultant
> > > > > > > > >
> > > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Matthew de Detrich
> > > > > > >
> > > > > > > *Aiven Deutschland GmbH*
> > > > > > >
> > > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > > >
> > > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > > >
> > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > > >
> > > > > > > *m:* +491603708037
> > > > > > >
> > > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Matthew de Detrich
> > > > >
> > > > > *Aiven Deutschland GmbH*
> > > > >
> > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > >
> > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > >
> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > >
> > > > > *m:* +491603708037
> > > > >
> > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > >
> > > >
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>
>

Re: [DISCUSS] Building Binary Package

Posted by Arnout Engelen <en...@apache.org>.
On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr
<cl...@aiven.io.invalid> wrote:
> Assume Java 11 is installed and no Java 8 is installed on the system where
> Pekko is being deployed.
> Assume Java 11 compilation with rt.jar present for the build of the Pekko
> package being deployed. (My understanding that even with Java 11 the rt.jar
> is required for compilation)
>
> When the code is executed on the deployed system and the class from the
> rt.jar is required, what happens?
> I would expect that a NoClassDefFoundError to be thrown.   If the error is
> not thrown I can only see two possibilities:
>
>    1. the class(es) from the rt.jar are defined somewhere else in which
>    case why can that not be used during the build?
>    2. the class(es) are not used and the dependency on the rt.jar can be
>    removed.
>
> So what happens and why?

No NoClassDefFoundError will be thrown, because the runtime for Java
11 is backwards compatible with the runtime for Java 8, so there
should be no classes/methods that exist in the jdk8 rt.jar that are
not also available in the jdk11 runtime.

From your two possibilities, '1' is true: they are defined both in the
Java 8 and in the Java 11 runtimes. Indeed if we removed the rt.jar
everything would still work on Java 11. Just no longer on Java 8. This
is because the runtimes are not *forwards* compatible: there are
things in the Java 11 runtime that are not in the Java 8 one.

To illustrate and make this more tangible, one specific example is
ByteBuffer.clear(): on Java 8, this method returned a Buffer. On Java
11, there is an implementation that returns a ByteBuffer. If you
compile a class that calls this method against the Java 8 runtime and
then run it against the Java 11 one, this is fine, because Java 11 has
*both* a clear() method that returns a ByteBuffer and (for backwards
compatibility) one that returns a Buffer. If you compile the same code
against the Java 11 runtime and then run it against Java 8, it will
pick the one that returns a ByteBuffer at compile time, and you will
get a NoSuchMethod exception at runtime because this new method does
not exist in the Java 8 runtime.

Of course we could change the code to make sure this *particular*
problem does not occur in Pekko, but that is risky as we don't know
how many similar situations there are.

Is there a better solution to this than compiling with two JVM's
installed? Yes! Invoking the JDK11 compiler with '--release 8' will
also generate the correct reference. However, '--release 8' does not
allow access to sun.misc.Unsafe API's, which unfortunately Pekko still
relies on - so we can't do that just yet.


Kind regards,

Arnout

> On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich
> <ma...@aiven.io.invalid> wrote:
>
> > > If you build Pekko with both Java 11 and Java 8 installed, the classes
> > we will produce in our jars will be in Java 8 class format.
> >
> > More specifically, normally if you compile Java/Scala source with JDK 11
> > (which is what Pekko does, JDK 8 is only necessary for its rt.jar) it will
> > produce
> > classfiles compatible with JDK 11 and above. You can however provide a flag
> > to
> > javac/scalac so that it generates class files with a specific bytecode
> > format, i.e.
> >
> > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60
> > and
> >
> > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73
> > does.
> >
> > In short, Pekko is built using JDK 11 with Xtarget/release flags to force
> > JDK 8
> > compatible bytecode generation in addition to providing the JDK8 rt.jar via
> > the
> > -bootclasspath javac flag.
> >
> > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <fa...@gmail.com> wrote:
> >
> > > > Q: does the build package the rt.jar in the final binary package?
> > >
> > > No. We only produce our own jars that we push to Maven - and these do
> > > not embed any rt.jar classes (shaded or otherwise). We have no other
> > > type of 'binary' package.
> > >
> > > > Q: if it does not, what happens if you try to run using Java 11 on a
> > > system
> > > > that does not have Java 8 nor the rt.jar installed?
> > >
> > > Java compiler produces class files. These have a Java version
> > > associated with them and you can't use a class that is in Java 11
> > > format in a Java 8 runtime.
> > > If you build Pekko with both Java 11 and Java 8 installed, the classes
> > > we will produce in our jars will be in Java 8 class format.
> > >
> > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich
> > > <ma...@aiven.io.invalid> wrote:
> > > >
> > > > afaik Pekko will just fail to build at all if you use JDK 11 without
> > > > pointing to JDK 8 rt.jar.
> > > >
> > > > This is because Pekko uses sun.misc.unsafe which is only shipping with
> > > JDK
> > > > 8 and the
> > > > reason why we have to build on JDK 11 is because there are features
> > > > shipping in JDK 11
> > > > which pekko uses.
> > > >
> > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
> > > > <cl...@aiven.io.invalid> wrote:
> > > >
> > > > > I have a couple of questions:
> > > > >
> > > > > Q: does the build package the rt.jar in the final binary package?
> > > > >
> > > > > Q: if it does not, what happens if you try to run using Java 11 on a
> > > system
> > > > > that does not have Java 8 nor the rt.jar installed?
> > > > >
> > > > >
> > > > >
> > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> > > > > <ma...@aiven.io.invalid> wrote:
> > > > >
> > > > > > > We could provide a Docker image with the right sbt and Java setup
> > > for
> > > > > > convenience but I think it is fair not to require users to use
> > > Docker.
> > > > > >
> > > > > > Considering how much implicit behaviour there is in making a full
> > > build
> > > > > > (as Arnout mentioned earlier, you can compile Pekko fine with just
> > > JDK 8
> > > > > > just fine however it will be missing post java 8 optional
> > features) I
> > > > > would
> > > > > > prefer
> > > > > > to strongly encourage people to use Docker to do a proper build.
> > As a
> > > > > bare
> > > > > > minimum we can also print a warning in sbt telling users if they
> > are
> > > not
> > > > > > going
> > > > > > to be making a complete Pekko build when building on JDK 8 (as
> > > stated it
> > > > > > already errors out if you build on JDK 11 without pointing to a JDK
> > > > > > rt.jar).
> > > > > >
> > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <fa...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > Thanks Arnout.
> > > > > > >
> > > > > > > I think if we clearly document what you need to set up to do a
> > > build,
> > > > > > > that should be enough. I think it is fair to assume that users
> > can
> > > > > > > install Java and sbt - and that they should be able to install 2
> > > > > > > versions of Java if we are explicit about how to do this and why
> > > this
> > > > > > > is the best setup for building Pekko.
> > > > > > >
> > > > > > > We will need to expand what we have in
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> > https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > > > > > >
> > > > > > > This README file will appear in the source releases.
> > > > > > >
> > > > > > > We could provide a Docker image with the right sbt and Java setup
> > > for
> > > > > > > convenience but I think it is fair not to require users to use
> > > Docker.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <engelen@apache.org
> > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <
> > fanningpj@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > > > Ok. It seems the nightly job to publish the snapshot jars
> > uses
> > > Java
> > > > > > > > > 11. It does not explicitly install Java 8 but maybe, it's
> > > possible
> > > > > > > > > that Java 8 is installed silently,
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> > https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > > > > > suggests so, yes.
> > > > > > > >
> > > > > > > > > The nightly build does produce jars that can be used by Java
> > 8.
> > > > > > > > >
> > > > > > > > > So if this nightly build relies on the silent installation of
> > > Java
> > > > > 8,
> > > > > > > > > should we add a check in our sbt script that will fail if
> > Java
> > > 8 is
> > > > > > > > > not found? This could be useful for users who are building on
> > > their
> > > > > > > > > own machines.
> > > > > > > >
> > > > > > > > This check already exists: when I build with jdk11 without
> > having
> > > > > jdk8
> > > > > > > > available the build fails with:
> > > > > > > >
> > > > > > > > [error] A JDK 8 installation was not found, but is required to
> > > build
> > > > > > > > Apache Pekko. To manually specify a JDK 8 installation, use the
> > > "set
> > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you have no
> > > JDK 8
> > > > > > > > installation, target your system JDK with the "set every
> > > > > > > > targetSystemJdk := true" sbt command, but beware resulting
> > > artifacts
> > > > > > > > will not work on JDK 8
> > > > > > > >
> > > > > > > > It would be good to also document the JAVA_8_HOME environment
> > > > > variable
> > > > > > > > in this error message. Created
> > > > > > > > https://github.com/apache/incubator-pekko/pull/333 for that.
> > > > > > > >
> > > > > > > >
> > > > > > > > Kind regards,
> > > > > > > >
> > > > > > > > Arnout
> > > > > > > >
> > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <
> > > engelen@apache.org>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <
> > > fanningpj@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > > > > > > Only building the website docs needs Java 11 (and up).
> > > > > Everything
> > > > > > > else
> > > > > > > > > > > builds with Java 8 (and up).
> > > > > > > > > >
> > > > > > > > > > Yes, though builds created with only Java 8 will miss some
> > > > > optional
> > > > > > > > > > post-Java-8 features/implementations for the "actor-typed",
> > > > > > "stream",
> > > > > > > > > > "remote" and "cluster-sharding" modules - so for a faithful
> > > test
> > > > > > > you'd
> > > > > > > > > > ideally build with JDK 9-or-later while having the Java 8
> > > runtime
> > > > > > > > > > available.
> > > > > > > > > >
> > > > > > > > > > (TBH I don't think this is _too_ different from other
> > > projects
> > > > > > where
> > > > > > > > > > you have to have some compile-time dependencies installed
> > to
> > > be
> > > > > > able
> > > > > > > > > > to build the project)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Kind regards,
> > > > > > > > > >
> > > > > > > > > > Arnout
> > > > > > > > > >
> > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Justin,
> > > > > > > > > > > >
> > > > > > > > > > > > There is an issue in the code base that is reasonably
> > > common
> > > > > > > among systems
> > > > > > > > > > > > that operate at low levels and are migrating from Java
> > 8
> > > > > > > implementation
> > > > > > > > > > > > wherein they need the rt.jar from java 8 for
> > compilation.
> > > > > > >  There are
> > > > > > > > > > > > several ways to solve this problem, Pekko (and I
> > suppose
> > > > > Akka)
> > > > > > > require that
> > > > > > > > > > > > Java 8 be installed so that the build process can find
> > > the
> > > > > Java
> > > > > > > 8 rt.jar.
> > > > > > > > > > > > I suspect that all that is actually required to get the
> > > > > current
> > > > > > > code to
> > > > > > > > > > > > compile is setting JAVA_8_HOME to point to the
> > directory
> > > with
> > > > > > > the rt.jar in
> > > > > > > > > > > > it.  However, the Java 8 rt.jar is still required.  I
> > > believe
> > > > > > > there is a
> > > > > > > > > > > > ticket to resolve this issue but I may be mistaken
> > about
> > > > > this.
> > > > > > > > > > > >
> > > > > > > > > > > > Claude
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> > > > > > > justin@classsoftware.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Part of the requirement of a release is that someone
> > > can
> > > > > > build
> > > > > > > it on their
> > > > > > > > > > > > > system, and that ideally should be easy to do. I
> > would
> > > > > assume
> > > > > > > by their
> > > > > > > > > > > > > system than means a single version of the JDK which
> > > they
> > > > > use
> > > > > > > and is
> > > > > > > > > > > > > supported. I’m not sure there is be a need for
> > someone
> > > > > > testing
> > > > > > > the release
> > > > > > > > > > > > > to compile against multiple JDKs?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Kind Regards,
> > > > > > > > > > > > > Justin
> > > > > > > > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > > > To unsubscribe, e-mail:
> > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > For additional commands, e-mail:
> > > dev-help@pekko.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> > dev-help@pekko.apache.org
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Arnout Engelen
> > > > > > > > > > ASF Security Response
> > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > Committer on NixOS
> > > > > > > > > > Independent Open Source consultant
> > > > > > > > > >
> > > > > > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Arnout Engelen
> > > > > > > > ASF Security Response
> > > > > > > > Committer on Apache Pekko
> > > > > > > > Committer on NixOS
> > > > > > > > Independent Open Source consultant
> > > > > > > >
> > > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Matthew de Detrich
> > > > > >
> > > > > > *Aiven Deutschland GmbH*
> > > > > >
> > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > >
> > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > >
> > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > >
> > > > > > *m:* +491603708037
> > > > > >
> > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Matthew de Detrich
> > > >
> > > > *Aiven Deutschland GmbH*
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > *m:* +491603708037
> > > >
> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> > >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
> Assume Java 11 is installed and no Java 8 is installed on the system where
Pekko is being deployed.

If by deployed you mean compile (i.e. generate .class files) this will just
fail to compile with the following error

[error] A JDK 8 installation was not found, but is required to build Apache
Pekko. To manually specify a JDK 8 installation, set the JAVA_8_HOME
environment variable to its path or use the "set every jdk8home :=
\"/path/to/jdk\" sbt command. If you have no JDK 8 installation, target
your system JDK with the "set every targetSystemJdk := true" sbt command,
but beware resulting artifacts will not work on JDK 8

If by deployed you mean running, Pekko will run fine on JDK 8. We have
github actions CI setup that triggers on every pull request to verify this.

> Assume Java 11 compilation with rt.jar present for the build of the Pekko
package being deployed. (My understanding that even with Java 11 the rt.jar
is required for compilation)

Under the hood when Pekko is built correctly it uses JDK 11 with specific
javac/scalc flags to generate JDK 8 bytecode will passing in JDK 8's rt.jar
using -bootclasspath>

> When the code is executed on the deployed system and the class from the
rt.jar is required, what happens?

Afaik The reason why rt.jar from JDK 8 is required is otherwise javac will
fail in linking during compile face because it cannot find references to
sun.misc.unsafe within JDK 8's rt.jar that Pekko uses (and this is because
we are building with JDK 11)

> I would expect that a NoClassDefFoundError to be thrown.   If the error is
not thrown I can only see two possibilities:

It either wouldn't compile or you would get a NoMethodFound error or
something along those lines at runtime (as you state). In Pekko you have
source code that references certain methods in sun.misc.unsafe which was
removed in JDK 11 (or more specifically JDK 11's rt.jar doesn't contain
does methods).

On a related note, sun.misc.unsafe is a critical part of Pekko. It's used
in the core actor module for understandable reasons (actor's need to be as
fast as possible, it's not uncommon to have millions of instances of actors
just like in Erlang and it's the core concurrency primitive in Akka). In
order to not use sun.misc.unsafe we have to migrate to the more modern
replacements.

With this being said, while it's definitely enlightening to have this
conversation to figure out how and why Pekko compiles/builds, I would be
cautious in deliberating on this point too much. Without dropping JDK 8 we
can't really solve this issue, and as pointed out by Arnout the sbt build
has already been very comprehensively setup so that you cannot shoot
yourself in the foot (my very minor complaint is that if Pekko sees that
you are building on Java 8 it should warn you that certain artifacts will
not have full functionality). The best way forward is to simply document
and to make things easier (i.e. providing a Docker build). Even if we could
improve this somehow, there is a very big risk we can break something
silently without knowing, last time I tried to improve this part of the
build it involved having to bring in maintainers of Scalac (see
https://github.com/apache/incubator-pekko/pull/258) along with manually
inspecting JVM bytecode to make sure everything went as planned so I would
say thats its not a controversial statement that the community (especially
myself) doesn't really have a tolerance for touching this part of the code
right now.

A much more ideal solution would be to just make a release via github
actions CI as we do now with snapshots, but for various reasons (some
related to ASF process and some technical) we are not at the stage to
explore it and even if we could it would very likely be a lot more work
than a Docker build hence the timing not the best.




On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr
<cl...@aiven.io.invalid> wrote:

> Assume Java 11 is installed and no Java 8 is installed on the system where
> Pekko is being deployed.
> Assume Java 11 compilation with rt.jar present for the build of the Pekko
> package being deployed. (My understanding that even with Java 11 the rt.jar
> is required for compilation)
>
> When the code is executed on the deployed system and the class from the
> rt.jar is required, what happens?
> I would expect that a NoClassDefFoundError to be thrown.   If the error is
> not thrown I can only see two possibilities:
>
>    1. the class(es) from the rt.jar are defined somewhere else in which
>    case why can that not be used during the build?
>    2. the class(es) are not used and the dependency on the rt.jar can be
>    removed.
>
> So what happens and why?
>
>
> On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich
> <ma...@aiven.io.invalid> wrote:
>
> > > If you build Pekko with both Java 11 and Java 8 installed, the classes
> > we will produce in our jars will be in Java 8 class format.
> >
> > More specifically, normally if you compile Java/Scala source with JDK 11
> > (which is what Pekko does, JDK 8 is only necessary for its rt.jar) it
> will
> > produce
> > classfiles compatible with JDK 11 and above. You can however provide a
> flag
> > to
> > javac/scalac so that it generates class files with a specific bytecode
> > format, i.e.
> >
> >
> https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60
> > and
> >
> >
> https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73
> > does.
> >
> > In short, Pekko is built using JDK 11 with Xtarget/release flags to force
> > JDK 8
> > compatible bytecode generation in addition to providing the JDK8 rt.jar
> via
> > the
> > -bootclasspath javac flag.
> >
> > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <fa...@gmail.com> wrote:
> >
> > > > Q: does the build package the rt.jar in the final binary package?
> > >
> > > No. We only produce our own jars that we push to Maven - and these do
> > > not embed any rt.jar classes (shaded or otherwise). We have no other
> > > type of 'binary' package.
> > >
> > > > Q: if it does not, what happens if you try to run using Java 11 on a
> > > system
> > > > that does not have Java 8 nor the rt.jar installed?
> > >
> > > Java compiler produces class files. These have a Java version
> > > associated with them and you can't use a class that is in Java 11
> > > format in a Java 8 runtime.
> > > If you build Pekko with both Java 11 and Java 8 installed, the classes
> > > we will produce in our jars will be in Java 8 class format.
> > >
> > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich
> > > <ma...@aiven.io.invalid> wrote:
> > > >
> > > > afaik Pekko will just fail to build at all if you use JDK 11 without
> > > > pointing to JDK 8 rt.jar.
> > > >
> > > > This is because Pekko uses sun.misc.unsafe which is only shipping
> with
> > > JDK
> > > > 8 and the
> > > > reason why we have to build on JDK 11 is because there are features
> > > > shipping in JDK 11
> > > > which pekko uses.
> > > >
> > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
> > > > <cl...@aiven.io.invalid> wrote:
> > > >
> > > > > I have a couple of questions:
> > > > >
> > > > > Q: does the build package the rt.jar in the final binary package?
> > > > >
> > > > > Q: if it does not, what happens if you try to run using Java 11 on
> a
> > > system
> > > > > that does not have Java 8 nor the rt.jar installed?
> > > > >
> > > > >
> > > > >
> > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> > > > > <ma...@aiven.io.invalid> wrote:
> > > > >
> > > > > > > We could provide a Docker image with the right sbt and Java
> setup
> > > for
> > > > > > convenience but I think it is fair not to require users to use
> > > Docker.
> > > > > >
> > > > > > Considering how much implicit behaviour there is in making a full
> > > build
> > > > > > (as Arnout mentioned earlier, you can compile Pekko fine with
> just
> > > JDK 8
> > > > > > just fine however it will be missing post java 8 optional
> > features) I
> > > > > would
> > > > > > prefer
> > > > > > to strongly encourage people to use Docker to do a proper build.
> > As a
> > > > > bare
> > > > > > minimum we can also print a warning in sbt telling users if they
> > are
> > > not
> > > > > > going
> > > > > > to be making a complete Pekko build when building on JDK 8 (as
> > > stated it
> > > > > > already errors out if you build on JDK 11 without pointing to a
> JDK
> > > > > > rt.jar).
> > > > > >
> > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <fanningpj@gmail.com
> >
> > > wrote:
> > > > > >
> > > > > > > Thanks Arnout.
> > > > > > >
> > > > > > > I think if we clearly document what you need to set up to do a
> > > build,
> > > > > > > that should be enough. I think it is fair to assume that users
> > can
> > > > > > > install Java and sbt - and that they should be able to install
> 2
> > > > > > > versions of Java if we are explicit about how to do this and
> why
> > > this
> > > > > > > is the best setup for building Pekko.
> > > > > > >
> > > > > > > We will need to expand what we have in
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
> https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > > > > > >
> > > > > > > This README file will appear in the source releases.
> > > > > > >
> > > > > > > We could provide a Docker image with the right sbt and Java
> setup
> > > for
> > > > > > > convenience but I think it is fair not to require users to use
> > > Docker.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <
> engelen@apache.org
> > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <
> > fanningpj@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > > > Ok. It seems the nightly job to publish the snapshot jars
> > uses
> > > Java
> > > > > > > > > 11. It does not explicitly install Java 8 but maybe, it's
> > > possible
> > > > > > > > > that Java 8 is installed silently,
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
> https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > > > > > suggests so, yes.
> > > > > > > >
> > > > > > > > > The nightly build does produce jars that can be used by
> Java
> > 8.
> > > > > > > > >
> > > > > > > > > So if this nightly build relies on the silent installation
> of
> > > Java
> > > > > 8,
> > > > > > > > > should we add a check in our sbt script that will fail if
> > Java
> > > 8 is
> > > > > > > > > not found? This could be useful for users who are building
> on
> > > their
> > > > > > > > > own machines.
> > > > > > > >
> > > > > > > > This check already exists: when I build with jdk11 without
> > having
> > > > > jdk8
> > > > > > > > available the build fails with:
> > > > > > > >
> > > > > > > > [error] A JDK 8 installation was not found, but is required
> to
> > > build
> > > > > > > > Apache Pekko. To manually specify a JDK 8 installation, use
> the
> > > "set
> > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you have
> no
> > > JDK 8
> > > > > > > > installation, target your system JDK with the "set every
> > > > > > > > targetSystemJdk := true" sbt command, but beware resulting
> > > artifacts
> > > > > > > > will not work on JDK 8
> > > > > > > >
> > > > > > > > It would be good to also document the JAVA_8_HOME environment
> > > > > variable
> > > > > > > > in this error message. Created
> > > > > > > > https://github.com/apache/incubator-pekko/pull/333 for that.
> > > > > > > >
> > > > > > > >
> > > > > > > > Kind regards,
> > > > > > > >
> > > > > > > > Arnout
> > > > > > > >
> > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <
> > > engelen@apache.org>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <
> > > fanningpj@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > > > > > > Only building the website docs needs Java 11 (and up).
> > > > > Everything
> > > > > > > else
> > > > > > > > > > > builds with Java 8 (and up).
> > > > > > > > > >
> > > > > > > > > > Yes, though builds created with only Java 8 will miss
> some
> > > > > optional
> > > > > > > > > > post-Java-8 features/implementations for the
> "actor-typed",
> > > > > > "stream",
> > > > > > > > > > "remote" and "cluster-sharding" modules - so for a
> faithful
> > > test
> > > > > > > you'd
> > > > > > > > > > ideally build with JDK 9-or-later while having the Java 8
> > > runtime
> > > > > > > > > > available.
> > > > > > > > > >
> > > > > > > > > > (TBH I don't think this is _too_ different from other
> > > projects
> > > > > > where
> > > > > > > > > > you have to have some compile-time dependencies installed
> > to
> > > be
> > > > > > able
> > > > > > > > > > to build the project)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Kind regards,
> > > > > > > > > >
> > > > > > > > > > Arnout
> > > > > > > > > >
> > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Justin,
> > > > > > > > > > > >
> > > > > > > > > > > > There is an issue in the code base that is reasonably
> > > common
> > > > > > > among systems
> > > > > > > > > > > > that operate at low levels and are migrating from
> Java
> > 8
> > > > > > > implementation
> > > > > > > > > > > > wherein they need the rt.jar from java 8 for
> > compilation.
> > > > > > >  There are
> > > > > > > > > > > > several ways to solve this problem, Pekko (and I
> > suppose
> > > > > Akka)
> > > > > > > require that
> > > > > > > > > > > > Java 8 be installed so that the build process can
> find
> > > the
> > > > > Java
> > > > > > > 8 rt.jar.
> > > > > > > > > > > > I suspect that all that is actually required to get
> the
> > > > > current
> > > > > > > code to
> > > > > > > > > > > > compile is setting JAVA_8_HOME to point to the
> > directory
> > > with
> > > > > > > the rt.jar in
> > > > > > > > > > > > it.  However, the Java 8 rt.jar is still required.  I
> > > believe
> > > > > > > there is a
> > > > > > > > > > > > ticket to resolve this issue but I may be mistaken
> > about
> > > > > this.
> > > > > > > > > > > >
> > > > > > > > > > > > Claude
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> > > > > > > justin@classsoftware.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Part of the requirement of a release is that
> someone
> > > can
> > > > > > build
> > > > > > > it on their
> > > > > > > > > > > > > system, and that ideally should be easy to do. I
> > would
> > > > > assume
> > > > > > > by their
> > > > > > > > > > > > > system than means a single version of the JDK which
> > > they
> > > > > use
> > > > > > > and is
> > > > > > > > > > > > > supported. I’m not sure there is be a need for
> > someone
> > > > > > testing
> > > > > > > the release
> > > > > > > > > > > > > to compile against multiple JDKs?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Kind Regards,
> > > > > > > > > > > > > Justin
> > > > > > > > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > > > To unsubscribe, e-mail:
> > > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > > For additional commands, e-mail:
> > > dev-help@pekko.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> > dev-help@pekko.apache.org
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Arnout Engelen
> > > > > > > > > > ASF Security Response
> > > > > > > > > > Committer on Apache Pekko
> > > > > > > > > > Committer on NixOS
> > > > > > > > > > Independent Open Source consultant
> > > > > > > > > >
> > > > > > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > For additional commands, e-mail:
> dev-help@pekko.apache.org
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Arnout Engelen
> > > > > > > > ASF Security Response
> > > > > > > > Committer on Apache Pekko
> > > > > > > > Committer on NixOS
> > > > > > > > Independent Open Source consultant
> > > > > > > >
> > > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Matthew de Detrich
> > > > > >
> > > > > > *Aiven Deutschland GmbH*
> > > > > >
> > > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > > >
> > > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > > >
> > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > > >
> > > > > > *m:* +491603708037
> > > > > >
> > > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Matthew de Detrich
> > > >
> > > > *Aiven Deutschland GmbH*
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > *m:* +491603708037
> > > >
> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> > >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] Building Binary Package

Posted by "Claude Warren, Jr" <cl...@aiven.io.INVALID>.
Assume Java 11 is installed and no Java 8 is installed on the system where
Pekko is being deployed.
Assume Java 11 compilation with rt.jar present for the build of the Pekko
package being deployed. (My understanding that even with Java 11 the rt.jar
is required for compilation)

When the code is executed on the deployed system and the class from the
rt.jar is required, what happens?
I would expect that a NoClassDefFoundError to be thrown.   If the error is
not thrown I can only see two possibilities:

   1. the class(es) from the rt.jar are defined somewhere else in which
   case why can that not be used during the build?
   2. the class(es) are not used and the dependency on the rt.jar can be
   removed.

So what happens and why?


On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich
<ma...@aiven.io.invalid> wrote:

> > If you build Pekko with both Java 11 and Java 8 installed, the classes
> we will produce in our jars will be in Java 8 class format.
>
> More specifically, normally if you compile Java/Scala source with JDK 11
> (which is what Pekko does, JDK 8 is only necessary for its rt.jar) it will
> produce
> classfiles compatible with JDK 11 and above. You can however provide a flag
> to
> javac/scalac so that it generates class files with a specific bytecode
> format, i.e.
>
> https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60
> and
>
> https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73
> does.
>
> In short, Pekko is built using JDK 11 with Xtarget/release flags to force
> JDK 8
> compatible bytecode generation in addition to providing the JDK8 rt.jar via
> the
> -bootclasspath javac flag.
>
> On Wed, May 24, 2023 at 9:44 PM PJ Fanning <fa...@gmail.com> wrote:
>
> > > Q: does the build package the rt.jar in the final binary package?
> >
> > No. We only produce our own jars that we push to Maven - and these do
> > not embed any rt.jar classes (shaded or otherwise). We have no other
> > type of 'binary' package.
> >
> > > Q: if it does not, what happens if you try to run using Java 11 on a
> > system
> > > that does not have Java 8 nor the rt.jar installed?
> >
> > Java compiler produces class files. These have a Java version
> > associated with them and you can't use a class that is in Java 11
> > format in a Java 8 runtime.
> > If you build Pekko with both Java 11 and Java 8 installed, the classes
> > we will produce in our jars will be in Java 8 class format.
> >
> > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich
> > <ma...@aiven.io.invalid> wrote:
> > >
> > > afaik Pekko will just fail to build at all if you use JDK 11 without
> > > pointing to JDK 8 rt.jar.
> > >
> > > This is because Pekko uses sun.misc.unsafe which is only shipping with
> > JDK
> > > 8 and the
> > > reason why we have to build on JDK 11 is because there are features
> > > shipping in JDK 11
> > > which pekko uses.
> > >
> > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
> > > <cl...@aiven.io.invalid> wrote:
> > >
> > > > I have a couple of questions:
> > > >
> > > > Q: does the build package the rt.jar in the final binary package?
> > > >
> > > > Q: if it does not, what happens if you try to run using Java 11 on a
> > system
> > > > that does not have Java 8 nor the rt.jar installed?
> > > >
> > > >
> > > >
> > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> > > > <ma...@aiven.io.invalid> wrote:
> > > >
> > > > > > We could provide a Docker image with the right sbt and Java setup
> > for
> > > > > convenience but I think it is fair not to require users to use
> > Docker.
> > > > >
> > > > > Considering how much implicit behaviour there is in making a full
> > build
> > > > > (as Arnout mentioned earlier, you can compile Pekko fine with just
> > JDK 8
> > > > > just fine however it will be missing post java 8 optional
> features) I
> > > > would
> > > > > prefer
> > > > > to strongly encourage people to use Docker to do a proper build.
> As a
> > > > bare
> > > > > minimum we can also print a warning in sbt telling users if they
> are
> > not
> > > > > going
> > > > > to be making a complete Pekko build when building on JDK 8 (as
> > stated it
> > > > > already errors out if you build on JDK 11 without pointing to a JDK
> > > > > rt.jar).
> > > > >
> > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <fa...@gmail.com>
> > wrote:
> > > > >
> > > > > > Thanks Arnout.
> > > > > >
> > > > > > I think if we clearly document what you need to set up to do a
> > build,
> > > > > > that should be enough. I think it is fair to assume that users
> can
> > > > > > install Java and sbt - and that they should be able to install 2
> > > > > > versions of Java if we are explicit about how to do this and why
> > this
> > > > > > is the best setup for building Pekko.
> > > > > >
> > > > > > We will need to expand what we have in
> > > > > >
> > > > > >
> > > > >
> > > >
> >
> https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > > > > >
> > > > > > This README file will appear in the source releases.
> > > > > >
> > > > > > We could provide a Docker image with the right sbt and Java setup
> > for
> > > > > > convenience but I think it is fair not to require users to use
> > Docker.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <engelen@apache.org
> >
> > > > wrote:
> > > > > > >
> > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <
> fanningpj@gmail.com
> > >
> > > > > wrote:
> > > > > > > > Ok. It seems the nightly job to publish the snapshot jars
> uses
> > Java
> > > > > > > > 11. It does not explicitly install Java 8 but maybe, it's
> > possible
> > > > > > > > that Java 8 is installed silently,
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
> https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > > > > suggests so, yes.
> > > > > > >
> > > > > > > > The nightly build does produce jars that can be used by Java
> 8.
> > > > > > > >
> > > > > > > > So if this nightly build relies on the silent installation of
> > Java
> > > > 8,
> > > > > > > > should we add a check in our sbt script that will fail if
> Java
> > 8 is
> > > > > > > > not found? This could be useful for users who are building on
> > their
> > > > > > > > own machines.
> > > > > > >
> > > > > > > This check already exists: when I build with jdk11 without
> having
> > > > jdk8
> > > > > > > available the build fails with:
> > > > > > >
> > > > > > > [error] A JDK 8 installation was not found, but is required to
> > build
> > > > > > > Apache Pekko. To manually specify a JDK 8 installation, use the
> > "set
> > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you have no
> > JDK 8
> > > > > > > installation, target your system JDK with the "set every
> > > > > > > targetSystemJdk := true" sbt command, but beware resulting
> > artifacts
> > > > > > > will not work on JDK 8
> > > > > > >
> > > > > > > It would be good to also document the JAVA_8_HOME environment
> > > > variable
> > > > > > > in this error message. Created
> > > > > > > https://github.com/apache/incubator-pekko/pull/333 for that.
> > > > > > >
> > > > > > >
> > > > > > > Kind regards,
> > > > > > >
> > > > > > > Arnout
> > > > > > >
> > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <
> > engelen@apache.org>
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <
> > fanningpj@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > > > > > Only building the website docs needs Java 11 (and up).
> > > > Everything
> > > > > > else
> > > > > > > > > > builds with Java 8 (and up).
> > > > > > > > >
> > > > > > > > > Yes, though builds created with only Java 8 will miss some
> > > > optional
> > > > > > > > > post-Java-8 features/implementations for the "actor-typed",
> > > > > "stream",
> > > > > > > > > "remote" and "cluster-sharding" modules - so for a faithful
> > test
> > > > > > you'd
> > > > > > > > > ideally build with JDK 9-or-later while having the Java 8
> > runtime
> > > > > > > > > available.
> > > > > > > > >
> > > > > > > > > (TBH I don't think this is _too_ different from other
> > projects
> > > > > where
> > > > > > > > > you have to have some compile-time dependencies installed
> to
> > be
> > > > > able
> > > > > > > > > to build the project)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Kind regards,
> > > > > > > > >
> > > > > > > > > Arnout
> > > > > > > > >
> > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > > > >
> > > > > > > > > > > Justin,
> > > > > > > > > > >
> > > > > > > > > > > There is an issue in the code base that is reasonably
> > common
> > > > > > among systems
> > > > > > > > > > > that operate at low levels and are migrating from Java
> 8
> > > > > > implementation
> > > > > > > > > > > wherein they need the rt.jar from java 8 for
> compilation.
> > > > > >  There are
> > > > > > > > > > > several ways to solve this problem, Pekko (and I
> suppose
> > > > Akka)
> > > > > > require that
> > > > > > > > > > > Java 8 be installed so that the build process can find
> > the
> > > > Java
> > > > > > 8 rt.jar.
> > > > > > > > > > > I suspect that all that is actually required to get the
> > > > current
> > > > > > code to
> > > > > > > > > > > compile is setting JAVA_8_HOME to point to the
> directory
> > with
> > > > > > the rt.jar in
> > > > > > > > > > > it.  However, the Java 8 rt.jar is still required.  I
> > believe
> > > > > > there is a
> > > > > > > > > > > ticket to resolve this issue but I may be mistaken
> about
> > > > this.
> > > > > > > > > > >
> > > > > > > > > > > Claude
> > > > > > > > > > >
> > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> > > > > > justin@classsoftware.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi,
> > > > > > > > > > > >
> > > > > > > > > > > > Part of the requirement of a release is that someone
> > can
> > > > > build
> > > > > > it on their
> > > > > > > > > > > > system, and that ideally should be easy to do. I
> would
> > > > assume
> > > > > > by their
> > > > > > > > > > > > system than means a single version of the JDK which
> > they
> > > > use
> > > > > > and is
> > > > > > > > > > > > supported. I’m not sure there is be a need for
> someone
> > > > > testing
> > > > > > the release
> > > > > > > > > > > > to compile against multiple JDKs?
> > > > > > > > > > > >
> > > > > > > > > > > > Kind Regards,
> > > > > > > > > > > > Justin
> > > > > > > > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > > > To unsubscribe, e-mail:
> > dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > > For additional commands, e-mail:
> > dev-help@pekko.apache.org
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > For additional commands, e-mail:
> dev-help@pekko.apache.org
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Arnout Engelen
> > > > > > > > > ASF Security Response
> > > > > > > > > Committer on Apache Pekko
> > > > > > > > > Committer on NixOS
> > > > > > > > > Independent Open Source consultant
> > > > > > > > >
> > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Arnout Engelen
> > > > > > > ASF Security Response
> > > > > > > Committer on Apache Pekko
> > > > > > > Committer on NixOS
> > > > > > > Independent Open Source consultant
> > > > > > >
> > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Matthew de Detrich
> > > > >
> > > > > *Aiven Deutschland GmbH*
> > > > >
> > > > > Immanuelkirchstraße 26, 10405 Berlin
> > > > >
> > > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > > >
> > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > >
> > > > > *m:* +491603708037
> > > > >
> > > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
> >
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>

Re: [DISCUSS] Building Binary Package

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
> If you build Pekko with both Java 11 and Java 8 installed, the classes
we will produce in our jars will be in Java 8 class format.

More specifically, normally if you compile Java/Scala source with JDK 11
(which is what Pekko does, JDK 8 is only necessary for its rt.jar) it will
produce
classfiles compatible with JDK 11 and above. You can however provide a flag
to
javac/scalac so that it generates class files with a specific bytecode
format, i.e.
https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60
and
https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73
does.

In short, Pekko is built using JDK 11 with Xtarget/release flags to force
JDK 8
compatible bytecode generation in addition to providing the JDK8 rt.jar via
the
-bootclasspath javac flag.

On Wed, May 24, 2023 at 9:44 PM PJ Fanning <fa...@gmail.com> wrote:

> > Q: does the build package the rt.jar in the final binary package?
>
> No. We only produce our own jars that we push to Maven - and these do
> not embed any rt.jar classes (shaded or otherwise). We have no other
> type of 'binary' package.
>
> > Q: if it does not, what happens if you try to run using Java 11 on a
> system
> > that does not have Java 8 nor the rt.jar installed?
>
> Java compiler produces class files. These have a Java version
> associated with them and you can't use a class that is in Java 11
> format in a Java 8 runtime.
> If you build Pekko with both Java 11 and Java 8 installed, the classes
> we will produce in our jars will be in Java 8 class format.
>
> On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich
> <ma...@aiven.io.invalid> wrote:
> >
> > afaik Pekko will just fail to build at all if you use JDK 11 without
> > pointing to JDK 8 rt.jar.
> >
> > This is because Pekko uses sun.misc.unsafe which is only shipping with
> JDK
> > 8 and the
> > reason why we have to build on JDK 11 is because there are features
> > shipping in JDK 11
> > which pekko uses.
> >
> > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
> > <cl...@aiven.io.invalid> wrote:
> >
> > > I have a couple of questions:
> > >
> > > Q: does the build package the rt.jar in the final binary package?
> > >
> > > Q: if it does not, what happens if you try to run using Java 11 on a
> system
> > > that does not have Java 8 nor the rt.jar installed?
> > >
> > >
> > >
> > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> > > <ma...@aiven.io.invalid> wrote:
> > >
> > > > > We could provide a Docker image with the right sbt and Java setup
> for
> > > > convenience but I think it is fair not to require users to use
> Docker.
> > > >
> > > > Considering how much implicit behaviour there is in making a full
> build
> > > > (as Arnout mentioned earlier, you can compile Pekko fine with just
> JDK 8
> > > > just fine however it will be missing post java 8 optional features) I
> > > would
> > > > prefer
> > > > to strongly encourage people to use Docker to do a proper build. As a
> > > bare
> > > > minimum we can also print a warning in sbt telling users if they are
> not
> > > > going
> > > > to be making a complete Pekko build when building on JDK 8 (as
> stated it
> > > > already errors out if you build on JDK 11 without pointing to a JDK
> > > > rt.jar).
> > > >
> > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <fa...@gmail.com>
> wrote:
> > > >
> > > > > Thanks Arnout.
> > > > >
> > > > > I think if we clearly document what you need to set up to do a
> build,
> > > > > that should be enough. I think it is fair to assume that users can
> > > > > install Java and sbt - and that they should be able to install 2
> > > > > versions of Java if we are explicit about how to do this and why
> this
> > > > > is the best setup for building Pekko.
> > > > >
> > > > > We will need to expand what we have in
> > > > >
> > > > >
> > > >
> > >
> https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > > > >
> > > > > This README file will appear in the source releases.
> > > > >
> > > > > We could provide a Docker image with the right sbt and Java setup
> for
> > > > > convenience but I think it is fair not to require users to use
> Docker.
> > > > >
> > > > >
> > > > >
> > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <en...@apache.org>
> > > wrote:
> > > > > >
> > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <fanningpj@gmail.com
> >
> > > > wrote:
> > > > > > > Ok. It seems the nightly job to publish the snapshot jars uses
> Java
> > > > > > > 11. It does not explicitly install Java 8 but maybe, it's
> possible
> > > > > > > that Java 8 is installed silently,
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > > > suggests so, yes.
> > > > > >
> > > > > > > The nightly build does produce jars that can be used by Java 8.
> > > > > > >
> > > > > > > So if this nightly build relies on the silent installation of
> Java
> > > 8,
> > > > > > > should we add a check in our sbt script that will fail if Java
> 8 is
> > > > > > > not found? This could be useful for users who are building on
> their
> > > > > > > own machines.
> > > > > >
> > > > > > This check already exists: when I build with jdk11 without having
> > > jdk8
> > > > > > available the build fails with:
> > > > > >
> > > > > > [error] A JDK 8 installation was not found, but is required to
> build
> > > > > > Apache Pekko. To manually specify a JDK 8 installation, use the
> "set
> > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you have no
> JDK 8
> > > > > > installation, target your system JDK with the "set every
> > > > > > targetSystemJdk := true" sbt command, but beware resulting
> artifacts
> > > > > > will not work on JDK 8
> > > > > >
> > > > > > It would be good to also document the JAVA_8_HOME environment
> > > variable
> > > > > > in this error message. Created
> > > > > > https://github.com/apache/incubator-pekko/pull/333 for that.
> > > > > >
> > > > > >
> > > > > > Kind regards,
> > > > > >
> > > > > > Arnout
> > > > > >
> > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <
> engelen@apache.org>
> > > > > wrote:
> > > > > > > >
> > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <
> fanningpj@gmail.com
> > > >
> > > > > wrote:
> > > > > > > > > Only building the website docs needs Java 11 (and up).
> > > Everything
> > > > > else
> > > > > > > > > builds with Java 8 (and up).
> > > > > > > >
> > > > > > > > Yes, though builds created with only Java 8 will miss some
> > > optional
> > > > > > > > post-Java-8 features/implementations for the "actor-typed",
> > > > "stream",
> > > > > > > > "remote" and "cluster-sharding" modules - so for a faithful
> test
> > > > > you'd
> > > > > > > > ideally build with JDK 9-or-later while having the Java 8
> runtime
> > > > > > > > available.
> > > > > > > >
> > > > > > > > (TBH I don't think this is _too_ different from other
> projects
> > > > where
> > > > > > > > you have to have some compile-time dependencies installed to
> be
> > > > able
> > > > > > > > to build the project)
> > > > > > > >
> > > > > > > >
> > > > > > > > Kind regards,
> > > > > > > >
> > > > > > > > Arnout
> > > > > > > >
> > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > > >
> > > > > > > > > > Justin,
> > > > > > > > > >
> > > > > > > > > > There is an issue in the code base that is reasonably
> common
> > > > > among systems
> > > > > > > > > > that operate at low levels and are migrating from Java 8
> > > > > implementation
> > > > > > > > > > wherein they need the rt.jar from java 8 for compilation.
> > > > >  There are
> > > > > > > > > > several ways to solve this problem, Pekko (and I suppose
> > > Akka)
> > > > > require that
> > > > > > > > > > Java 8 be installed so that the build process can find
> the
> > > Java
> > > > > 8 rt.jar.
> > > > > > > > > > I suspect that all that is actually required to get the
> > > current
> > > > > code to
> > > > > > > > > > compile is setting JAVA_8_HOME to point to the directory
> with
> > > > > the rt.jar in
> > > > > > > > > > it.  However, the Java 8 rt.jar is still required.  I
> believe
> > > > > there is a
> > > > > > > > > > ticket to resolve this issue but I may be mistaken about
> > > this.
> > > > > > > > > >
> > > > > > > > > > Claude
> > > > > > > > > >
> > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> > > > > justin@classsoftware.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > Part of the requirement of a release is that someone
> can
> > > > build
> > > > > it on their
> > > > > > > > > > > system, and that ideally should be easy to do. I would
> > > assume
> > > > > by their
> > > > > > > > > > > system than means a single version of the JDK which
> they
> > > use
> > > > > and is
> > > > > > > > > > > supported. I’m not sure there is be a need for someone
> > > > testing
> > > > > the release
> > > > > > > > > > > to compile against multiple JDKs?
> > > > > > > > > > >
> > > > > > > > > > > Kind Regards,
> > > > > > > > > > > Justin
> > > > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@pekko.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> dev-help@pekko.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Arnout Engelen
> > > > > > > > ASF Security Response
> > > > > > > > Committer on Apache Pekko
> > > > > > > > Committer on NixOS
> > > > > > > > Independent Open Source consultant
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Arnout Engelen
> > > > > > ASF Security Response
> > > > > > Committer on Apache Pekko
> > > > > > Committer on NixOS
> > > > > > Independent Open Source consultant
> > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Matthew de Detrich
> > > >
> > > > *Aiven Deutschland GmbH*
> > > >
> > > > Immanuelkirchstraße 26, 10405 Berlin
> > > >
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > >
> > > > *m:* +491603708037
> > > >
> > > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > > >
> > >
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>
>

-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] Building Binary Package

Posted by PJ Fanning <fa...@gmail.com>.
> Q: does the build package the rt.jar in the final binary package?

No. We only produce our own jars that we push to Maven - and these do
not embed any rt.jar classes (shaded or otherwise). We have no other
type of 'binary' package.

> Q: if it does not, what happens if you try to run using Java 11 on a system
> that does not have Java 8 nor the rt.jar installed?

Java compiler produces class files. These have a Java version
associated with them and you can't use a class that is in Java 11
format in a Java 8 runtime.
If you build Pekko with both Java 11 and Java 8 installed, the classes
we will produce in our jars will be in Java 8 class format.

On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich
<ma...@aiven.io.invalid> wrote:
>
> afaik Pekko will just fail to build at all if you use JDK 11 without
> pointing to JDK 8 rt.jar.
>
> This is because Pekko uses sun.misc.unsafe which is only shipping with JDK
> 8 and the
> reason why we have to build on JDK 11 is because there are features
> shipping in JDK 11
> which pekko uses.
>
> On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
> <cl...@aiven.io.invalid> wrote:
>
> > I have a couple of questions:
> >
> > Q: does the build package the rt.jar in the final binary package?
> >
> > Q: if it does not, what happens if you try to run using Java 11 on a system
> > that does not have Java 8 nor the rt.jar installed?
> >
> >
> >
> > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> > <ma...@aiven.io.invalid> wrote:
> >
> > > > We could provide a Docker image with the right sbt and Java setup for
> > > convenience but I think it is fair not to require users to use Docker.
> > >
> > > Considering how much implicit behaviour there is in making a full build
> > > (as Arnout mentioned earlier, you can compile Pekko fine with just JDK 8
> > > just fine however it will be missing post java 8 optional features) I
> > would
> > > prefer
> > > to strongly encourage people to use Docker to do a proper build. As a
> > bare
> > > minimum we can also print a warning in sbt telling users if they are not
> > > going
> > > to be making a complete Pekko build when building on JDK 8 (as stated it
> > > already errors out if you build on JDK 11 without pointing to a JDK
> > > rt.jar).
> > >
> > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <fa...@gmail.com> wrote:
> > >
> > > > Thanks Arnout.
> > > >
> > > > I think if we clearly document what you need to set up to do a build,
> > > > that should be enough. I think it is fair to assume that users can
> > > > install Java and sbt - and that they should be able to install 2
> > > > versions of Java if we are explicit about how to do this and why this
> > > > is the best setup for building Pekko.
> > > >
> > > > We will need to expand what we have in
> > > >
> > > >
> > >
> > https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > > >
> > > > This README file will appear in the source releases.
> > > >
> > > > We could provide a Docker image with the right sbt and Java setup for
> > > > convenience but I think it is fair not to require users to use Docker.
> > > >
> > > >
> > > >
> > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <en...@apache.org>
> > wrote:
> > > > >
> > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <fa...@gmail.com>
> > > wrote:
> > > > > > Ok. It seems the nightly job to publish the snapshot jars uses Java
> > > > > > 11. It does not explicitly install Java 8 but maybe, it's possible
> > > > > > that Java 8 is installed silently,
> > > > >
> > > > >
> > > >
> > >
> > https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > > suggests so, yes.
> > > > >
> > > > > > The nightly build does produce jars that can be used by Java 8.
> > > > > >
> > > > > > So if this nightly build relies on the silent installation of Java
> > 8,
> > > > > > should we add a check in our sbt script that will fail if Java 8 is
> > > > > > not found? This could be useful for users who are building on their
> > > > > > own machines.
> > > > >
> > > > > This check already exists: when I build with jdk11 without having
> > jdk8
> > > > > available the build fails with:
> > > > >
> > > > > [error] A JDK 8 installation was not found, but is required to build
> > > > > Apache Pekko. To manually specify a JDK 8 installation, use the "set
> > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you have no JDK 8
> > > > > installation, target your system JDK with the "set every
> > > > > targetSystemJdk := true" sbt command, but beware resulting artifacts
> > > > > will not work on JDK 8
> > > > >
> > > > > It would be good to also document the JAVA_8_HOME environment
> > variable
> > > > > in this error message. Created
> > > > > https://github.com/apache/incubator-pekko/pull/333 for that.
> > > > >
> > > > >
> > > > > Kind regards,
> > > > >
> > > > > Arnout
> > > > >
> > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <en...@apache.org>
> > > > wrote:
> > > > > > >
> > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <fanningpj@gmail.com
> > >
> > > > wrote:
> > > > > > > > Only building the website docs needs Java 11 (and up).
> > Everything
> > > > else
> > > > > > > > builds with Java 8 (and up).
> > > > > > >
> > > > > > > Yes, though builds created with only Java 8 will miss some
> > optional
> > > > > > > post-Java-8 features/implementations for the "actor-typed",
> > > "stream",
> > > > > > > "remote" and "cluster-sharding" modules - so for a faithful test
> > > > you'd
> > > > > > > ideally build with JDK 9-or-later while having the Java 8 runtime
> > > > > > > available.
> > > > > > >
> > > > > > > (TBH I don't think this is _too_ different from other projects
> > > where
> > > > > > > you have to have some compile-time dependencies installed to be
> > > able
> > > > > > > to build the project)
> > > > > > >
> > > > > > >
> > > > > > > Kind regards,
> > > > > > >
> > > > > > > Arnout
> > > > > > >
> > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > > >
> > > > > > > > > Justin,
> > > > > > > > >
> > > > > > > > > There is an issue in the code base that is reasonably common
> > > > among systems
> > > > > > > > > that operate at low levels and are migrating from Java 8
> > > > implementation
> > > > > > > > > wherein they need the rt.jar from java 8 for compilation.
> > > >  There are
> > > > > > > > > several ways to solve this problem, Pekko (and I suppose
> > Akka)
> > > > require that
> > > > > > > > > Java 8 be installed so that the build process can find the
> > Java
> > > > 8 rt.jar.
> > > > > > > > > I suspect that all that is actually required to get the
> > current
> > > > code to
> > > > > > > > > compile is setting JAVA_8_HOME to point to the directory with
> > > > the rt.jar in
> > > > > > > > > it.  However, the Java 8 rt.jar is still required.  I believe
> > > > there is a
> > > > > > > > > ticket to resolve this issue but I may be mistaken about
> > this.
> > > > > > > > >
> > > > > > > > > Claude
> > > > > > > > >
> > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> > > > justin@classsoftware.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > Part of the requirement of a release is that someone can
> > > build
> > > > it on their
> > > > > > > > > > system, and that ideally should be easy to do. I would
> > assume
> > > > by their
> > > > > > > > > > system than means a single version of the JDK which they
> > use
> > > > and is
> > > > > > > > > > supported. I’m not sure there is be a need for someone
> > > testing
> > > > the release
> > > > > > > > > > to compile against multiple JDKs?
> > > > > > > > > >
> > > > > > > > > > Kind Regards,
> > > > > > > > > > Justin
> > > > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Arnout Engelen
> > > > > > > ASF Security Response
> > > > > > > Committer on Apache Pekko
> > > > > > > Committer on NixOS
> > > > > > > Independent Open Source consultant
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Arnout Engelen
> > > > > ASF Security Response
> > > > > Committer on Apache Pekko
> > > > > Committer on NixOS
> > > > > Independent Open Source consultant
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > >
> > > >
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
afaik Pekko will just fail to build at all if you use JDK 11 without
pointing to JDK 8 rt.jar.

This is because Pekko uses sun.misc.unsafe which is only shipping with JDK
8 and the
reason why we have to build on JDK 11 is because there are features
shipping in JDK 11
which pekko uses.

On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr
<cl...@aiven.io.invalid> wrote:

> I have a couple of questions:
>
> Q: does the build package the rt.jar in the final binary package?
>
> Q: if it does not, what happens if you try to run using Java 11 on a system
> that does not have Java 8 nor the rt.jar installed?
>
>
>
> On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
> <ma...@aiven.io.invalid> wrote:
>
> > > We could provide a Docker image with the right sbt and Java setup for
> > convenience but I think it is fair not to require users to use Docker.
> >
> > Considering how much implicit behaviour there is in making a full build
> > (as Arnout mentioned earlier, you can compile Pekko fine with just JDK 8
> > just fine however it will be missing post java 8 optional features) I
> would
> > prefer
> > to strongly encourage people to use Docker to do a proper build. As a
> bare
> > minimum we can also print a warning in sbt telling users if they are not
> > going
> > to be making a complete Pekko build when building on JDK 8 (as stated it
> > already errors out if you build on JDK 11 without pointing to a JDK
> > rt.jar).
> >
> > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <fa...@gmail.com> wrote:
> >
> > > Thanks Arnout.
> > >
> > > I think if we clearly document what you need to set up to do a build,
> > > that should be enough. I think it is fair to assume that users can
> > > install Java and sbt - and that they should be able to install 2
> > > versions of Java if we are explicit about how to do this and why this
> > > is the best setup for building Pekko.
> > >
> > > We will need to expand what we have in
> > >
> > >
> >
> https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> > >
> > > This README file will appear in the source releases.
> > >
> > > We could provide a Docker image with the right sbt and Java setup for
> > > convenience but I think it is fair not to require users to use Docker.
> > >
> > >
> > >
> > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <en...@apache.org>
> wrote:
> > > >
> > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <fa...@gmail.com>
> > wrote:
> > > > > Ok. It seems the nightly job to publish the snapshot jars uses Java
> > > > > 11. It does not explicitly install Java 8 but maybe, it's possible
> > > > > that Java 8 is installed silently,
> > > >
> > > >
> > >
> >
> https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > > suggests so, yes.
> > > >
> > > > > The nightly build does produce jars that can be used by Java 8.
> > > > >
> > > > > So if this nightly build relies on the silent installation of Java
> 8,
> > > > > should we add a check in our sbt script that will fail if Java 8 is
> > > > > not found? This could be useful for users who are building on their
> > > > > own machines.
> > > >
> > > > This check already exists: when I build with jdk11 without having
> jdk8
> > > > available the build fails with:
> > > >
> > > > [error] A JDK 8 installation was not found, but is required to build
> > > > Apache Pekko. To manually specify a JDK 8 installation, use the "set
> > > > every jdk8home := \"/path/to/jdk\" sbt command. If you have no JDK 8
> > > > installation, target your system JDK with the "set every
> > > > targetSystemJdk := true" sbt command, but beware resulting artifacts
> > > > will not work on JDK 8
> > > >
> > > > It would be good to also document the JAVA_8_HOME environment
> variable
> > > > in this error message. Created
> > > > https://github.com/apache/incubator-pekko/pull/333 for that.
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Arnout
> > > >
> > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <en...@apache.org>
> > > wrote:
> > > > > >
> > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <fanningpj@gmail.com
> >
> > > wrote:
> > > > > > > Only building the website docs needs Java 11 (and up).
> Everything
> > > else
> > > > > > > builds with Java 8 (and up).
> > > > > >
> > > > > > Yes, though builds created with only Java 8 will miss some
> optional
> > > > > > post-Java-8 features/implementations for the "actor-typed",
> > "stream",
> > > > > > "remote" and "cluster-sharding" modules - so for a faithful test
> > > you'd
> > > > > > ideally build with JDK 9-or-later while having the Java 8 runtime
> > > > > > available.
> > > > > >
> > > > > > (TBH I don't think this is _too_ different from other projects
> > where
> > > > > > you have to have some compile-time dependencies installed to be
> > able
> > > > > > to build the project)
> > > > > >
> > > > > >
> > > > > > Kind regards,
> > > > > >
> > > > > > Arnout
> > > > > >
> > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > > >
> > > > > > > > Justin,
> > > > > > > >
> > > > > > > > There is an issue in the code base that is reasonably common
> > > among systems
> > > > > > > > that operate at low levels and are migrating from Java 8
> > > implementation
> > > > > > > > wherein they need the rt.jar from java 8 for compilation.
> > >  There are
> > > > > > > > several ways to solve this problem, Pekko (and I suppose
> Akka)
> > > require that
> > > > > > > > Java 8 be installed so that the build process can find the
> Java
> > > 8 rt.jar.
> > > > > > > > I suspect that all that is actually required to get the
> current
> > > code to
> > > > > > > > compile is setting JAVA_8_HOME to point to the directory with
> > > the rt.jar in
> > > > > > > > it.  However, the Java 8 rt.jar is still required.  I believe
> > > there is a
> > > > > > > > ticket to resolve this issue but I may be mistaken about
> this.
> > > > > > > >
> > > > > > > > Claude
> > > > > > > >
> > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> > > justin@classsoftware.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > Part of the requirement of a release is that someone can
> > build
> > > it on their
> > > > > > > > > system, and that ideally should be easy to do. I would
> assume
> > > by their
> > > > > > > > > system than means a single version of the JDK which they
> use
> > > and is
> > > > > > > > > supported. I’m not sure there is be a need for someone
> > testing
> > > the release
> > > > > > > > > to compile against multiple JDKs?
> > > > > > > > >
> > > > > > > > > Kind Regards,
> > > > > > > > > Justin
> > > > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Arnout Engelen
> > > > > > ASF Security Response
> > > > > > Committer on Apache Pekko
> > > > > > Committer on NixOS
> > > > > > Independent Open Source consultant
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > >
> > > >
> > > >
> > > > --
> > > > Arnout Engelen
> > > > ASF Security Response
> > > > Committer on Apache Pekko
> > > > Committer on NixOS
> > > > Independent Open Source consultant
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> > >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] Building Binary Package

Posted by "Claude Warren, Jr" <cl...@aiven.io.INVALID>.
I have a couple of questions:

Q: does the build package the rt.jar in the final binary package?

Q: if it does not, what happens if you try to run using Java 11 on a system
that does not have Java 8 nor the rt.jar installed?



On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich
<ma...@aiven.io.invalid> wrote:

> > We could provide a Docker image with the right sbt and Java setup for
> convenience but I think it is fair not to require users to use Docker.
>
> Considering how much implicit behaviour there is in making a full build
> (as Arnout mentioned earlier, you can compile Pekko fine with just JDK 8
> just fine however it will be missing post java 8 optional features) I would
> prefer
> to strongly encourage people to use Docker to do a proper build. As a bare
> minimum we can also print a warning in sbt telling users if they are not
> going
> to be making a complete Pekko build when building on JDK 8 (as stated it
> already errors out if you build on JDK 11 without pointing to a JDK
> rt.jar).
>
> On Wed, May 24, 2023 at 11:22 AM PJ Fanning <fa...@gmail.com> wrote:
>
> > Thanks Arnout.
> >
> > I think if we clearly document what you need to set up to do a build,
> > that should be enough. I think it is fair to assume that users can
> > install Java and sbt - and that they should be able to install 2
> > versions of Java if we are explicit about how to do this and why this
> > is the best setup for building Pekko.
> >
> > We will need to expand what we have in
> >
> >
> https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
> >
> > This README file will appear in the source releases.
> >
> > We could provide a Docker image with the right sbt and Java setup for
> > convenience but I think it is fair not to require users to use Docker.
> >
> >
> >
> > On Wed, 24 May 2023 at 10:09, Arnout Engelen <en...@apache.org> wrote:
> > >
> > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <fa...@gmail.com>
> wrote:
> > > > Ok. It seems the nightly job to publish the snapshot jars uses Java
> > > > 11. It does not explicitly install Java 8 but maybe, it's possible
> > > > that Java 8 is installed silently,
> > >
> > >
> >
> https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > > suggests so, yes.
> > >
> > > > The nightly build does produce jars that can be used by Java 8.
> > > >
> > > > So if this nightly build relies on the silent installation of Java 8,
> > > > should we add a check in our sbt script that will fail if Java 8 is
> > > > not found? This could be useful for users who are building on their
> > > > own machines.
> > >
> > > This check already exists: when I build with jdk11 without having jdk8
> > > available the build fails with:
> > >
> > > [error] A JDK 8 installation was not found, but is required to build
> > > Apache Pekko. To manually specify a JDK 8 installation, use the "set
> > > every jdk8home := \"/path/to/jdk\" sbt command. If you have no JDK 8
> > > installation, target your system JDK with the "set every
> > > targetSystemJdk := true" sbt command, but beware resulting artifacts
> > > will not work on JDK 8
> > >
> > > It would be good to also document the JAVA_8_HOME environment variable
> > > in this error message. Created
> > > https://github.com/apache/incubator-pekko/pull/333 for that.
> > >
> > >
> > > Kind regards,
> > >
> > > Arnout
> > >
> > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <en...@apache.org>
> > wrote:
> > > > >
> > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <fa...@gmail.com>
> > wrote:
> > > > > > Only building the website docs needs Java 11 (and up). Everything
> > else
> > > > > > builds with Java 8 (and up).
> > > > >
> > > > > Yes, though builds created with only Java 8 will miss some optional
> > > > > post-Java-8 features/implementations for the "actor-typed",
> "stream",
> > > > > "remote" and "cluster-sharding" modules - so for a faithful test
> > you'd
> > > > > ideally build with JDK 9-or-later while having the Java 8 runtime
> > > > > available.
> > > > >
> > > > > (TBH I don't think this is _too_ different from other projects
> where
> > > > > you have to have some compile-time dependencies installed to be
> able
> > > > > to build the project)
> > > > >
> > > > >
> > > > > Kind regards,
> > > > >
> > > > > Arnout
> > > > >
> > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > > <cl...@aiven.io.invalid> wrote:
> > > > > > >
> > > > > > > Justin,
> > > > > > >
> > > > > > > There is an issue in the code base that is reasonably common
> > among systems
> > > > > > > that operate at low levels and are migrating from Java 8
> > implementation
> > > > > > > wherein they need the rt.jar from java 8 for compilation.
> >  There are
> > > > > > > several ways to solve this problem, Pekko (and I suppose Akka)
> > require that
> > > > > > > Java 8 be installed so that the build process can find the Java
> > 8 rt.jar.
> > > > > > > I suspect that all that is actually required to get the current
> > code to
> > > > > > > compile is setting JAVA_8_HOME to point to the directory with
> > the rt.jar in
> > > > > > > it.  However, the Java 8 rt.jar is still required.  I believe
> > there is a
> > > > > > > ticket to resolve this issue but I may be mistaken about this.
> > > > > > >
> > > > > > > Claude
> > > > > > >
> > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> > justin@classsoftware.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > Part of the requirement of a release is that someone can
> build
> > it on their
> > > > > > > > system, and that ideally should be easy to do. I would assume
> > by their
> > > > > > > > system than means a single version of the JDK which they use
> > and is
> > > > > > > > supported. I’m not sure there is be a need for someone
> testing
> > the release
> > > > > > > > to compile against multiple JDKs?
> > > > > > > >
> > > > > > > > Kind Regards,
> > > > > > > > Justin
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Arnout Engelen
> > > > > ASF Security Response
> > > > > Committer on Apache Pekko
> > > > > Committer on NixOS
> > > > > Independent Open Source consultant
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > >
> > >
> > >
> > > --
> > > Arnout Engelen
> > > ASF Security Response
> > > Committer on Apache Pekko
> > > Committer on NixOS
> > > Independent Open Source consultant
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
> >
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>

Re: [DISCUSS] Building Binary Package

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
> We could provide a Docker image with the right sbt and Java setup for
convenience but I think it is fair not to require users to use Docker.

Considering how much implicit behaviour there is in making a full build
(as Arnout mentioned earlier, you can compile Pekko fine with just JDK 8
just fine however it will be missing post java 8 optional features) I would
prefer
to strongly encourage people to use Docker to do a proper build. As a bare
minimum we can also print a warning in sbt telling users if they are not
going
to be making a complete Pekko build when building on JDK 8 (as stated it
already errors out if you build on JDK 11 without pointing to a JDK rt.jar).

On Wed, May 24, 2023 at 11:22 AM PJ Fanning <fa...@gmail.com> wrote:

> Thanks Arnout.
>
> I think if we clearly document what you need to set up to do a build,
> that should be enough. I think it is fair to assume that users can
> install Java and sbt - and that they should be able to install 2
> versions of Java if we are explicit about how to do this and why this
> is the best setup for building Pekko.
>
> We will need to expand what we have in
>
> https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source
>
> This README file will appear in the source releases.
>
> We could provide a Docker image with the right sbt and Java setup for
> convenience but I think it is fair not to require users to use Docker.
>
>
>
> On Wed, 24 May 2023 at 10:09, Arnout Engelen <en...@apache.org> wrote:
> >
> > On Wed, May 24, 2023 at 10:57 AM PJ Fanning <fa...@gmail.com> wrote:
> > > Ok. It seems the nightly job to publish the snapshot jars uses Java
> > > 11. It does not explicitly install Java 8 but maybe, it's possible
> > > that Java 8 is installed silently,
> >
> >
> https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> > suggests so, yes.
> >
> > > The nightly build does produce jars that can be used by Java 8.
> > >
> > > So if this nightly build relies on the silent installation of Java 8,
> > > should we add a check in our sbt script that will fail if Java 8 is
> > > not found? This could be useful for users who are building on their
> > > own machines.
> >
> > This check already exists: when I build with jdk11 without having jdk8
> > available the build fails with:
> >
> > [error] A JDK 8 installation was not found, but is required to build
> > Apache Pekko. To manually specify a JDK 8 installation, use the "set
> > every jdk8home := \"/path/to/jdk\" sbt command. If you have no JDK 8
> > installation, target your system JDK with the "set every
> > targetSystemJdk := true" sbt command, but beware resulting artifacts
> > will not work on JDK 8
> >
> > It would be good to also document the JAVA_8_HOME environment variable
> > in this error message. Created
> > https://github.com/apache/incubator-pekko/pull/333 for that.
> >
> >
> > Kind regards,
> >
> > Arnout
> >
> > > On Wed, 24 May 2023 at 09:38, Arnout Engelen <en...@apache.org>
> wrote:
> > > >
> > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <fa...@gmail.com>
> wrote:
> > > > > Only building the website docs needs Java 11 (and up). Everything
> else
> > > > > builds with Java 8 (and up).
> > > >
> > > > Yes, though builds created with only Java 8 will miss some optional
> > > > post-Java-8 features/implementations for the "actor-typed", "stream",
> > > > "remote" and "cluster-sharding" modules - so for a faithful test
> you'd
> > > > ideally build with JDK 9-or-later while having the Java 8 runtime
> > > > available.
> > > >
> > > > (TBH I don't think this is _too_ different from other projects where
> > > > you have to have some compile-time dependencies installed to be able
> > > > to build the project)
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Arnout
> > > >
> > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > > <cl...@aiven.io.invalid> wrote:
> > > > > >
> > > > > > Justin,
> > > > > >
> > > > > > There is an issue in the code base that is reasonably common
> among systems
> > > > > > that operate at low levels and are migrating from Java 8
> implementation
> > > > > > wherein they need the rt.jar from java 8 for compilation.
>  There are
> > > > > > several ways to solve this problem, Pekko (and I suppose Akka)
> require that
> > > > > > Java 8 be installed so that the build process can find the Java
> 8 rt.jar.
> > > > > > I suspect that all that is actually required to get the current
> code to
> > > > > > compile is setting JAVA_8_HOME to point to the directory with
> the rt.jar in
> > > > > > it.  However, the Java 8 rt.jar is still required.  I believe
> there is a
> > > > > > ticket to resolve this issue but I may be mistaken about this.
> > > > > >
> > > > > > Claude
> > > > > >
> > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <
> justin@classsoftware.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Part of the requirement of a release is that someone can build
> it on their
> > > > > > > system, and that ideally should be easy to do. I would assume
> by their
> > > > > > > system than means a single version of the JDK which they use
> and is
> > > > > > > supported. I’m not sure there is be a need for someone testing
> the release
> > > > > > > to compile against multiple JDKs?
> > > > > > >
> > > > > > > Kind Regards,
> > > > > > > Justin
> > > > > > >
> ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > >
> > > >
> > > >
> > > > --
> > > > Arnout Engelen
> > > > ASF Security Response
> > > > Committer on Apache Pekko
> > > > Committer on NixOS
> > > > Independent Open Source consultant
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> >
> >
> > --
> > Arnout Engelen
> > ASF Security Response
> > Committer on Apache Pekko
> > Committer on NixOS
> > Independent Open Source consultant
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>
>

-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] Building Binary Package

Posted by PJ Fanning <fa...@gmail.com>.
Thanks Arnout.

I think if we clearly document what you need to set up to do a build,
that should be enough. I think it is fair to assume that users can
install Java and sbt - and that they should be able to install 2
versions of Java if we are explicit about how to do this and why this
is the best setup for building Pekko.

We will need to expand what we have in
https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source

This README file will appear in the source releases.

We could provide a Docker image with the right sbt and Java setup for
convenience but I think it is fair not to require users to use Docker.



On Wed, 24 May 2023 at 10:09, Arnout Engelen <en...@apache.org> wrote:
>
> On Wed, May 24, 2023 at 10:57 AM PJ Fanning <fa...@gmail.com> wrote:
> > Ok. It seems the nightly job to publish the snapshot jars uses Java
> > 11. It does not explicitly install Java 8 but maybe, it's possible
> > that Java 8 is installed silently,
>
> https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
> suggests so, yes.
>
> > The nightly build does produce jars that can be used by Java 8.
> >
> > So if this nightly build relies on the silent installation of Java 8,
> > should we add a check in our sbt script that will fail if Java 8 is
> > not found? This could be useful for users who are building on their
> > own machines.
>
> This check already exists: when I build with jdk11 without having jdk8
> available the build fails with:
>
> [error] A JDK 8 installation was not found, but is required to build
> Apache Pekko. To manually specify a JDK 8 installation, use the "set
> every jdk8home := \"/path/to/jdk\" sbt command. If you have no JDK 8
> installation, target your system JDK with the "set every
> targetSystemJdk := true" sbt command, but beware resulting artifacts
> will not work on JDK 8
>
> It would be good to also document the JAVA_8_HOME environment variable
> in this error message. Created
> https://github.com/apache/incubator-pekko/pull/333 for that.
>
>
> Kind regards,
>
> Arnout
>
> > On Wed, 24 May 2023 at 09:38, Arnout Engelen <en...@apache.org> wrote:
> > >
> > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <fa...@gmail.com> wrote:
> > > > Only building the website docs needs Java 11 (and up). Everything else
> > > > builds with Java 8 (and up).
> > >
> > > Yes, though builds created with only Java 8 will miss some optional
> > > post-Java-8 features/implementations for the "actor-typed", "stream",
> > > "remote" and "cluster-sharding" modules - so for a faithful test you'd
> > > ideally build with JDK 9-or-later while having the Java 8 runtime
> > > available.
> > >
> > > (TBH I don't think this is _too_ different from other projects where
> > > you have to have some compile-time dependencies installed to be able
> > > to build the project)
> > >
> > >
> > > Kind regards,
> > >
> > > Arnout
> > >
> > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > > <cl...@aiven.io.invalid> wrote:
> > > > >
> > > > > Justin,
> > > > >
> > > > > There is an issue in the code base that is reasonably common among systems
> > > > > that operate at low levels and are migrating from Java 8 implementation
> > > > > wherein they need the rt.jar from java 8 for compilation.   There are
> > > > > several ways to solve this problem, Pekko (and I suppose Akka) require that
> > > > > Java 8 be installed so that the build process can find the Java 8 rt.jar.
> > > > > I suspect that all that is actually required to get the current code to
> > > > > compile is setting JAVA_8_HOME to point to the directory with the rt.jar in
> > > > > it.  However, the Java 8 rt.jar is still required.  I believe there is a
> > > > > ticket to resolve this issue but I may be mistaken about this.
> > > > >
> > > > > Claude
> > > > >
> > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <ju...@classsoftware.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Part of the requirement of a release is that someone can build it on their
> > > > > > system, and that ideally should be easy to do. I would assume by their
> > > > > > system than means a single version of the JDK which they use and is
> > > > > > supported. I’m not sure there is be a need for someone testing the release
> > > > > > to compile against multiple JDKs?
> > > > > >
> > > > > > Kind Regards,
> > > > > > Justin
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > > >
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > >
> > >
> > >
> > > --
> > > Arnout Engelen
> > > ASF Security Response
> > > Committer on Apache Pekko
> > > Committer on NixOS
> > > Independent Open Source consultant
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
>
>
> --
> Arnout Engelen
> ASF Security Response
> Committer on Apache Pekko
> Committer on NixOS
> Independent Open Source consultant
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by Arnout Engelen <en...@apache.org>.
On Wed, May 24, 2023 at 10:57 AM PJ Fanning <fa...@gmail.com> wrote:
> Ok. It seems the nightly job to publish the snapshot jars uses Java
> 11. It does not explicitly install Java 8 but maybe, it's possible
> that Java 8 is installed silently,

https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md
suggests so, yes.

> The nightly build does produce jars that can be used by Java 8.
>
> So if this nightly build relies on the silent installation of Java 8,
> should we add a check in our sbt script that will fail if Java 8 is
> not found? This could be useful for users who are building on their
> own machines.

This check already exists: when I build with jdk11 without having jdk8
available the build fails with:

[error] A JDK 8 installation was not found, but is required to build
Apache Pekko. To manually specify a JDK 8 installation, use the "set
every jdk8home := \"/path/to/jdk\" sbt command. If you have no JDK 8
installation, target your system JDK with the "set every
targetSystemJdk := true" sbt command, but beware resulting artifacts
will not work on JDK 8

It would be good to also document the JAVA_8_HOME environment variable
in this error message. Created
https://github.com/apache/incubator-pekko/pull/333 for that.


Kind regards,

Arnout

> On Wed, 24 May 2023 at 09:38, Arnout Engelen <en...@apache.org> wrote:
> >
> > On Wed, May 24, 2023 at 10:12 AM PJ Fanning <fa...@gmail.com> wrote:
> > > Only building the website docs needs Java 11 (and up). Everything else
> > > builds with Java 8 (and up).
> >
> > Yes, though builds created with only Java 8 will miss some optional
> > post-Java-8 features/implementations for the "actor-typed", "stream",
> > "remote" and "cluster-sharding" modules - so for a faithful test you'd
> > ideally build with JDK 9-or-later while having the Java 8 runtime
> > available.
> >
> > (TBH I don't think this is _too_ different from other projects where
> > you have to have some compile-time dependencies installed to be able
> > to build the project)
> >
> >
> > Kind regards,
> >
> > Arnout
> >
> > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > > <cl...@aiven.io.invalid> wrote:
> > > >
> > > > Justin,
> > > >
> > > > There is an issue in the code base that is reasonably common among systems
> > > > that operate at low levels and are migrating from Java 8 implementation
> > > > wherein they need the rt.jar from java 8 for compilation.   There are
> > > > several ways to solve this problem, Pekko (and I suppose Akka) require that
> > > > Java 8 be installed so that the build process can find the Java 8 rt.jar.
> > > > I suspect that all that is actually required to get the current code to
> > > > compile is setting JAVA_8_HOME to point to the directory with the rt.jar in
> > > > it.  However, the Java 8 rt.jar is still required.  I believe there is a
> > > > ticket to resolve this issue but I may be mistaken about this.
> > > >
> > > > Claude
> > > >
> > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <ju...@classsoftware.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Part of the requirement of a release is that someone can build it on their
> > > > > system, and that ideally should be easy to do. I would assume by their
> > > > > system than means a single version of the JDK which they use and is
> > > > > supported. I’m not sure there is be a need for someone testing the release
> > > > > to compile against multiple JDKs?
> > > > >
> > > > > Kind Regards,
> > > > > Justin
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> >
> >
> > --
> > Arnout Engelen
> > ASF Security Response
> > Committer on Apache Pekko
> > Committer on NixOS
> > Independent Open Source consultant
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>


-- 
Arnout Engelen
ASF Security Response
Committer on Apache Pekko
Committer on NixOS
Independent Open Source consultant

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by PJ Fanning <fa...@gmail.com>.
Ok. It seems the nightly job to publish the snapshot jars uses Java
11. It does not explicitly install Java 8 but maybe, it's possible
that Java 8 is installed silently,

The nightly build does produce jars that can be used by Java 8.

So if this nightly build relies on the silent installation of Java 8,
should we add a check in our sbt script that will fail if Java 8 is
not found? This could be useful for users who are building on their
own machines.


On Wed, 24 May 2023 at 09:38, Arnout Engelen <en...@apache.org> wrote:
>
> On Wed, May 24, 2023 at 10:12 AM PJ Fanning <fa...@gmail.com> wrote:
> > Only building the website docs needs Java 11 (and up). Everything else
> > builds with Java 8 (and up).
>
> Yes, though builds created with only Java 8 will miss some optional
> post-Java-8 features/implementations for the "actor-typed", "stream",
> "remote" and "cluster-sharding" modules - so for a faithful test you'd
> ideally build with JDK 9-or-later while having the Java 8 runtime
> available.
>
> (TBH I don't think this is _too_ different from other projects where
> you have to have some compile-time dependencies installed to be able
> to build the project)
>
>
> Kind regards,
>
> Arnout
>
> > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> > <cl...@aiven.io.invalid> wrote:
> > >
> > > Justin,
> > >
> > > There is an issue in the code base that is reasonably common among systems
> > > that operate at low levels and are migrating from Java 8 implementation
> > > wherein they need the rt.jar from java 8 for compilation.   There are
> > > several ways to solve this problem, Pekko (and I suppose Akka) require that
> > > Java 8 be installed so that the build process can find the Java 8 rt.jar.
> > > I suspect that all that is actually required to get the current code to
> > > compile is setting JAVA_8_HOME to point to the directory with the rt.jar in
> > > it.  However, the Java 8 rt.jar is still required.  I believe there is a
> > > ticket to resolve this issue but I may be mistaken about this.
> > >
> > > Claude
> > >
> > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <ju...@classsoftware.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Part of the requirement of a release is that someone can build it on their
> > > > system, and that ideally should be easy to do. I would assume by their
> > > > system than means a single version of the JDK which they use and is
> > > > supported. I’m not sure there is be a need for someone testing the release
> > > > to compile against multiple JDKs?
> > > >
> > > > Kind Regards,
> > > > Justin
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
>
>
> --
> Arnout Engelen
> ASF Security Response
> Committer on Apache Pekko
> Committer on NixOS
> Independent Open Source consultant
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by Arnout Engelen <en...@apache.org>.
On Wed, May 24, 2023 at 10:12 AM PJ Fanning <fa...@gmail.com> wrote:
> Only building the website docs needs Java 11 (and up). Everything else
> builds with Java 8 (and up).

Yes, though builds created with only Java 8 will miss some optional
post-Java-8 features/implementations for the "actor-typed", "stream",
"remote" and "cluster-sharding" modules - so for a faithful test you'd
ideally build with JDK 9-or-later while having the Java 8 runtime
available.

(TBH I don't think this is _too_ different from other projects where
you have to have some compile-time dependencies installed to be able
to build the project)


Kind regards,

Arnout

> On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
> <cl...@aiven.io.invalid> wrote:
> >
> > Justin,
> >
> > There is an issue in the code base that is reasonably common among systems
> > that operate at low levels and are migrating from Java 8 implementation
> > wherein they need the rt.jar from java 8 for compilation.   There are
> > several ways to solve this problem, Pekko (and I suppose Akka) require that
> > Java 8 be installed so that the build process can find the Java 8 rt.jar.
> > I suspect that all that is actually required to get the current code to
> > compile is setting JAVA_8_HOME to point to the directory with the rt.jar in
> > it.  However, the Java 8 rt.jar is still required.  I believe there is a
> > ticket to resolve this issue but I may be mistaken about this.
> >
> > Claude
> >
> > On Wed, May 24, 2023 at 7:38 AM Justin Mclean <ju...@classsoftware.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Part of the requirement of a release is that someone can build it on their
> > > system, and that ideally should be easy to do. I would assume by their
> > > system than means a single version of the JDK which they use and is
> > > supported. I’m not sure there is be a need for someone testing the release
> > > to compile against multiple JDKs?
> > >
> > > Kind Regards,
> > > Justin
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>


-- 
Arnout Engelen
ASF Security Response
Committer on Apache Pekko
Committer on NixOS
Independent Open Source consultant

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by PJ Fanning <fa...@gmail.com>.
Only building the website docs needs Java 11 (and up). Everything else
builds with Java 8 (and up).

On Wed, 24 May 2023 at 08:05, Claude Warren, Jr
<cl...@aiven.io.invalid> wrote:
>
> Justin,
>
> There is an issue in the code base that is reasonably common among systems
> that operate at low levels and are migrating from Java 8 implementation
> wherein they need the rt.jar from java 8 for compilation.   There are
> several ways to solve this problem, Pekko (and I suppose Akka) require that
> Java 8 be installed so that the build process can find the Java 8 rt.jar.
> I suspect that all that is actually required to get the current code to
> compile is setting JAVA_8_HOME to point to the directory with the rt.jar in
> it.  However, the Java 8 rt.jar is still required.  I believe there is a
> ticket to resolve this issue but I may be mistaken about this.
>
> Claude
>
> On Wed, May 24, 2023 at 7:38 AM Justin Mclean <ju...@classsoftware.com>
> wrote:
>
> > Hi,
> >
> > Part of the requirement of a release is that someone can build it on their
> > system, and that ideally should be easy to do. I would assume by their
> > system than means a single version of the JDK which they use and is
> > supported. I’m not sure there is be a need for someone testing the release
> > to compile against multiple JDKs?
> >
> > Kind Regards,
> > Justin
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
> >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by "Claude Warren, Jr" <cl...@aiven.io.INVALID>.
Justin,

There is an issue in the code base that is reasonably common among systems
that operate at low levels and are migrating from Java 8 implementation
wherein they need the rt.jar from java 8 for compilation.   There are
several ways to solve this problem, Pekko (and I suppose Akka) require that
Java 8 be installed so that the build process can find the Java 8 rt.jar.
I suspect that all that is actually required to get the current code to
compile is setting JAVA_8_HOME to point to the directory with the rt.jar in
it.  However, the Java 8 rt.jar is still required.  I believe there is a
ticket to resolve this issue but I may be mistaken about this.

Claude

On Wed, May 24, 2023 at 7:38 AM Justin Mclean <ju...@classsoftware.com>
wrote:

> Hi,
>
> Part of the requirement of a release is that someone can build it on their
> system, and that ideally should be easy to do. I would assume by their
> system than means a single version of the JDK which they use and is
> supported. I’m not sure there is be a need for someone testing the release
> to compile against multiple JDKs?
>
> Kind Regards,
> Justin
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
>
>

Re: [DISCUSS] Building Binary Package

Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,

Part of the requirement of a release is that someone can build it on their system, and that ideally should be easy to do. I would assume by their system than means a single version of the JDK which they use and is supported. I’m not sure there is be a need for someone testing the release to compile against multiple JDKs?

Kind Regards,
Justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] Building Binary Package

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
> But more to the point here, is there a Dockerfile to build the Docker
image?

There is an in progress PR at
https://github.com/apache/incubator-pekko/pull/188

>  Can we spell out how to configure a standard linux system to build
Pekko without the docker image?

Not entirely sure what you mean by "standard linux system to build".
Linux's have
different distributions/package managers and most developers install things
like
OpenJDK via a package manager. While that in of itself is not problematic,
the issue
here is the "multiple installs" part (or to be more accurate you need
OpenJDK 11
to be installed and you have to download a distribution of OpenJDK 8 to get
the
rt.jar so that Pekko can compile sources that refer to sun.misc.unsafe and
feed that
into sbt). There are other solutions like sdkman which can handle multiple
JDK
installs, but the obvious issue here is the explosion of installation
instructions (
also this presumes that developers primary machines are Linux).

The one big advantage that the Docker build has (aside from it just working
on any major OS) is that it uses the same docker image that github actions
does
(at least this appears to be the case from a cursory glance).

I will try and spend some time getting
https://github.com/apache/incubator-pekko/pull/188
finished and merged this week.

On Tue, May 23, 2023 at 3:00 PM Claude Warren, Jr
<cl...@aiven.io.invalid> wrote:

> >
> > Also I would like to point out/remind that even though making a source
> > package release is important
> > for Apache, it's useless for Pekko's users. It's been stated many times
> on
> > the mailing list that the people
> > actually using Pekko ultimately only care about the maven binary jars
> (i.e.
> > 1.0.0 release) and that a lot
> > of potential contributors are waiting for that 1.0.0 maven jar binary
> > release. The ironic thing here is that
> > we already have a method of creating proper artifacts, it's just done by
> > github actions CI which doesn't
> > adhere to Apache's processes for a release and that is an example of
> what's
> > making it more complicated
> > (otherwise we could have been pushing RC's weeks/months ago).
> >
> >
> I want to point out that most of the projects I work on are libraries, and
> as you say most users just want the binary packages.
>
> But more to the point here, is there a Dockerfile to build the Docker
> image?   Can we spell out how to configure a standard linux system to build
> Pekko without the docker image?
>
> Claude
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] Building Binary Package

Posted by "Claude Warren, Jr" <cl...@aiven.io.INVALID>.
>
> Also I would like to point out/remind that even though making a source
> package release is important
> for Apache, it's useless for Pekko's users. It's been stated many times on
> the mailing list that the people
> actually using Pekko ultimately only care about the maven binary jars (i.e.
> 1.0.0 release) and that a lot
> of potential contributors are waiting for that 1.0.0 maven jar binary
> release. The ironic thing here is that
> we already have a method of creating proper artifacts, it's just done by
> github actions CI which doesn't
> adhere to Apache's processes for a release and that is an example of what's
> making it more complicated
> (otherwise we could have been pushing RC's weeks/months ago).
>
>
I want to point out that most of the projects I work on are libraries, and
as you say most users just want the binary packages.

But more to the point here, is there a Dockerfile to build the Docker
image?   Can we spell out how to configure a standard linux system to build
Pekko without the docker image?

Claude

Re: [DISCUSS] RC1 build

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
> No one here is suggesting any kind of automation, this was already clear
months ago.

I just checked the github milestone/project and I realized there are still
a lot of automation tickets there.
This should have been removed some time ago, so I can see how this caused
confusion. Since Claude
has already created a vote on this we can manage it there.

On Tue, May 23, 2023 at 2:31 PM Matthew Benedict de Detrich <
matthew.dedetrich@aiven.io> wrote:

> > There are a number of issues that have been identified with respect to
> how
> to do the release process.  Things like a full automation of the process is
> putting the cart before the horse.
>
> No one here is suggesting any kind of automation, this was already clear
> months ago.
>
> > What we want to see here is not perfection, we want to see the process
> being built.
>
> I don't think we are even at a stage where perfection is even a thought,
> the current discussions we
> have are just the basics of making an artifact. For example unless you
> specifically setup your host
> system to have multiple JDK's installed at once, Pekko will not even build
> because it relies on
> java.misc.unsafe. This is why there is a Docker image, which while not
> strictly required makes it
> easier/simpler and even more importantly for other committers to verify
> the release.
>
> Also I would like to point out/remind that even though making a source
> package release is important
> for Apache, it's useless for Pekko's users. It's been stated many times on
> the mailing list that the people
> actually using Pekko ultimately only care about the maven binary jars
> (i.e. 1.0.0 release) and that a lot
> of potential contributors are waiting for that 1.0.0 maven jar binary
> release. The ironic thing here is that
> we already have a method of creating proper artifacts, it's just done by
> github actions CI which doesn't
> adhere to Apache's processes for a release and that is an example of
> what's making it more complicated
> (otherwise we could have been pushing RC's weeks/months ago).
>
> On another note, while it's definitely true that we should strongly
> encourage getting a release out sooner
> than later, in my view arbitrary checkboxing of some process doesn't do
> anyone any favors (whether it's
> Apache, IPMC or Pekko users).
>
>
>
> On Tue, May 23, 2023 at 1:55 PM Claude Warren, Jr
> <cl...@aiven.io.invalid> wrote:
>
>> If the milestone is placed in the snapshot area it is not a release.  I
>> strongly recommend that Pekko seriously consider making the milestone a
>> release and plan to work through a number of release candidates to get the
>> process ironed out.
>>
>> There are a number of issues that have been identified with respect to how
>> to do the release process.  Things like a full automation of the process
>> is
>> putting the cart before the horse.  You can't know what to automate until
>> you know what to do.  So let's agree what goes into RC1, get those issues
>> resolved and document a step-by-step release for RC1 as it is done.  When
>> complete and reviewed we will have issues that will require another RC.
>> Now you can probably automate some steps.  Others will require human
>> intervention.
>>
>> What we want to see here is not perfection, we want to see the process
>> being built.  We want to bring first hand knowledge of the tricks and
>> pitfalls of the release process to a wider community.  We want to get some
>> packaging out for review by the IPMC.
>>
>> Claude
>>
>>
>>
>> On Tue, May 23, 2023 at 11:56 AM Matthew Benedict de Detrich
>> <ma...@aiven.io.invalid> wrote:
>>
>> > > (i.e. first release after a hard work)
>> >
>> > This is meant to say "(i.e. first release after a hard fork)"
>> >
>> > On Tue, May 23, 2023 at 12:53 PM Matthew Benedict de Detrich <
>> > matthew.dedetrich@aiven.io> wrote:
>> >
>> > > > I think that it is time to start to take a hard look at a release
>> > > candidate.  I do not expect the release candidate to succeed, however,
>> > > there is a lot to be learned in the failure.
>> > >
>> > > Our current strategy is to release a MILESTONE which should hopefully
>> > > happen this week. This is because doing a RC is more problematic due
>> to
>> > how
>> > > difficult it is to build Pekko outside of CI (with a MILESTONE we can
>> > > release it under Apache Snapshots Repo)
>> > >
>> > > > I think we should start by determining if there are items that are
>> not
>> > > required for the release and items that are missing.  I am opening two
>> > > issues today that I will add to the milestone.  They are the addition
>> of
>> > > the LICENSE and NOTICE files to the snapshots, under the assumption
>> that
>> > > doing so will also add them to the release.
>> > >
>> > > FYI this has already been done for all Pekko modules, the core logic
>> of
>> > > doing so is abstracted in
>> > > https://github.com/mdedetrich/sbt-apache-sonatype. Every single Pekko
>> > > repo contains the relevant LICENSE, NOTICE and DISCLAIMER files both
>> in
>> > the
>> > > source dist package and also in the maven jar convenience package
>> > >
>> > > > 1. do not work on a change without a ticket.
>> > >
>> > > This can be considered somewhat unnecessary with regards to how github
>> > > works, more specifically a PR in github **is a ticket**  and since we
>> use
>> > > github for both pull requests and tickets this distinction can be
>> > > considered somewhat arbitrary.
>> > >
>> > > On a related note, the reason why people worked on features without
>> > > creating issues first is simply due to velocity. If we had to go
>> through
>> > a
>> > > formal process we would have achieved a fraction of what we have done
>> > > currently. I would suggest the most appropriate time to introduce new
>> > > process would be after the first release, not now. Its already been
>> both
>> > > communicated and demonstrated multiple times that the current
>> development
>> > > process of Pekko is highly tailored to the unique scenario we are in
>> > (i.e.
>> > > first release after a hard work) and that it does need to change, but
>> > doing
>> > > so now in my opinion would introduce a real risk of slowing things
>> down
>> > > which is the exact opposite of what we want to do now.
>> > >
>> > > > 2. make sure the ticket is associated with the milestone.
>> > >
>> > > This is already being done, at least for tickets/pull requests that
>> are
>> > > relevant to a milestone. See https://github.com/orgs/apache/projects/
>> > and
>> > > https://github.com/apache/incubator-pekko/milestones
>> > >
>> > > >   3. when you are working on a ticket, assign it to yourself.  If
>> you
>> > can
>> > > not assign it to yourself, post a note on the dev mailing list with a
>> > > subject prefix of [ACCESS] noting the ticket that you do not have
>> access
>> > to
>> > > and then add a comment to the ticket indicating that you are working
>> on
>> > it.
>> > >
>> > > Github doesn't allow you to assign tickets if you are not a maintainer
>> > > (which for Apache projects means comitter). I have noted in the past
>> that
>> > > we need to be a lot better about assigning in general (this has been
>> > noted
>> > > multiple times) however I will raise the some points raised in
>> response
>> > to
>> > > point number 1 here.
>> > >
>> > > >   4. any change that is not a simple break/fix should be discussed
>> in
>> > the
>> > > dev mail list with the subject prefix of [DISUCSS].
>> > >
>> > > What is the mentality behind this? I strongly disagree with making
>> such a
>> > > rule, we shouldn't be preventing people from making contributions
>> even if
>> > > they are not focused on the current milestone/release candidate. I
>> don't
>> > > have any issue in freezing the repository during a release process but
>> > this
>> > > is one step too far. Its especially damaging in the case of Pekko
>> because
>> > > since we have made an agreement that version 1.0.x doesn't introduce
>> new
>> > > features, people have been making pull requests for new features over
>> > time
>> > > with the expectation they will get merged later. Such a rule send a
>> very
>> > > negative signal to potential contributors.
>> > >
>> > > On Tue, May 23, 2023 at 12:38 PM Claude Warren, Jr
>> > > <cl...@aiven.io.invalid> wrote:
>> > >
>> > >> Greetings,
>> > >>
>> > >>
>> > >> I think that it is time to start to take a hard look at a release
>> > >> candidate.  I do not expect the release candidate to succeed,
>> however,
>> > >> there is a lot to be learned in the failure.
>> > >>
>> > >> There is a milestone [1] currently open.  Let's take that and work
>> > toward
>> > >> the release candidate.
>> > >>
>> > >> I think we should start by determining if there are items that are
>> not
>> > >> required for the release and items that are missing.  I am opening
>> two
>> > >> issues today that I will add to the milestone.  They are the
>> addition of
>> > >> the LICENSE and NOTICE files to the snapshots, under the assumption
>> that
>> > >> doing so will also add them to the release.  Let's clean the list and
>> > then
>> > >> concentrate on only those items that will get us to RC1.  So let's
>> hold
>> > a
>> > >> vote on the items in the current milestone and see if they all need
>> to
>> > be
>> > >> in release 1.0
>> > >>
>> > >> In this process we will need to ensure that everyone is communicating
>> > what
>> > >> they are doing.  So, I think we should do the following:
>> > >>   1. do not work on a change without a ticket.
>> > >>   2. make sure the ticket is associated with the milestone.
>> > >>   3. when you are working on a ticket, assign it to yourself.  If you
>> > can
>> > >> not assign it to yourself, post a note on the dev mailing list with a
>> > >> subject prefix of [ACCESS] noting the ticket that you do not have
>> access
>> > >> to
>> > >> and then add a comment to the ticket indicating that you are working
>> on
>> > >> it.
>> > >>   4. any change that is not a simple break/fix should be discussed in
>> > the
>> > >> dev mail list with the subject prefix of [DISUCSS].
>> > >>
>> > >> We should also be looking on the dev list for offers of assistance
>> and
>> > >> point them to the milestone first.  We need to develop standard
>> > operating
>> > >> procedures that are welcoming to new contributors.
>> > >>
>> > >> [1] https://github.com/apache/incubator-pekko/milestone/1
>> > >>
>> > >
>> > >
>> > > --
>> > >
>> > > Matthew de Detrich
>> > >
>> > > *Aiven Deutschland GmbH*
>> > >
>> > > Immanuelkirchstraße 26, 10405 Berlin
>> > >
>> > > Amtsgericht Charlottenburg, HRB 209739 B
>> > >
>> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>> > >
>> > > *m:* +491603708037
>> > >
>> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>> > >
>> >
>> >
>> > --
>> >
>> > Matthew de Detrich
>> >
>> > *Aiven Deutschland GmbH*
>> >
>> > Immanuelkirchstraße 26, 10405 Berlin
>> >
>> > Amtsgericht Charlottenburg, HRB 209739 B
>> >
>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>> >
>> > *m:* +491603708037
>> >
>> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>> >
>>
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] RC1 build

Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,

> Also I would like to point out/remind that even though making a source
> package release is important for Apache, it's useless for Pekko's users.

It’s not useless. It provides confidence to users that the IP, licensing and other ASF process have been followed so that they can have a high level of trust in the release. They they can look at the source any time they want to verify what is in it. They can also verify what they have is what was released. Also making a release is an act of the foundation and gives the people involved (i.e. the Pekko PPMC) legal protection.

Kind Regards,
Justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] RC1 build

Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,

> Also I would like to point out/remind that even though making a source
> package release is important for Apache, it's useless for Pekko's users.

It’s not useless. It provides confidence to users that the IP, licensing and other ASF process have been followed so that they can have a high level of trust in the release. They they can look at the source any time they want to verify what is in it. They can also verify what they have is what was released. Also making a release is an act of the foundation and gives the people involved (i.e. the Pekko PPMC) legal protection.

Kind Regards,
Justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
For additional commands, e-mail: dev-help@pekko.apache.org


Re: [DISCUSS] RC1 build

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
> There are a number of issues that have been identified with respect to how
to do the release process.  Things like a full automation of the process is
putting the cart before the horse.

No one here is suggesting any kind of automation, this was already clear
months ago.

> What we want to see here is not perfection, we want to see the process
being built.

I don't think we are even at a stage where perfection is even a thought,
the current discussions we
have are just the basics of making an artifact. For example unless you
specifically setup your host
system to have multiple JDK's installed at once, Pekko will not even build
because it relies on
java.misc.unsafe. This is why there is a Docker image, which while not
strictly required makes it
easier/simpler and even more importantly for other committers to verify the
release.

Also I would like to point out/remind that even though making a source
package release is important
for Apache, it's useless for Pekko's users. It's been stated many times on
the mailing list that the people
actually using Pekko ultimately only care about the maven binary jars (i.e.
1.0.0 release) and that a lot
of potential contributors are waiting for that 1.0.0 maven jar binary
release. The ironic thing here is that
we already have a method of creating proper artifacts, it's just done by
github actions CI which doesn't
adhere to Apache's processes for a release and that is an example of what's
making it more complicated
(otherwise we could have been pushing RC's weeks/months ago).

On another note, while it's definitely true that we should strongly
encourage getting a release out sooner
than later, in my view arbitrary checkboxing of some process doesn't do
anyone any favors (whether it's
Apache, IPMC or Pekko users).



On Tue, May 23, 2023 at 1:55 PM Claude Warren, Jr
<cl...@aiven.io.invalid> wrote:

> If the milestone is placed in the snapshot area it is not a release.  I
> strongly recommend that Pekko seriously consider making the milestone a
> release and plan to work through a number of release candidates to get the
> process ironed out.
>
> There are a number of issues that have been identified with respect to how
> to do the release process.  Things like a full automation of the process is
> putting the cart before the horse.  You can't know what to automate until
> you know what to do.  So let's agree what goes into RC1, get those issues
> resolved and document a step-by-step release for RC1 as it is done.  When
> complete and reviewed we will have issues that will require another RC.
> Now you can probably automate some steps.  Others will require human
> intervention.
>
> What we want to see here is not perfection, we want to see the process
> being built.  We want to bring first hand knowledge of the tricks and
> pitfalls of the release process to a wider community.  We want to get some
> packaging out for review by the IPMC.
>
> Claude
>
>
>
> On Tue, May 23, 2023 at 11:56 AM Matthew Benedict de Detrich
> <ma...@aiven.io.invalid> wrote:
>
> > > (i.e. first release after a hard work)
> >
> > This is meant to say "(i.e. first release after a hard fork)"
> >
> > On Tue, May 23, 2023 at 12:53 PM Matthew Benedict de Detrich <
> > matthew.dedetrich@aiven.io> wrote:
> >
> > > > I think that it is time to start to take a hard look at a release
> > > candidate.  I do not expect the release candidate to succeed, however,
> > > there is a lot to be learned in the failure.
> > >
> > > Our current strategy is to release a MILESTONE which should hopefully
> > > happen this week. This is because doing a RC is more problematic due to
> > how
> > > difficult it is to build Pekko outside of CI (with a MILESTONE we can
> > > release it under Apache Snapshots Repo)
> > >
> > > > I think we should start by determining if there are items that are
> not
> > > required for the release and items that are missing.  I am opening two
> > > issues today that I will add to the milestone.  They are the addition
> of
> > > the LICENSE and NOTICE files to the snapshots, under the assumption
> that
> > > doing so will also add them to the release.
> > >
> > > FYI this has already been done for all Pekko modules, the core logic of
> > > doing so is abstracted in
> > > https://github.com/mdedetrich/sbt-apache-sonatype. Every single Pekko
> > > repo contains the relevant LICENSE, NOTICE and DISCLAIMER files both in
> > the
> > > source dist package and also in the maven jar convenience package
> > >
> > > > 1. do not work on a change without a ticket.
> > >
> > > This can be considered somewhat unnecessary with regards to how github
> > > works, more specifically a PR in github **is a ticket**  and since we
> use
> > > github for both pull requests and tickets this distinction can be
> > > considered somewhat arbitrary.
> > >
> > > On a related note, the reason why people worked on features without
> > > creating issues first is simply due to velocity. If we had to go
> through
> > a
> > > formal process we would have achieved a fraction of what we have done
> > > currently. I would suggest the most appropriate time to introduce new
> > > process would be after the first release, not now. Its already been
> both
> > > communicated and demonstrated multiple times that the current
> development
> > > process of Pekko is highly tailored to the unique scenario we are in
> > (i.e.
> > > first release after a hard work) and that it does need to change, but
> > doing
> > > so now in my opinion would introduce a real risk of slowing things down
> > > which is the exact opposite of what we want to do now.
> > >
> > > > 2. make sure the ticket is associated with the milestone.
> > >
> > > This is already being done, at least for tickets/pull requests that are
> > > relevant to a milestone. See https://github.com/orgs/apache/projects/
> > and
> > > https://github.com/apache/incubator-pekko/milestones
> > >
> > > >   3. when you are working on a ticket, assign it to yourself.  If you
> > can
> > > not assign it to yourself, post a note on the dev mailing list with a
> > > subject prefix of [ACCESS] noting the ticket that you do not have
> access
> > to
> > > and then add a comment to the ticket indicating that you are working on
> > it.
> > >
> > > Github doesn't allow you to assign tickets if you are not a maintainer
> > > (which for Apache projects means comitter). I have noted in the past
> that
> > > we need to be a lot better about assigning in general (this has been
> > noted
> > > multiple times) however I will raise the some points raised in response
> > to
> > > point number 1 here.
> > >
> > > >   4. any change that is not a simple break/fix should be discussed in
> > the
> > > dev mail list with the subject prefix of [DISUCSS].
> > >
> > > What is the mentality behind this? I strongly disagree with making
> such a
> > > rule, we shouldn't be preventing people from making contributions even
> if
> > > they are not focused on the current milestone/release candidate. I
> don't
> > > have any issue in freezing the repository during a release process but
> > this
> > > is one step too far. Its especially damaging in the case of Pekko
> because
> > > since we have made an agreement that version 1.0.x doesn't introduce
> new
> > > features, people have been making pull requests for new features over
> > time
> > > with the expectation they will get merged later. Such a rule send a
> very
> > > negative signal to potential contributors.
> > >
> > > On Tue, May 23, 2023 at 12:38 PM Claude Warren, Jr
> > > <cl...@aiven.io.invalid> wrote:
> > >
> > >> Greetings,
> > >>
> > >>
> > >> I think that it is time to start to take a hard look at a release
> > >> candidate.  I do not expect the release candidate to succeed, however,
> > >> there is a lot to be learned in the failure.
> > >>
> > >> There is a milestone [1] currently open.  Let's take that and work
> > toward
> > >> the release candidate.
> > >>
> > >> I think we should start by determining if there are items that are not
> > >> required for the release and items that are missing.  I am opening two
> > >> issues today that I will add to the milestone.  They are the addition
> of
> > >> the LICENSE and NOTICE files to the snapshots, under the assumption
> that
> > >> doing so will also add them to the release.  Let's clean the list and
> > then
> > >> concentrate on only those items that will get us to RC1.  So let's
> hold
> > a
> > >> vote on the items in the current milestone and see if they all need to
> > be
> > >> in release 1.0
> > >>
> > >> In this process we will need to ensure that everyone is communicating
> > what
> > >> they are doing.  So, I think we should do the following:
> > >>   1. do not work on a change without a ticket.
> > >>   2. make sure the ticket is associated with the milestone.
> > >>   3. when you are working on a ticket, assign it to yourself.  If you
> > can
> > >> not assign it to yourself, post a note on the dev mailing list with a
> > >> subject prefix of [ACCESS] noting the ticket that you do not have
> access
> > >> to
> > >> and then add a comment to the ticket indicating that you are working
> on
> > >> it.
> > >>   4. any change that is not a simple break/fix should be discussed in
> > the
> > >> dev mail list with the subject prefix of [DISUCSS].
> > >>
> > >> We should also be looking on the dev list for offers of assistance and
> > >> point them to the milestone first.  We need to develop standard
> > operating
> > >> procedures that are welcoming to new contributors.
> > >>
> > >> [1] https://github.com/apache/incubator-pekko/milestone/1
> > >>
> > >
> > >
> > > --
> > >
> > > Matthew de Detrich
> > >
> > > *Aiven Deutschland GmbH*
> > >
> > > Immanuelkirchstraße 26, 10405 Berlin
> > >
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >
> > > *m:* +491603708037
> > >
> > > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> > >
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

Re: [DISCUSS] RC1 build

Posted by "Claude Warren, Jr" <cl...@aiven.io.INVALID>.
If the milestone is placed in the snapshot area it is not a release.  I
strongly recommend that Pekko seriously consider making the milestone a
release and plan to work through a number of release candidates to get the
process ironed out.

There are a number of issues that have been identified with respect to how
to do the release process.  Things like a full automation of the process is
putting the cart before the horse.  You can't know what to automate until
you know what to do.  So let's agree what goes into RC1, get those issues
resolved and document a step-by-step release for RC1 as it is done.  When
complete and reviewed we will have issues that will require another RC.
Now you can probably automate some steps.  Others will require human
intervention.

What we want to see here is not perfection, we want to see the process
being built.  We want to bring first hand knowledge of the tricks and
pitfalls of the release process to a wider community.  We want to get some
packaging out for review by the IPMC.

Claude



On Tue, May 23, 2023 at 11:56 AM Matthew Benedict de Detrich
<ma...@aiven.io.invalid> wrote:

> > (i.e. first release after a hard work)
>
> This is meant to say "(i.e. first release after a hard fork)"
>
> On Tue, May 23, 2023 at 12:53 PM Matthew Benedict de Detrich <
> matthew.dedetrich@aiven.io> wrote:
>
> > > I think that it is time to start to take a hard look at a release
> > candidate.  I do not expect the release candidate to succeed, however,
> > there is a lot to be learned in the failure.
> >
> > Our current strategy is to release a MILESTONE which should hopefully
> > happen this week. This is because doing a RC is more problematic due to
> how
> > difficult it is to build Pekko outside of CI (with a MILESTONE we can
> > release it under Apache Snapshots Repo)
> >
> > > I think we should start by determining if there are items that are not
> > required for the release and items that are missing.  I am opening two
> > issues today that I will add to the milestone.  They are the addition of
> > the LICENSE and NOTICE files to the snapshots, under the assumption that
> > doing so will also add them to the release.
> >
> > FYI this has already been done for all Pekko modules, the core logic of
> > doing so is abstracted in
> > https://github.com/mdedetrich/sbt-apache-sonatype. Every single Pekko
> > repo contains the relevant LICENSE, NOTICE and DISCLAIMER files both in
> the
> > source dist package and also in the maven jar convenience package
> >
> > > 1. do not work on a change without a ticket.
> >
> > This can be considered somewhat unnecessary with regards to how github
> > works, more specifically a PR in github **is a ticket**  and since we use
> > github for both pull requests and tickets this distinction can be
> > considered somewhat arbitrary.
> >
> > On a related note, the reason why people worked on features without
> > creating issues first is simply due to velocity. If we had to go through
> a
> > formal process we would have achieved a fraction of what we have done
> > currently. I would suggest the most appropriate time to introduce new
> > process would be after the first release, not now. Its already been both
> > communicated and demonstrated multiple times that the current development
> > process of Pekko is highly tailored to the unique scenario we are in
> (i.e.
> > first release after a hard work) and that it does need to change, but
> doing
> > so now in my opinion would introduce a real risk of slowing things down
> > which is the exact opposite of what we want to do now.
> >
> > > 2. make sure the ticket is associated with the milestone.
> >
> > This is already being done, at least for tickets/pull requests that are
> > relevant to a milestone. See https://github.com/orgs/apache/projects/
> and
> > https://github.com/apache/incubator-pekko/milestones
> >
> > >   3. when you are working on a ticket, assign it to yourself.  If you
> can
> > not assign it to yourself, post a note on the dev mailing list with a
> > subject prefix of [ACCESS] noting the ticket that you do not have access
> to
> > and then add a comment to the ticket indicating that you are working on
> it.
> >
> > Github doesn't allow you to assign tickets if you are not a maintainer
> > (which for Apache projects means comitter). I have noted in the past that
> > we need to be a lot better about assigning in general (this has been
> noted
> > multiple times) however I will raise the some points raised in response
> to
> > point number 1 here.
> >
> > >   4. any change that is not a simple break/fix should be discussed in
> the
> > dev mail list with the subject prefix of [DISUCSS].
> >
> > What is the mentality behind this? I strongly disagree with making such a
> > rule, we shouldn't be preventing people from making contributions even if
> > they are not focused on the current milestone/release candidate. I don't
> > have any issue in freezing the repository during a release process but
> this
> > is one step too far. Its especially damaging in the case of Pekko because
> > since we have made an agreement that version 1.0.x doesn't introduce new
> > features, people have been making pull requests for new features over
> time
> > with the expectation they will get merged later. Such a rule send a very
> > negative signal to potential contributors.
> >
> > On Tue, May 23, 2023 at 12:38 PM Claude Warren, Jr
> > <cl...@aiven.io.invalid> wrote:
> >
> >> Greetings,
> >>
> >>
> >> I think that it is time to start to take a hard look at a release
> >> candidate.  I do not expect the release candidate to succeed, however,
> >> there is a lot to be learned in the failure.
> >>
> >> There is a milestone [1] currently open.  Let's take that and work
> toward
> >> the release candidate.
> >>
> >> I think we should start by determining if there are items that are not
> >> required for the release and items that are missing.  I am opening two
> >> issues today that I will add to the milestone.  They are the addition of
> >> the LICENSE and NOTICE files to the snapshots, under the assumption that
> >> doing so will also add them to the release.  Let's clean the list and
> then
> >> concentrate on only those items that will get us to RC1.  So let's hold
> a
> >> vote on the items in the current milestone and see if they all need to
> be
> >> in release 1.0
> >>
> >> In this process we will need to ensure that everyone is communicating
> what
> >> they are doing.  So, I think we should do the following:
> >>   1. do not work on a change without a ticket.
> >>   2. make sure the ticket is associated with the milestone.
> >>   3. when you are working on a ticket, assign it to yourself.  If you
> can
> >> not assign it to yourself, post a note on the dev mailing list with a
> >> subject prefix of [ACCESS] noting the ticket that you do not have access
> >> to
> >> and then add a comment to the ticket indicating that you are working on
> >> it.
> >>   4. any change that is not a simple break/fix should be discussed in
> the
> >> dev mail list with the subject prefix of [DISUCSS].
> >>
> >> We should also be looking on the dev list for offers of assistance and
> >> point them to the milestone first.  We need to develop standard
> operating
> >> procedures that are welcoming to new contributors.
> >>
> >> [1] https://github.com/apache/incubator-pekko/milestone/1
> >>
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>

Re: [DISCUSS] RC1 build

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
> (i.e. first release after a hard work)

This is meant to say "(i.e. first release after a hard fork)"

On Tue, May 23, 2023 at 12:53 PM Matthew Benedict de Detrich <
matthew.dedetrich@aiven.io> wrote:

> > I think that it is time to start to take a hard look at a release
> candidate.  I do not expect the release candidate to succeed, however,
> there is a lot to be learned in the failure.
>
> Our current strategy is to release a MILESTONE which should hopefully
> happen this week. This is because doing a RC is more problematic due to how
> difficult it is to build Pekko outside of CI (with a MILESTONE we can
> release it under Apache Snapshots Repo)
>
> > I think we should start by determining if there are items that are not
> required for the release and items that are missing.  I am opening two
> issues today that I will add to the milestone.  They are the addition of
> the LICENSE and NOTICE files to the snapshots, under the assumption that
> doing so will also add them to the release.
>
> FYI this has already been done for all Pekko modules, the core logic of
> doing so is abstracted in
> https://github.com/mdedetrich/sbt-apache-sonatype. Every single Pekko
> repo contains the relevant LICENSE, NOTICE and DISCLAIMER files both in the
> source dist package and also in the maven jar convenience package
>
> > 1. do not work on a change without a ticket.
>
> This can be considered somewhat unnecessary with regards to how github
> works, more specifically a PR in github **is a ticket**  and since we use
> github for both pull requests and tickets this distinction can be
> considered somewhat arbitrary.
>
> On a related note, the reason why people worked on features without
> creating issues first is simply due to velocity. If we had to go through a
> formal process we would have achieved a fraction of what we have done
> currently. I would suggest the most appropriate time to introduce new
> process would be after the first release, not now. Its already been both
> communicated and demonstrated multiple times that the current development
> process of Pekko is highly tailored to the unique scenario we are in (i.e.
> first release after a hard work) and that it does need to change, but doing
> so now in my opinion would introduce a real risk of slowing things down
> which is the exact opposite of what we want to do now.
>
> > 2. make sure the ticket is associated with the milestone.
>
> This is already being done, at least for tickets/pull requests that are
> relevant to a milestone. See https://github.com/orgs/apache/projects/ and
> https://github.com/apache/incubator-pekko/milestones
>
> >   3. when you are working on a ticket, assign it to yourself.  If you can
> not assign it to yourself, post a note on the dev mailing list with a
> subject prefix of [ACCESS] noting the ticket that you do not have access to
> and then add a comment to the ticket indicating that you are working on it.
>
> Github doesn't allow you to assign tickets if you are not a maintainer
> (which for Apache projects means comitter). I have noted in the past that
> we need to be a lot better about assigning in general (this has been noted
> multiple times) however I will raise the some points raised in response to
> point number 1 here.
>
> >   4. any change that is not a simple break/fix should be discussed in the
> dev mail list with the subject prefix of [DISUCSS].
>
> What is the mentality behind this? I strongly disagree with making such a
> rule, we shouldn't be preventing people from making contributions even if
> they are not focused on the current milestone/release candidate. I don't
> have any issue in freezing the repository during a release process but this
> is one step too far. Its especially damaging in the case of Pekko because
> since we have made an agreement that version 1.0.x doesn't introduce new
> features, people have been making pull requests for new features over time
> with the expectation they will get merged later. Such a rule send a very
> negative signal to potential contributors.
>
> On Tue, May 23, 2023 at 12:38 PM Claude Warren, Jr
> <cl...@aiven.io.invalid> wrote:
>
>> Greetings,
>>
>>
>> I think that it is time to start to take a hard look at a release
>> candidate.  I do not expect the release candidate to succeed, however,
>> there is a lot to be learned in the failure.
>>
>> There is a milestone [1] currently open.  Let's take that and work toward
>> the release candidate.
>>
>> I think we should start by determining if there are items that are not
>> required for the release and items that are missing.  I am opening two
>> issues today that I will add to the milestone.  They are the addition of
>> the LICENSE and NOTICE files to the snapshots, under the assumption that
>> doing so will also add them to the release.  Let's clean the list and then
>> concentrate on only those items that will get us to RC1.  So let's hold a
>> vote on the items in the current milestone and see if they all need to be
>> in release 1.0
>>
>> In this process we will need to ensure that everyone is communicating what
>> they are doing.  So, I think we should do the following:
>>   1. do not work on a change without a ticket.
>>   2. make sure the ticket is associated with the milestone.
>>   3. when you are working on a ticket, assign it to yourself.  If you can
>> not assign it to yourself, post a note on the dev mailing list with a
>> subject prefix of [ACCESS] noting the ticket that you do not have access
>> to
>> and then add a comment to the ticket indicating that you are working on
>> it.
>>   4. any change that is not a simple break/fix should be discussed in the
>> dev mail list with the subject prefix of [DISUCSS].
>>
>> We should also be looking on the dev list for offers of assistance and
>> point them to the milestone first.  We need to develop standard operating
>> procedures that are welcoming to new contributors.
>>
>> [1] https://github.com/apache/incubator-pekko/milestone/1
>>
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io

[DISCUSS] Ticket process

Posted by "Claude Warren, Jr" <cl...@aiven.io.INVALID>.
This was part of "[DISCUSS] RC1 build"  but seems like it might need to be
discussed separately.

> 1. do not work on a change without a ticket.
>
> This can be considered somewhat unnecessary with regards to how github
> works, more specifically a PR in github **is a ticket**  and since we use
> github for both pull requests and tickets this distinction can be
> considered somewhat arbitrary.
>

The reason this is necessary is because people coming into the project
looking for things to work on are going to look at tickets not pull
requests.  In order to keep a handle on what is being done we need to have
issues, and we need to work to the issues.  If you work on a change and I
work on the same change and neither of us has a ticket open then we have no
coordination.  In addition, anyone else that is interested in the outcome
or design of the ticket does not have a place to a) discover the work is
being done or b) contribute to the discussion/work.  Not working to tickets
makes the process more opaque.


> On a related note, the reason why people worked on features without
> creating issues first is simply due to velocity. If we had to go through a
> formal process we would have achieved a fraction of what we have done
> currently. I would suggest the most appropriate time to introduce new
> process would be after the first release, not now. Its already been both
> communicated and demonstrated multiple times that the current development
> process of Pekko is highly tailored to the unique scenario we are in (i.e.
> first release after a hard work) and that it does need to change, but doing
> so now in my opinion would introduce a real risk of slowing things down
> which is the exact opposite of what we want to do now.


How much time does it take to open a ticket if you want to do some
particular piece of work?  Does it slow you down?  Sure 5 minutes to open a
ticket and then you are off and running.  You've got documentation of the
issue, you've put yourself out there as a contact for the resolution of the
issue.  You can point to the issue on the mailing list.  You have a place
for discussion about the issue.

If there is not documentation for what was done.  If there is not a trail
discussing what needs to be done.  If that trail is not searchable.  If
that trail is not immutable.  If there is not opportunity for new
contributors to easily discover where they can contribute and to contribute
to the project then the probability of getting the release approved by  the
IPMC is slim.  I am not saying Pekko has to meet all the items listed
above, but a preponderance would be good.  I understand the original need
to get things in shape quickly.  But the team needs to understand that the
time for speed may be passed and the time to show that Pekko believes in
Community over Code is upon us.   From what I have seen, Pekko works.  It
can be used to replace Akka in multiple environments.  Yeoman' work has
been completed, and the team is to be congratulated for that.  But now turn
attention to the processes that Apache puts to the fore.  Community over
Code is not just a slogan, Pekko needs to work to be more transparent and
more accessible.


> > 2. make sure the ticket is associated with the milestone.
>
> This is already being done, at least for tickets/pull requests that are
> relevant to a milestone. See https://github.com/orgs/apache/projects/ and
> https://github.com/apache/incubator-pekko/milestones


I wanted to make it clear that we need to focus on tickets that are in the
milestone and leave the others alone.  When added to the above "No work
without a ticket" it means we are working to get the milestone closed.

>   3. when you are working on a ticket, assign it to yourself.  If you can
> not assign it to yourself, post a note on the dev mailing list with a
> subject prefix of [ACCESS] noting the ticket that you do not have access to
> and then add a comment to the ticket indicating that you are working on it.
>
> Github doesn't allow you to assign tickets if you are not a maintainer
> (which for Apache projects means comitter). I have noted in the past that
> we need to be a lot better about assigning in general (this has been noted
> multiple times) however I will raise the some points raised in response to
> point number 1 here.


Collaborators can assign issues, label them, and close issues. It's
configurable via .asf.yaml.  So we can add non-committers as collaborators
and they can begin to work within this framework.  An example can be found
in the Air Flow project [1].  This can be a good first step to welcome
contributors that may become committers later.

>
> >   4. any change that is not a simple break/fix should be discussed in the
> dev mail list with the subject prefix of [DISUCSS].
>
> What is the mentality behind this? I strongly disagree with making such a
> rule, we shouldn't be preventing people from making contributions even if
> they are not focused on the current milestone/release candidate. I don't
> have any issue in freezing the repository during a release process but this
> is one step too far. Its especially damaging in the case of Pekko because
> since we have made an agreement that version 1.0.x doesn't introduce new
> features, people have been making pull requests for new features over time
> with the expectation they will get merged later. Such a rule send a very
> negative signal to potential contributors.
>

Sorry, I was not clear here.  I meant this to apply to any ticket in the
current milestone [2].  As you state the goal is to release without change,
so this is to enforce that within the current milestone.  I do not propose
that we not accept any improvements as fixes for CVE for example should be
accepted -- after proper due diligence.

Claude

[1] https://github.com/apache/airflow/blob/main/.asf.yaml#L73
[2] https://github.com/apache/incubator-pekko/milestone/1

Re: [DISCUSS] RC1 build

Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
> I think that it is time to start to take a hard look at a release
candidate.  I do not expect the release candidate to succeed, however,
there is a lot to be learned in the failure.

Our current strategy is to release a MILESTONE which should hopefully
happen this week. This is because doing a RC is more problematic due to how
difficult it is to build Pekko outside of CI (with a MILESTONE we can
release it under Apache Snapshots Repo)

> I think we should start by determining if there are items that are not
required for the release and items that are missing.  I am opening two
issues today that I will add to the milestone.  They are the addition of
the LICENSE and NOTICE files to the snapshots, under the assumption that
doing so will also add them to the release.

FYI this has already been done for all Pekko modules, the core logic of
doing so is abstracted in https://github.com/mdedetrich/sbt-apache-sonatype.
Every single Pekko repo contains the relevant LICENSE, NOTICE and
DISCLAIMER files both in the source dist package and also in the maven jar
convenience package

> 1. do not work on a change without a ticket.

This can be considered somewhat unnecessary with regards to how github
works, more specifically a PR in github **is a ticket**  and since we use
github for both pull requests and tickets this distinction can be
considered somewhat arbitrary.

On a related note, the reason why people worked on features without
creating issues first is simply due to velocity. If we had to go through a
formal process we would have achieved a fraction of what we have done
currently. I would suggest the most appropriate time to introduce new
process would be after the first release, not now. Its already been both
communicated and demonstrated multiple times that the current development
process of Pekko is highly tailored to the unique scenario we are in (i.e.
first release after a hard work) and that it does need to change, but doing
so now in my opinion would introduce a real risk of slowing things down
which is the exact opposite of what we want to do now.

> 2. make sure the ticket is associated with the milestone.

This is already being done, at least for tickets/pull requests that are
relevant to a milestone. See https://github.com/orgs/apache/projects/ and
https://github.com/apache/incubator-pekko/milestones

>   3. when you are working on a ticket, assign it to yourself.  If you can
not assign it to yourself, post a note on the dev mailing list with a
subject prefix of [ACCESS] noting the ticket that you do not have access to
and then add a comment to the ticket indicating that you are working on it.

Github doesn't allow you to assign tickets if you are not a maintainer
(which for Apache projects means comitter). I have noted in the past that
we need to be a lot better about assigning in general (this has been noted
multiple times) however I will raise the some points raised in response to
point number 1 here.

>   4. any change that is not a simple break/fix should be discussed in the
dev mail list with the subject prefix of [DISUCSS].

What is the mentality behind this? I strongly disagree with making such a
rule, we shouldn't be preventing people from making contributions even if
they are not focused on the current milestone/release candidate. I don't
have any issue in freezing the repository during a release process but this
is one step too far. Its especially damaging in the case of Pekko because
since we have made an agreement that version 1.0.x doesn't introduce new
features, people have been making pull requests for new features over time
with the expectation they will get merged later. Such a rule send a very
negative signal to potential contributors.

On Tue, May 23, 2023 at 12:38 PM Claude Warren, Jr
<cl...@aiven.io.invalid> wrote:

> Greetings,
>
>
> I think that it is time to start to take a hard look at a release
> candidate.  I do not expect the release candidate to succeed, however,
> there is a lot to be learned in the failure.
>
> There is a milestone [1] currently open.  Let's take that and work toward
> the release candidate.
>
> I think we should start by determining if there are items that are not
> required for the release and items that are missing.  I am opening two
> issues today that I will add to the milestone.  They are the addition of
> the LICENSE and NOTICE files to the snapshots, under the assumption that
> doing so will also add them to the release.  Let's clean the list and then
> concentrate on only those items that will get us to RC1.  So let's hold a
> vote on the items in the current milestone and see if they all need to be
> in release 1.0
>
> In this process we will need to ensure that everyone is communicating what
> they are doing.  So, I think we should do the following:
>   1. do not work on a change without a ticket.
>   2. make sure the ticket is associated with the milestone.
>   3. when you are working on a ticket, assign it to yourself.  If you can
> not assign it to yourself, post a note on the dev mailing list with a
> subject prefix of [ACCESS] noting the ticket that you do not have access to
> and then add a comment to the ticket indicating that you are working on it.
>   4. any change that is not a simple break/fix should be discussed in the
> dev mail list with the subject prefix of [DISUCSS].
>
> We should also be looking on the dev list for offers of assistance and
> point them to the milestone first.  We need to develop standard operating
> procedures that are welcoming to new contributors.
>
> [1] https://github.com/apache/incubator-pekko/milestone/1
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetrich@aiven.io