You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openwhisk.apache.org by Martin Henke <ma...@web.de> on 2020/01/28 13:05:17 UTC
Re: JDK 11
Folks,
as indicated, we performed performance tests of the OpenJDK upgrade PR https://github.com/apache/openwhisk/pull/4706
in our performance test environment (we are using the exact set of Gatling tests that are contributed to Openwhisk).
Though our first impressions looked good, we found severe degradations in the latency and warm invocation tests afterwards.
Latency tests dropped to around 50% of the JDK8 results, warm invocation to a staggering 25%.
These results were stable with using the original and the updated Gatling image and switching to parallelGC on OpenJDK11.
Since GC does not seem to matter, the degradation might be caused by a changed
implementation in the OpenJDK 11 (e.g. UUID generation).
We did not perform any deeper research.
Next we tried to change to use the OpenJ9 JDK11 instead of OpenJDK.
Change in common/scala/Dockerfile:
"
-FROM adoptopenjdk/openjdk11:x86_64-alpine-jdk-11.0.3_7
+FROM adoptopenjdk/openjdk11-openj9:x86_64-alpine-jdk-11.0.5_10_openj9-0.17.0
"
With OpenJ9 in default configuration all performance metrics were looking good again.
We did not see any degredation anymore.
In the light of these test results, I herewith would like to start the discussion to base the JDK update PR
to openJ9 JDK11.
Kind Regards,
Martin
On 2019/12/23 01:05:20, Rodric Rabbah <r....@gmail.com> wrote:
> Thanks Martin. What I’m most interested in is knowing what other than the Gatling tests which we are running in Travis and can run in Jenkins if anything you’re doing differently so that the project can replicate and automate it where necessary. >
>
> -r>
>
> > On Dec 22, 2019, at 2:34 PM, Martin Henke <ma...@web.de> wrote:>
> > >
> > Rodric,>
> > >
> > I forgot to mention. When we find a degradation (and I honestly do not hope to see one) and we find the reason, we will of course share the information needed to setup tests in the Apache environment to catch those. >
> > >
> > Regards,>
> > Martin>
> > >
> >> Am 22.12.2019 um 20:29 schrieb Martin Henke <ma...@web.de>:>
> >> >
> >> Rodric,>
> >> >
> >> It seems to be natural that the IBM environment differs in machine configuration and network bandwidth and any other parameters from any other OW installation.>
> >> >
> >> Nevertheless I see value for the project to test the JDK upgrade additionally in our env to identify potential problems. As said, we saw performance degredations before, which were not caught in the Apache environments.>
> >> >
> >> Regards,>
> >> Martin>
> >> >
> >> P.S I am away from keyboard for the next time and might be slow in answering.>
> >> >
> >>>> Am 22.12.2019 um 19:33 schrieb Rodric Rabbah <ro...@gmail.com>:>
> >>> >
> >>> Thanks Martin. Can you share the environment details/how it’s different from the Jenkins job Vincent set up at one point for the project - so that any testing important to the project can be done in apache land. >
> >>> >
> >>> -r>
> >>> >
> >>>> On Dec 22, 2019, at 9:35 AM, Martin Henke <ma...@web.de> wrote:>
> >>>> >
> >>>> Rodric,>
> >>>> >
> >>>> there are no special tests beside the ones that were already contributed by Christian from our team. >
> >>>> >
> >>>> The unknowns are given by our environment. >
> >>>> >
> >>>> Regards,>
> >>>> Martin>
> >>>> >
> >>>> Von meinem iPhone gesendet>
> >>>> >
> >>>>> Am 21.12.2019 um 12:45 schrieb Rodric Rabbah <ro...@gmail.com>:>
> >>>>> >
> >>>>> Martin thanks for the input. Do you or others plan to contribute any of the>
> >>>>> performance testing that you're doing/will do to the project - and what>
> >>>>> would it take to facilitate this so that we are also increasing the project>
> >>>>> velocity?>
> >>>>> >
> >>>>> -r>
> >>>>> >
> >>>>>>> On Sat, Dec 21, 2019 at 5:08 AM Martin Henke <ma...@web.de> wrote:>
> >>>>>> >
> >>>>>> Rodric,>
> >>>>>> >
> >>>>>> thank you for taking on this dangling task.>
> >>>>>> >
> >>>>>> Since we saw some severe performance impacts when we tried to move OW to>
> >>>>>> other Java 8 JDK versions>
> >>>>>> before, we would like to run the final change through our performance>
> >>>>>> tests.>
> >>>>>> >
> >>>>>> Therefore I would like to ask for a short deferral until midth of January>
> >>>>>> when people have returned>
> >>>>>> from their Christmas holidays (you know that we Germans tend to have long>
> >>>>>> holidays :-)).>
> >>>>>> >
> >>>>>> Regards,>
> >>>>>> Martin>
> >>>>>> >
> >>>>>> >
> >>>>>>> On 2019/12/20 21:56:12, Rodric Rabbah <r....@gmail.com> wrote:>
> >>>>>>> This PR https://github.com/apache/openwhisk/pull/4706 move openwhisk>
> >>>>>> to>>
> >>>>>>> using Java 11 for building and running the controller/invoker>
> >>>>>> containers.>>
> >>>>>>> It has been open since Oct 30 with no comments since Nov 14. It was>
> >>>>>> covered>>
> >>>>>>> at the Dec 4 community call>>
> >>>>>>> >
> >>>>>> https://cwiki.apache.org/confluence/display/OPENWHISK/2019-12-04+OW+Tech+Interchange+Meeting+Notes>>
> >>>>>> >
> >>>>>>> and there was agreement to merge the PR.>>
> >>>>>>> >
> >>>>>>> Accordingly, I will merge it next week.>>
> >>>>>>> >
> >>>>>>> -r>>
> >>>>>>> >
> >>>> >
> >> >
> > >
>
Re: JDK 11
Posted by Rodric Rabbah <ro...@gmail.com>.
Can we make the base image configurable in the dockerfile? The Apache
project should adopt one image but it's not likely to address everyone's
requirements so if we can make it easier to specify the base image it might
be helpful to others.
-r
Re: JDK 11
Posted by dan McWeeney <mc...@adobe.com.INVALID>.
Looking at some of our internal documentation seems like we are looking to move towards Azul Zulu[0] for our JRE/JDK. Any have any experience with this or know if its license is compatible with Apache projects?
-d
[0] - https://www.azul.com/downloads/zulu-community/?&architecture=x86-64-bit&package=jdk
On 1/30/20, 11:21 AM, "Markus Thömmes" <ma...@apache.org> wrote:
FWIW, the UUID entropy workaround where not JVM specific IIRC but OS
dependent, as we switched from random to urandom as a source of entropy.
I don't feel strongly for either OpenJDK vs. OpenJ9, but it'd be nice to
understand where the degradation comes from and thus make a deliberate
decision to choose one over the other based on that research. If anything,
that might be very valuable input for the OpenJDK community to fix this
regression, maybe? That research could also help us better understand where
the bottlenecks of our system are and thus point us towards improvements to
be made.
We could of course switch to OpenJ9 to bypass the degradation if we want to
move to JDK11 quickly.
Thanks for sharing the results,
Markus
Am Di., 28. Jan. 2020 um 18:24 Uhr schrieb Martin Henke <martin.henke@web.de
>:
> Rodric,
>
> UUID creation was just an example from the past where workarounds were
> needed to circumvent waiting for enough entropy for the random numbers used
> for UUID creation. As said we did no further investigation.
>
> Yes. OpenJ9 is Apache licensed and distributed via AdoptOpenJDK. There are
> no further changes needed.
>
> Regards,
> Martin
>
> > Am 28.01.2020 um 18:09 schrieb Rodric Rabbah <ro...@gmail.com>:
> >
> > Thanks Martin for sharing the results with the community - That's a
> steep
> > performance degradation.
> > Open J9 is Apache licensed. Are other changes needed?
> >
> > You hinted at uuid - do you know that's implicated or a guess?
> >
> > -r
> >
> >> On Tue, Jan 28, 2020 at 8:05 AM Martin Henke <ma...@web.de>
> wrote:
> >>
> >> Folks,
> >>
> >> as indicated, we performed performance tests of the OpenJDK upgrade PR
> >> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fopenwhisk%2Fpull%2F4706&data=02%7C01%7Cmcweeney%40adobe.com%7C763c5b76ea74463e572808d7a5b99f7d%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637160088976247846&sdata=MYWLUKkh733ckOEVNCL5727Y%2FIgyi0rcz5P0WJ4qjy0%3D&reserved=0
> >> in our performance test environment (we are using the exact set of
> Gatling
> >> tests that are contributed to Openwhisk).
> >>
> >> Though our first impressions looked good, we found severe degradations
> in
> >> the latency and warm invocation tests afterwards.
> >>
> >> Latency tests dropped to around 50% of the JDK8 results, warm invocation
> >> to a staggering 25%.
> >> These results were stable with using the original and the updated
> Gatling
> >> image and switching to parallelGC on OpenJDK11.
> >>
> >> Since GC does not seem to matter, the degradation might be caused by a
> >> changed
> >> implementation in the OpenJDK 11 (e.g. UUID generation).
> >> We did not perform any deeper research.
> >>
> >> Next we tried to change to use the OpenJ9 JDK11 instead of OpenJDK.
> >> Change in common/scala/Dockerfile:
> >> "
> >> -FROM adoptopenjdk/openjdk11:x86_64-alpine-jdk-11.0.3_7
> >> +FROM
> >> adoptopenjdk/openjdk11-openj9:x86_64-alpine-jdk-11.0.5_10_openj9-0.17.0
> >> "
> >> With OpenJ9 in default configuration all performance metrics were
> looking
> >> good again.
> >> We did not see any degredation anymore.
> >>
> >> In the light of these test results, I herewith would like to start the
> >> discussion to base the JDK update PR
> >> to openJ9 JDK11.
> >>
> >> Kind Regards,
> >> Martin
> >>
>
>
Re: JDK 11
Posted by Markus Thömmes <ma...@apache.org>.
FWIW, the UUID entropy workaround where not JVM specific IIRC but OS
dependent, as we switched from random to urandom as a source of entropy.
I don't feel strongly for either OpenJDK vs. OpenJ9, but it'd be nice to
understand where the degradation comes from and thus make a deliberate
decision to choose one over the other based on that research. If anything,
that might be very valuable input for the OpenJDK community to fix this
regression, maybe? That research could also help us better understand where
the bottlenecks of our system are and thus point us towards improvements to
be made.
We could of course switch to OpenJ9 to bypass the degradation if we want to
move to JDK11 quickly.
Thanks for sharing the results,
Markus
Am Di., 28. Jan. 2020 um 18:24 Uhr schrieb Martin Henke <martin.henke@web.de
>:
> Rodric,
>
> UUID creation was just an example from the past where workarounds were
> needed to circumvent waiting for enough entropy for the random numbers used
> for UUID creation. As said we did no further investigation.
>
> Yes. OpenJ9 is Apache licensed and distributed via AdoptOpenJDK. There are
> no further changes needed.
>
> Regards,
> Martin
>
> > Am 28.01.2020 um 18:09 schrieb Rodric Rabbah <ro...@gmail.com>:
> >
> > Thanks Martin for sharing the results with the community - That's a
> steep
> > performance degradation.
> > Open J9 is Apache licensed. Are other changes needed?
> >
> > You hinted at uuid - do you know that's implicated or a guess?
> >
> > -r
> >
> >> On Tue, Jan 28, 2020 at 8:05 AM Martin Henke <ma...@web.de>
> wrote:
> >>
> >> Folks,
> >>
> >> as indicated, we performed performance tests of the OpenJDK upgrade PR
> >> https://github.com/apache/openwhisk/pull/4706
> >> in our performance test environment (we are using the exact set of
> Gatling
> >> tests that are contributed to Openwhisk).
> >>
> >> Though our first impressions looked good, we found severe degradations
> in
> >> the latency and warm invocation tests afterwards.
> >>
> >> Latency tests dropped to around 50% of the JDK8 results, warm invocation
> >> to a staggering 25%.
> >> These results were stable with using the original and the updated
> Gatling
> >> image and switching to parallelGC on OpenJDK11.
> >>
> >> Since GC does not seem to matter, the degradation might be caused by a
> >> changed
> >> implementation in the OpenJDK 11 (e.g. UUID generation).
> >> We did not perform any deeper research.
> >>
> >> Next we tried to change to use the OpenJ9 JDK11 instead of OpenJDK.
> >> Change in common/scala/Dockerfile:
> >> "
> >> -FROM adoptopenjdk/openjdk11:x86_64-alpine-jdk-11.0.3_7
> >> +FROM
> >> adoptopenjdk/openjdk11-openj9:x86_64-alpine-jdk-11.0.5_10_openj9-0.17.0
> >> "
> >> With OpenJ9 in default configuration all performance metrics were
> looking
> >> good again.
> >> We did not see any degredation anymore.
> >>
> >> In the light of these test results, I herewith would like to start the
> >> discussion to base the JDK update PR
> >> to openJ9 JDK11.
> >>
> >> Kind Regards,
> >> Martin
> >>
>
>
Re: JDK 11
Posted by Martin Henke <ma...@web.de>.
Rodric,
UUID creation was just an example from the past where workarounds were needed to circumvent waiting for enough entropy for the random numbers used for UUID creation. As said we did no further investigation.
Yes. OpenJ9 is Apache licensed and distributed via AdoptOpenJDK. There are no further changes needed.
Regards,
Martin
> Am 28.01.2020 um 18:09 schrieb Rodric Rabbah <ro...@gmail.com>:
>
> Thanks Martin for sharing the results with the community - That's a steep
> performance degradation.
> Open J9 is Apache licensed. Are other changes needed?
>
> You hinted at uuid - do you know that's implicated or a guess?
>
> -r
>
>> On Tue, Jan 28, 2020 at 8:05 AM Martin Henke <ma...@web.de> wrote:
>>
>> Folks,
>>
>> as indicated, we performed performance tests of the OpenJDK upgrade PR
>> https://github.com/apache/openwhisk/pull/4706
>> in our performance test environment (we are using the exact set of Gatling
>> tests that are contributed to Openwhisk).
>>
>> Though our first impressions looked good, we found severe degradations in
>> the latency and warm invocation tests afterwards.
>>
>> Latency tests dropped to around 50% of the JDK8 results, warm invocation
>> to a staggering 25%.
>> These results were stable with using the original and the updated Gatling
>> image and switching to parallelGC on OpenJDK11.
>>
>> Since GC does not seem to matter, the degradation might be caused by a
>> changed
>> implementation in the OpenJDK 11 (e.g. UUID generation).
>> We did not perform any deeper research.
>>
>> Next we tried to change to use the OpenJ9 JDK11 instead of OpenJDK.
>> Change in common/scala/Dockerfile:
>> "
>> -FROM adoptopenjdk/openjdk11:x86_64-alpine-jdk-11.0.3_7
>> +FROM
>> adoptopenjdk/openjdk11-openj9:x86_64-alpine-jdk-11.0.5_10_openj9-0.17.0
>> "
>> With OpenJ9 in default configuration all performance metrics were looking
>> good again.
>> We did not see any degredation anymore.
>>
>> In the light of these test results, I herewith would like to start the
>> discussion to base the JDK update PR
>> to openJ9 JDK11.
>>
>> Kind Regards,
>> Martin
>>
Re: JDK 11
Posted by Rodric Rabbah <ro...@gmail.com>.
Thanks Martin for sharing the results with the community - That's a steep
performance degradation.
Open J9 is Apache licensed. Are other changes needed?
You hinted at uuid - do you know that's implicated or a guess?
-r
On Tue, Jan 28, 2020 at 8:05 AM Martin Henke <ma...@web.de> wrote:
> Folks,
>
> as indicated, we performed performance tests of the OpenJDK upgrade PR
> https://github.com/apache/openwhisk/pull/4706
> in our performance test environment (we are using the exact set of Gatling
> tests that are contributed to Openwhisk).
>
> Though our first impressions looked good, we found severe degradations in
> the latency and warm invocation tests afterwards.
>
> Latency tests dropped to around 50% of the JDK8 results, warm invocation
> to a staggering 25%.
> These results were stable with using the original and the updated Gatling
> image and switching to parallelGC on OpenJDK11.
>
> Since GC does not seem to matter, the degradation might be caused by a
> changed
> implementation in the OpenJDK 11 (e.g. UUID generation).
> We did not perform any deeper research.
>
> Next we tried to change to use the OpenJ9 JDK11 instead of OpenJDK.
> Change in common/scala/Dockerfile:
> "
> -FROM adoptopenjdk/openjdk11:x86_64-alpine-jdk-11.0.3_7
> +FROM
> adoptopenjdk/openjdk11-openj9:x86_64-alpine-jdk-11.0.5_10_openj9-0.17.0
> "
> With OpenJ9 in default configuration all performance metrics were looking
> good again.
> We did not see any degredation anymore.
>
> In the light of these test results, I herewith would like to start the
> discussion to base the JDK update PR
> to openJ9 JDK11.
>
> Kind Regards,
> Martin
>