You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pekko.apache.org by Oliver Trosien <ol...@pocket-design.de> on 2023/11/14 20:51:52 UTC
Re: [DISCUSS] Be included in Techempower Framework Benchmark runs
Hi,
another quick update regarding the TechEmpower benchmark(eting)
contribution. While the final blog post is yet to come, the benchmark data
is already available (1), and overall pekko-http got a decent overall
score, ending up somewhere in the middle of the field (slightly above Akka
of course :). The screenshot attached shows maximum RPS for the following
test types (2): JSON: 197k, 1-query: 141k, 20-query: 10.5k, Fortunes: 96k,
Updates: 1.8k and Plaintext: 2.6M.
(1)
https://www.techempower.com/benchmarks/#hw=ph&test=composite§ion=data-r22
(2)
https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview
If we wanted to improve the score, there's for sure some low-hanging fruit,
including fixing unreasonable parallelism on the DB updates test (multiple
parallel update queries for each request without any transactional context)
which I intentionally didn't touch because my primary goal was a direct
comparison to the Akka implementation. After that I would kindly ask the
project to take over the annual polishing, including upgrades and fixes.
Cheers,
Oliver
Am Mo., 4. Sept. 2023 um 08:46 Uhr schrieb Oliver Trosien <
oli@pocket-design.de>:
> Hi,
>
> quick update: pekko-http got merged into the test suite (1), and a first
> preliminary test round (2) shows it's on par with akka-http (as expected -
> we didn't change the logic of the code itself). I'll post another update
> when the final numbers are ready.
>
> Oliver
>
> (1)
> https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Scala/pekko-http
> (2)
> https://www.techempower.com/benchmarks/#section=test&runid=074e8a70-d6fb-4f10-82f3-43e57c0965b5&test=json&l=zik0zj-35b&hw=ph&p=zik0z3-zik0zj-zik0zj-zik0zj-v2qiv3-1
>
> Am Mi., 23. Aug. 2023 um 14:57 Uhr schrieb PJ Fanning <fanningpj@gmail.com
> >:
>
>> I think we should be able to keep track of this in future.
>>
>> On Wed, 23 Aug 2023 at 13:20, Oliver Trosien <ol...@pocket-design.de>
>> wrote:
>> >
>> > Hi PJ,
>> >
>> > sure, I can do the initial submission. My main point is that it only
>> makes
>> > sense if the project is willing to take over the long-term maintainence.
>> >
>> > Oliver
>> >
>> >
>> > Am Mi., 23. Aug. 2023 um 11:15 Uhr schrieb PJ Fanning <
>> fanningpj@apache.org
>> > >:
>> >
>> > > Hi Olivier,
>> > >
>> > > Thanks for sharing your code. Are you planning on submitting that to
>> > > TechEmpower/FrameworkBenchmarks ?
>> > >
>> > > On Wed, 23 Aug 2023 at 07:13, Oliver Trosien <ol...@pocket-design.de>
>> wrote:
>> > > >
>> > > > Hi,
>> > > >
>> > > > fair point. I have working code here (just the plain variant, not
>> the one
>> > > > with slick from Lightbend),
>> > > > which is as you pointed out basically changing the imports plus
>> doing a
>> > > bit
>> > > > of cleanup:
>> > > > https://github.com/otrosien/FrameworkBenchmarks/pull/1/
>> > > >
>> > > > Main difference to the akka version is that it runs on JDK17. Feel
>> free
>> > > to
>> > > > use this as a baseline.
>> > > >
>> > > > Regards,
>> > > > Oliver
>> > > >
>> > > >
>> > > > Am Di., 22. Aug. 2023 um 19:31 Uhr schrieb PJ Fanning <
>> > > fanningpj@apache.org
>> > > > >:
>> > > >
>> > > > > Thanks Olivier. We have Pekko and Pekko HTTP releases that could
>> be
>> > > > > benchmarked. The Akka code could probably be pretty easily forked
>> and
>> > > > > used to create Pekko equivalents (mainly just changing the
>> imports and
>> > > > > possibly some config names).
>> > > > >
>> > > > > I wouldn't expect there to be much difference between the Akka and
>> > > > > Pekko performance numbers.
>> > > > >
>> > > > > If someone has time to do the initial setup, it should be fairly
>> > > > > straightforward for us to maintain it going forward.
>> > > > >
>> > > > > For my part, I'm more interested in getting all the Pekko modules
>> > > > > released and helping to get widely used Akka ecosystem libs to
>> support
>> > > > > Pekko too (or to create Pekko based forks).
>> > > > >
>> > > > > On Tue, 22 Aug 2023 at 18:10, Oliver Trosien <
>> oli@pocket-design.de>
>> > > wrote:
>> > > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > I wanted to ask the project team if they want to maintain an
>> > > > > implementation
>> > > > > > of the framework benchmark run for pekko (you can see the akka
>> > > > > > implementation here: (1))
>> > > > > >
>> > > > > > I would totally understand if you'd object this kind of
>> > > "Benchmarketing",
>> > > > > > but it still would give the project some additional visibility.
>> I do
>> > > > > have a
>> > > > > > local version which basically is a copy of the akka one. While
>> there
>> > > are
>> > > > > > some framework submissions driven by individuals, they usually
>> don't
>> > > have
>> > > > > > insights or long-term interest in maintaining these submissions
>> and
>> > > as a
>> > > > > > result performance regressions can shed bad light on the
>> project (2).
>> > > > > >
>> > > > > > They want to close submissions to the next round soon (3) - and
>> it
>> > > > > usually
>> > > > > > only happens once a year, so we would need to be quick to get
>> > > included.
>> > > > > > WDYT?
>> > > > > >
>> > > > > > Regards,
>> > > > > > Oliver
>> > > > > >
>> > > > > > (1)
>> > > > > >
>> > > > >
>> > >
>> https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Scala/akka-http
>> > > > > > (2)
>> > > > > >
>> > > > >
>> > >
>> https://github.com/TechEmpower/FrameworkBenchmarks/issues/7475#issuecomment-1607479108
>> > > > > > (3)
>> > > > > >
>> > > > >
>> > >
>> https://github.com/TechEmpower/FrameworkBenchmarks/issues/7475#issuecomment-1684087157
>> > > > >
>> > > > >
>> ---------------------------------------------------------------------
>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
>> > > > >
>> > > > >
>> > >
>> > > ---------------------------------------------------------------------
>> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>> > > For additional commands, e-mail: dev-help@pekko.apache.org
>> > >
>> > >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>> For additional commands, e-mail: dev-help@pekko.apache.org
>>
>>
Re: [DISCUSS] Be included in Techempower Framework Benchmark runs
Posted by Matthew de Detrich <ma...@aiven.io.INVALID>.
> Regarding the JDK version, we're already on JDK17 and I wouldn't recommend
downgrading.
For the relevant performance optimizations that the Scala inliner solves,
the modt critical ones are in JDK 21/GraalVM enterprise (which is now
"free"), i.e. see
https://docs.oracle.com/en/graalvm/jdk/21/docs/graalvm-as-a-platform/language-implementation-framework/splitting/ReportingPolymorphism/
As an example
Unfortunately it's also the case that a lot of users still use JDK 1.8 (and
its **those** users who ironically care about performance than most since
they are enterprise users). Not sure if this changes anything but it's food
for thought.
> The main question for me would be what's more important to the
project: a) keeping a direct comparison with akka or b) showing a good
overall score. If a) and b) then we'd need to apply code changes to both
pekko and akka submissions
Selfishly I want both a) and b) because even aside from the Scala 2
inliner, there were seperate performance improvements or 1.1.x series which
I'd like to see compared
On Sat, 18 Nov 2023, 12:05 Oliver Trosien, <ol...@pocket-design.de> wrote:
> Regarding the JDK version, we're already on JDK17 and I wouldn't recommend
> downgrading. The main question for me would be what's more important to the
> project: a) keeping a direct comparison with akka or b) showing a good
> overall score. If a) and b) then we'd need to apply code changes to both
> pekko and akka submissions. One example showing how important it is to
> understand the benchmark physics is the akka-http-slick-postgres
> submission, which even for a single-query per request is performing 50% max
> RPS of the blocking implementation in akka-http, and is failing the updates
> test when slick starts rejecting tasks based on max queue length (*).
>
> Finally, some frameworks actually use these benchmarks to find and dissolve
> bottlenecks in hotspots of their code, see Netty in the latest blog post
> which btw also features pekko as new submission (**).
>
> Oliver
>
> (*)
>
> https://tfb-status.techempower.com/unzip/results.2023-11-12-22-36-41-240.zip/results/20231106203152/akka-http-slick-postgres/run/akka-http-slick-postgres.log
>
> (**)
> https://www.techempower.com/blog/2023/11/15/framework-benchmarks-round-22/
>
>
> Am Sa., 18. Nov. 2023 um 10:12 Uhr schrieb Matthew de Detrich
> <ma...@aiven.io.invalid>:
>
> > I am not ontop of this too much, but one major feature that is planned to
> > be added specifically in Pekko 1.1.x release (and also in Pekko modules
> > 1.1.x) is enabling the Scala 2 inliner which should provide at least some
> > performance improvement (improvements would be more noticeable on early
> > JDK's, i.e. 8,11) so maybe it's a good idea to have a separate benchmarks
> > specifically for the 1.1.x series?
> >
> > On Tue, Nov 14, 2023 at 9:52 PM Oliver Trosien <ol...@pocket-design.de>
> > wrote:
> >
> > > Hi,
> > >
> > > another quick update regarding the TechEmpower benchmark(eting)
> > > contribution. While the final blog post is yet to come, the benchmark
> > data
> > > is already available (1), and overall pekko-http got a decent overall
> > > score, ending up somewhere in the middle of the field (slightly above
> > Akka
> > > of course :). The screenshot attached shows maximum RPS for the
> following
> > > test types (2): JSON: 197k, 1-query: 141k, 20-query: 10.5k, Fortunes:
> > 96k,
> > > Updates: 1.8k and Plaintext: 2.6M.
> > >
> > > (1)
> > >
> >
> https://www.techempower.com/benchmarks/#hw=ph&test=composite§ion=data-r22
> > > (2)
> > >
> >
> https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview
> > >
> > > If we wanted to improve the score, there's for sure some low-hanging
> > > fruit, including fixing unreasonable parallelism on the DB updates test
> > > (multiple parallel update queries for each request without any
> > > transactional context) which I intentionally didn't touch because my
> > > primary goal was a direct comparison to the Akka implementation. After
> > that
> > > I would kindly ask the project to take over the annual polishing,
> > including
> > > upgrades and fixes.
> > >
> > > Cheers,
> > > Oliver
> > >
> > >
> > >
> > > Am Mo., 4. Sept. 2023 um 08:46 Uhr schrieb Oliver Trosien <
> > > oli@pocket-design.de>:
> > >
> > >> Hi,
> > >>
> > >> quick update: pekko-http got merged into the test suite (1), and a
> first
> > >> preliminary test round (2) shows it's on par with akka-http (as
> > expected -
> > >> we didn't change the logic of the code itself). I'll post another
> update
> > >> when the final numbers are ready.
> > >>
> > >> Oliver
> > >>
> > >> (1)
> > >>
> >
> https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Scala/pekko-http
> > >> (2)
> > >>
> >
> https://www.techempower.com/benchmarks/#section=test&runid=074e8a70-d6fb-4f10-82f3-43e57c0965b5&test=json&l=zik0zj-35b&hw=ph&p=zik0z3-zik0zj-zik0zj-zik0zj-v2qiv3-1
> > >>
> > >> Am Mi., 23. Aug. 2023 um 14:57 Uhr schrieb PJ Fanning <
> > >> fanningpj@gmail.com>:
> > >>
> > >>> I think we should be able to keep track of this in future.
> > >>>
> > >>> On Wed, 23 Aug 2023 at 13:20, Oliver Trosien <ol...@pocket-design.de>
> > >>> wrote:
> > >>> >
> > >>> > Hi PJ,
> > >>> >
> > >>> > sure, I can do the initial submission. My main point is that it
> only
> > >>> makes
> > >>> > sense if the project is willing to take over the long-term
> > >>> maintainence.
> > >>> >
> > >>> > Oliver
> > >>> >
> > >>> >
> > >>> > Am Mi., 23. Aug. 2023 um 11:15 Uhr schrieb PJ Fanning <
> > >>> fanningpj@apache.org
> > >>> > >:
> > >>> >
> > >>> > > Hi Olivier,
> > >>> > >
> > >>> > > Thanks for sharing your code. Are you planning on submitting that
> > to
> > >>> > > TechEmpower/FrameworkBenchmarks ?
> > >>> > >
> > >>> > > On Wed, 23 Aug 2023 at 07:13, Oliver Trosien <
> oli@pocket-design.de
> > >
> > >>> wrote:
> > >>> > > >
> > >>> > > > Hi,
> > >>> > > >
> > >>> > > > fair point. I have working code here (just the plain variant,
> not
> > >>> the one
> > >>> > > > with slick from Lightbend),
> > >>> > > > which is as you pointed out basically changing the imports plus
> > >>> doing a
> > >>> > > bit
> > >>> > > > of cleanup:
> > >>> > > > https://github.com/otrosien/FrameworkBenchmarks/pull/1/
> > >>> > > >
> > >>> > > > Main difference to the akka version is that it runs on JDK17.
> > Feel
> > >>> free
> > >>> > > to
> > >>> > > > use this as a baseline.
> > >>> > > >
> > >>> > > > Regards,
> > >>> > > > Oliver
> > >>> > > >
> > >>> > > >
> > >>> > > > Am Di., 22. Aug. 2023 um 19:31 Uhr schrieb PJ Fanning <
> > >>> > > fanningpj@apache.org
> > >>> > > > >:
> > >>> > > >
> > >>> > > > > Thanks Olivier. We have Pekko and Pekko HTTP releases that
> > could
> > >>> be
> > >>> > > > > benchmarked. The Akka code could probably be pretty easily
> > >>> forked and
> > >>> > > > > used to create Pekko equivalents (mainly just changing the
> > >>> imports and
> > >>> > > > > possibly some config names).
> > >>> > > > >
> > >>> > > > > I wouldn't expect there to be much difference between the
> Akka
> > >>> and
> > >>> > > > > Pekko performance numbers.
> > >>> > > > >
> > >>> > > > > If someone has time to do the initial setup, it should be
> > fairly
> > >>> > > > > straightforward for us to maintain it going forward.
> > >>> > > > >
> > >>> > > > > For my part, I'm more interested in getting all the Pekko
> > modules
> > >>> > > > > released and helping to get widely used Akka ecosystem libs
> to
> > >>> support
> > >>> > > > > Pekko too (or to create Pekko based forks).
> > >>> > > > >
> > >>> > > > > On Tue, 22 Aug 2023 at 18:10, Oliver Trosien <
> > >>> oli@pocket-design.de>
> > >>> > > wrote:
> > >>> > > > > >
> > >>> > > > > > Hi,
> > >>> > > > > >
> > >>> > > > > > I wanted to ask the project team if they want to maintain
> an
> > >>> > > > > implementation
> > >>> > > > > > of the framework benchmark run for pekko (you can see the
> > akka
> > >>> > > > > > implementation here: (1))
> > >>> > > > > >
> > >>> > > > > > I would totally understand if you'd object this kind of
> > >>> > > "Benchmarketing",
> > >>> > > > > > but it still would give the project some additional
> > >>> visibility. I do
> > >>> > > > > have a
> > >>> > > > > > local version which basically is a copy of the akka one.
> > While
> > >>> there
> > >>> > > are
> > >>> > > > > > some framework submissions driven by individuals, they
> > usually
> > >>> don't
> > >>> > > have
> > >>> > > > > > insights or long-term interest in maintaining these
> > >>> submissions and
> > >>> > > as a
> > >>> > > > > > result performance regressions can shed bad light on the
> > >>> project (2).
> > >>> > > > > >
> > >>> > > > > > They want to close submissions to the next round soon (3) -
> > >>> and it
> > >>> > > > > usually
> > >>> > > > > > only happens once a year, so we would need to be quick to
> get
> > >>> > > included.
> > >>> > > > > > WDYT?
> > >>> > > > > >
> > >>> > > > > > Regards,
> > >>> > > > > > Oliver
> > >>> > > > > >
> > >>> > > > > > (1)
> > >>> > > > > >
> > >>> > > > >
> > >>> > >
> > >>>
> >
> https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Scala/akka-http
> > >>> > > > > > (2)
> > >>> > > > > >
> > >>> > > > >
> > >>> > >
> > >>>
> >
> https://github.com/TechEmpower/FrameworkBenchmarks/issues/7475#issuecomment-1607479108
> > >>> > > > > > (3)
> > >>> > > > > >
> > >>> > > > >
> > >>> > >
> > >>>
> >
> https://github.com/TechEmpower/FrameworkBenchmarks/issues/7475#issuecomment-1684087157
> > >>> > > > >
> > >>> > > > >
> > >>> ---------------------------------------------------------------------
> > >>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > >>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >>> > > > >
> > >>> > > > >
> > >>> > >
> > >>> > >
> > ---------------------------------------------------------------------
> > >>> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > >>> > > For additional commands, e-mail: dev-help@pekko.apache.org
> > >>> > >
> > >>> > >
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > >>> For additional commands, e-mail: dev-help@pekko.apache.org
> > >>>
> > >>>
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > > For additional commands, e-mail: dev-help@pekko.apache.org
> >
> >
> >
> > --
> >
> > Matthew de Detrich
> >
> > *Aiven Deutschland GmbH*
> >
> > Immanuelkirchstraße 26, 10405 Berlin
> >
> > Alexanderufer 3-7, 10117 Berlin
> >
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> >
> > *m:* +491603708037
> >
> > *w:* aiven.io *e:* matthew.dedetrich@aiven.io
> >
>
>
Re: [DISCUSS] Be included in Techempower Framework Benchmark runs
Posted by Oliver Trosien <ol...@pocket-design.de>.
Regarding the JDK version, we're already on JDK17 and I wouldn't recommend
downgrading. The main question for me would be what's more important to the
project: a) keeping a direct comparison with akka or b) showing a good
overall score. If a) and b) then we'd need to apply code changes to both
pekko and akka submissions. One example showing how important it is to
understand the benchmark physics is the akka-http-slick-postgres
submission, which even for a single-query per request is performing 50% max
RPS of the blocking implementation in akka-http, and is failing the updates
test when slick starts rejecting tasks based on max queue length (*).
Finally, some frameworks actually use these benchmarks to find and dissolve
bottlenecks in hotspots of their code, see Netty in the latest blog post
which btw also features pekko as new submission (**).
Oliver
(*)
https://tfb-status.techempower.com/unzip/results.2023-11-12-22-36-41-240.zip/results/20231106203152/akka-http-slick-postgres/run/akka-http-slick-postgres.log
(**)
https://www.techempower.com/blog/2023/11/15/framework-benchmarks-round-22/
Am Sa., 18. Nov. 2023 um 10:12 Uhr schrieb Matthew de Detrich
<ma...@aiven.io.invalid>:
> I am not ontop of this too much, but one major feature that is planned to
> be added specifically in Pekko 1.1.x release (and also in Pekko modules
> 1.1.x) is enabling the Scala 2 inliner which should provide at least some
> performance improvement (improvements would be more noticeable on early
> JDK's, i.e. 8,11) so maybe it's a good idea to have a separate benchmarks
> specifically for the 1.1.x series?
>
> On Tue, Nov 14, 2023 at 9:52 PM Oliver Trosien <ol...@pocket-design.de>
> wrote:
>
> > Hi,
> >
> > another quick update regarding the TechEmpower benchmark(eting)
> > contribution. While the final blog post is yet to come, the benchmark
> data
> > is already available (1), and overall pekko-http got a decent overall
> > score, ending up somewhere in the middle of the field (slightly above
> Akka
> > of course :). The screenshot attached shows maximum RPS for the following
> > test types (2): JSON: 197k, 1-query: 141k, 20-query: 10.5k, Fortunes:
> 96k,
> > Updates: 1.8k and Plaintext: 2.6M.
> >
> > (1)
> >
> https://www.techempower.com/benchmarks/#hw=ph&test=composite§ion=data-r22
> > (2)
> >
> https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview
> >
> > If we wanted to improve the score, there's for sure some low-hanging
> > fruit, including fixing unreasonable parallelism on the DB updates test
> > (multiple parallel update queries for each request without any
> > transactional context) which I intentionally didn't touch because my
> > primary goal was a direct comparison to the Akka implementation. After
> that
> > I would kindly ask the project to take over the annual polishing,
> including
> > upgrades and fixes.
> >
> > Cheers,
> > Oliver
> >
> >
> >
> > Am Mo., 4. Sept. 2023 um 08:46 Uhr schrieb Oliver Trosien <
> > oli@pocket-design.de>:
> >
> >> Hi,
> >>
> >> quick update: pekko-http got merged into the test suite (1), and a first
> >> preliminary test round (2) shows it's on par with akka-http (as
> expected -
> >> we didn't change the logic of the code itself). I'll post another update
> >> when the final numbers are ready.
> >>
> >> Oliver
> >>
> >> (1)
> >>
> https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Scala/pekko-http
> >> (2)
> >>
> https://www.techempower.com/benchmarks/#section=test&runid=074e8a70-d6fb-4f10-82f3-43e57c0965b5&test=json&l=zik0zj-35b&hw=ph&p=zik0z3-zik0zj-zik0zj-zik0zj-v2qiv3-1
> >>
> >> Am Mi., 23. Aug. 2023 um 14:57 Uhr schrieb PJ Fanning <
> >> fanningpj@gmail.com>:
> >>
> >>> I think we should be able to keep track of this in future.
> >>>
> >>> On Wed, 23 Aug 2023 at 13:20, Oliver Trosien <ol...@pocket-design.de>
> >>> wrote:
> >>> >
> >>> > Hi PJ,
> >>> >
> >>> > sure, I can do the initial submission. My main point is that it only
> >>> makes
> >>> > sense if the project is willing to take over the long-term
> >>> maintainence.
> >>> >
> >>> > Oliver
> >>> >
> >>> >
> >>> > Am Mi., 23. Aug. 2023 um 11:15 Uhr schrieb PJ Fanning <
> >>> fanningpj@apache.org
> >>> > >:
> >>> >
> >>> > > Hi Olivier,
> >>> > >
> >>> > > Thanks for sharing your code. Are you planning on submitting that
> to
> >>> > > TechEmpower/FrameworkBenchmarks ?
> >>> > >
> >>> > > On Wed, 23 Aug 2023 at 07:13, Oliver Trosien <oli@pocket-design.de
> >
> >>> wrote:
> >>> > > >
> >>> > > > Hi,
> >>> > > >
> >>> > > > fair point. I have working code here (just the plain variant, not
> >>> the one
> >>> > > > with slick from Lightbend),
> >>> > > > which is as you pointed out basically changing the imports plus
> >>> doing a
> >>> > > bit
> >>> > > > of cleanup:
> >>> > > > https://github.com/otrosien/FrameworkBenchmarks/pull/1/
> >>> > > >
> >>> > > > Main difference to the akka version is that it runs on JDK17.
> Feel
> >>> free
> >>> > > to
> >>> > > > use this as a baseline.
> >>> > > >
> >>> > > > Regards,
> >>> > > > Oliver
> >>> > > >
> >>> > > >
> >>> > > > Am Di., 22. Aug. 2023 um 19:31 Uhr schrieb PJ Fanning <
> >>> > > fanningpj@apache.org
> >>> > > > >:
> >>> > > >
> >>> > > > > Thanks Olivier. We have Pekko and Pekko HTTP releases that
> could
> >>> be
> >>> > > > > benchmarked. The Akka code could probably be pretty easily
> >>> forked and
> >>> > > > > used to create Pekko equivalents (mainly just changing the
> >>> imports and
> >>> > > > > possibly some config names).
> >>> > > > >
> >>> > > > > I wouldn't expect there to be much difference between the Akka
> >>> and
> >>> > > > > Pekko performance numbers.
> >>> > > > >
> >>> > > > > If someone has time to do the initial setup, it should be
> fairly
> >>> > > > > straightforward for us to maintain it going forward.
> >>> > > > >
> >>> > > > > For my part, I'm more interested in getting all the Pekko
> modules
> >>> > > > > released and helping to get widely used Akka ecosystem libs to
> >>> support
> >>> > > > > Pekko too (or to create Pekko based forks).
> >>> > > > >
> >>> > > > > On Tue, 22 Aug 2023 at 18:10, Oliver Trosien <
> >>> oli@pocket-design.de>
> >>> > > wrote:
> >>> > > > > >
> >>> > > > > > Hi,
> >>> > > > > >
> >>> > > > > > I wanted to ask the project team if they want to maintain an
> >>> > > > > implementation
> >>> > > > > > of the framework benchmark run for pekko (you can see the
> akka
> >>> > > > > > implementation here: (1))
> >>> > > > > >
> >>> > > > > > I would totally understand if you'd object this kind of
> >>> > > "Benchmarketing",
> >>> > > > > > but it still would give the project some additional
> >>> visibility. I do
> >>> > > > > have a
> >>> > > > > > local version which basically is a copy of the akka one.
> While
> >>> there
> >>> > > are
> >>> > > > > > some framework submissions driven by individuals, they
> usually
> >>> don't
> >>> > > have
> >>> > > > > > insights or long-term interest in maintaining these
> >>> submissions and
> >>> > > as a
> >>> > > > > > result performance regressions can shed bad light on the
> >>> project (2).
> >>> > > > > >
> >>> > > > > > They want to close submissions to the next round soon (3) -
> >>> and it
> >>> > > > > usually
> >>> > > > > > only happens once a year, so we would need to be quick to get
> >>> > > included.
> >>> > > > > > WDYT?
> >>> > > > > >
> >>> > > > > > Regards,
> >>> > > > > > Oliver
> >>> > > > > >
> >>> > > > > > (1)
> >>> > > > > >
> >>> > > > >
> >>> > >
> >>>
> https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Scala/akka-http
> >>> > > > > > (2)
> >>> > > > > >
> >>> > > > >
> >>> > >
> >>>
> https://github.com/TechEmpower/FrameworkBenchmarks/issues/7475#issuecomment-1607479108
> >>> > > > > > (3)
> >>> > > > > >
> >>> > > > >
> >>> > >
> >>>
> https://github.com/TechEmpower/FrameworkBenchmarks/issues/7475#issuecomment-1684087157
> >>> > > > >
> >>> > > > >
> >>> ---------------------------------------------------------------------
> >>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> >>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
> >>> > > > >
> >>> > > > >
> >>> > >
> >>> > >
> ---------------------------------------------------------------------
> >>> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> >>> > > For additional commands, e-mail: dev-help@pekko.apache.org
> >>> > >
> >>> > >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> >>> For additional commands, e-mail: dev-help@pekko.apache.org
> >>>
> >>>
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> > For additional commands, e-mail: dev-help@pekko.apache.org
>
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Alexanderufer 3-7, 10117 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetrich@aiven.io
>
Re: [DISCUSS] Be included in Techempower Framework Benchmark runs
Posted by Matthew de Detrich <ma...@aiven.io.INVALID>.
I am not ontop of this too much, but one major feature that is planned to
be added specifically in Pekko 1.1.x release (and also in Pekko modules
1.1.x) is enabling the Scala 2 inliner which should provide at least some
performance improvement (improvements would be more noticeable on early
JDK's, i.e. 8,11) so maybe it's a good idea to have a separate benchmarks
specifically for the 1.1.x series?
On Tue, Nov 14, 2023 at 9:52 PM Oliver Trosien <ol...@pocket-design.de> wrote:
> Hi,
>
> another quick update regarding the TechEmpower benchmark(eting)
> contribution. While the final blog post is yet to come, the benchmark data
> is already available (1), and overall pekko-http got a decent overall
> score, ending up somewhere in the middle of the field (slightly above Akka
> of course :). The screenshot attached shows maximum RPS for the following
> test types (2): JSON: 197k, 1-query: 141k, 20-query: 10.5k, Fortunes: 96k,
> Updates: 1.8k and Plaintext: 2.6M.
>
> (1)
> https://www.techempower.com/benchmarks/#hw=ph&test=composite§ion=data-r22
> (2)
> https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview
>
> If we wanted to improve the score, there's for sure some low-hanging
> fruit, including fixing unreasonable parallelism on the DB updates test
> (multiple parallel update queries for each request without any
> transactional context) which I intentionally didn't touch because my
> primary goal was a direct comparison to the Akka implementation. After that
> I would kindly ask the project to take over the annual polishing, including
> upgrades and fixes.
>
> Cheers,
> Oliver
>
>
>
> Am Mo., 4. Sept. 2023 um 08:46 Uhr schrieb Oliver Trosien <
> oli@pocket-design.de>:
>
>> Hi,
>>
>> quick update: pekko-http got merged into the test suite (1), and a first
>> preliminary test round (2) shows it's on par with akka-http (as expected -
>> we didn't change the logic of the code itself). I'll post another update
>> when the final numbers are ready.
>>
>> Oliver
>>
>> (1)
>> https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Scala/pekko-http
>> (2)
>> https://www.techempower.com/benchmarks/#section=test&runid=074e8a70-d6fb-4f10-82f3-43e57c0965b5&test=json&l=zik0zj-35b&hw=ph&p=zik0z3-zik0zj-zik0zj-zik0zj-v2qiv3-1
>>
>> Am Mi., 23. Aug. 2023 um 14:57 Uhr schrieb PJ Fanning <
>> fanningpj@gmail.com>:
>>
>>> I think we should be able to keep track of this in future.
>>>
>>> On Wed, 23 Aug 2023 at 13:20, Oliver Trosien <ol...@pocket-design.de>
>>> wrote:
>>> >
>>> > Hi PJ,
>>> >
>>> > sure, I can do the initial submission. My main point is that it only
>>> makes
>>> > sense if the project is willing to take over the long-term
>>> maintainence.
>>> >
>>> > Oliver
>>> >
>>> >
>>> > Am Mi., 23. Aug. 2023 um 11:15 Uhr schrieb PJ Fanning <
>>> fanningpj@apache.org
>>> > >:
>>> >
>>> > > Hi Olivier,
>>> > >
>>> > > Thanks for sharing your code. Are you planning on submitting that to
>>> > > TechEmpower/FrameworkBenchmarks ?
>>> > >
>>> > > On Wed, 23 Aug 2023 at 07:13, Oliver Trosien <ol...@pocket-design.de>
>>> wrote:
>>> > > >
>>> > > > Hi,
>>> > > >
>>> > > > fair point. I have working code here (just the plain variant, not
>>> the one
>>> > > > with slick from Lightbend),
>>> > > > which is as you pointed out basically changing the imports plus
>>> doing a
>>> > > bit
>>> > > > of cleanup:
>>> > > > https://github.com/otrosien/FrameworkBenchmarks/pull/1/
>>> > > >
>>> > > > Main difference to the akka version is that it runs on JDK17. Feel
>>> free
>>> > > to
>>> > > > use this as a baseline.
>>> > > >
>>> > > > Regards,
>>> > > > Oliver
>>> > > >
>>> > > >
>>> > > > Am Di., 22. Aug. 2023 um 19:31 Uhr schrieb PJ Fanning <
>>> > > fanningpj@apache.org
>>> > > > >:
>>> > > >
>>> > > > > Thanks Olivier. We have Pekko and Pekko HTTP releases that could
>>> be
>>> > > > > benchmarked. The Akka code could probably be pretty easily
>>> forked and
>>> > > > > used to create Pekko equivalents (mainly just changing the
>>> imports and
>>> > > > > possibly some config names).
>>> > > > >
>>> > > > > I wouldn't expect there to be much difference between the Akka
>>> and
>>> > > > > Pekko performance numbers.
>>> > > > >
>>> > > > > If someone has time to do the initial setup, it should be fairly
>>> > > > > straightforward for us to maintain it going forward.
>>> > > > >
>>> > > > > For my part, I'm more interested in getting all the Pekko modules
>>> > > > > released and helping to get widely used Akka ecosystem libs to
>>> support
>>> > > > > Pekko too (or to create Pekko based forks).
>>> > > > >
>>> > > > > On Tue, 22 Aug 2023 at 18:10, Oliver Trosien <
>>> oli@pocket-design.de>
>>> > > wrote:
>>> > > > > >
>>> > > > > > Hi,
>>> > > > > >
>>> > > > > > I wanted to ask the project team if they want to maintain an
>>> > > > > implementation
>>> > > > > > of the framework benchmark run for pekko (you can see the akka
>>> > > > > > implementation here: (1))
>>> > > > > >
>>> > > > > > I would totally understand if you'd object this kind of
>>> > > "Benchmarketing",
>>> > > > > > but it still would give the project some additional
>>> visibility. I do
>>> > > > > have a
>>> > > > > > local version which basically is a copy of the akka one. While
>>> there
>>> > > are
>>> > > > > > some framework submissions driven by individuals, they usually
>>> don't
>>> > > have
>>> > > > > > insights or long-term interest in maintaining these
>>> submissions and
>>> > > as a
>>> > > > > > result performance regressions can shed bad light on the
>>> project (2).
>>> > > > > >
>>> > > > > > They want to close submissions to the next round soon (3) -
>>> and it
>>> > > > > usually
>>> > > > > > only happens once a year, so we would need to be quick to get
>>> > > included.
>>> > > > > > WDYT?
>>> > > > > >
>>> > > > > > Regards,
>>> > > > > > Oliver
>>> > > > > >
>>> > > > > > (1)
>>> > > > > >
>>> > > > >
>>> > >
>>> https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Scala/akka-http
>>> > > > > > (2)
>>> > > > > >
>>> > > > >
>>> > >
>>> https://github.com/TechEmpower/FrameworkBenchmarks/issues/7475#issuecomment-1607479108
>>> > > > > > (3)
>>> > > > > >
>>> > > > >
>>> > >
>>> https://github.com/TechEmpower/FrameworkBenchmarks/issues/7475#issuecomment-1684087157
>>> > > > >
>>> > > > >
>>> ---------------------------------------------------------------------
>>> > > > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>> > > > > For additional commands, e-mail: dev-help@pekko.apache.org
>>> > > > >
>>> > > > >
>>> > >
>>> > > ---------------------------------------------------------------------
>>> > > To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>> > > For additional commands, e-mail: dev-help@pekko.apache.org
>>> > >
>>> > >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
>>> For additional commands, e-mail: dev-help@pekko.apache.org
>>>
>>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pekko.apache.org
> For additional commands, e-mail: dev-help@pekko.apache.org
--
Matthew de Detrich
*Aiven Deutschland GmbH*
Immanuelkirchstraße 26, 10405 Berlin
Alexanderufer 3-7, 10117 Berlin
Amtsgericht Charlottenburg, HRB 209739 B
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
*m:* +491603708037
*w:* aiven.io *e:* matthew.dedetrich@aiven.io