[Java] New NVMe ByteBuffer API in Java 14, potential super fast appender?

[Matt Sicker <bo...@gmail.com>]

It’s still super early in the JDK 14 development cycle (13 is in release
candidate phase as mentioned earlier), though one of the interesting new
features in development there is a standard API to access NVMe storage as a
ByteBuffer. Not only that, but there’s work being done to support long in
addition to int as the offset in ByteBuffer’s API, so we shouldn’t be
limited to a 2 GB buffer size max, either, which is useful for long running
logs without rotating.

What do you think? Would this work as a v14-specific variant of the
existing mmap appender, or an entirely new appender? I’m asking based on
two issues here: potential API changes that would warrant a second plugin,
or even simply the pain that is running multiple JDK versions in a single
project (already a pain point currently). I’d be disappointed if we have to
start offloading things like that into a separate repo entirely, but I’m
not really sure how to make this work well for both release purposes and
development ones.
-- 
Matt Sicker <bo...@gmail.com>

Re: [Java] New NVMe ByteBuffer API in Java 14, potential super fast appender?

[Ralph Goers <ra...@dslextreme.com>]

FWIW, the memory mapped file appender fails when you run the perf tests. I had to comment it out. I don’t recall what the problem was.

Ralph

> On Aug 25, 2019, at 9:27 AM, Carter Kozak <ck...@ckozak.net> wrote:
> 
> JEP 352: Non-Volatile Mapped Byte Buffers: https://openjdk.java.net/jeps/352
> This sounds interesting! I'm curious what kind of performance impact we could see in practice.
> 
> Regarding the repository configuration, I'm surprised we don't have tooling available to handle our use case, there must be projects which handle it seamlessly. A post from gradle https://blog.gradle.org/mrjars recommends separating these into different JARs in order to use and produce more standard tooling. I don't think that's quite what we want for log4j, however it may be helpful to anchor multi-version support in a single binding location, and request the current runtimes implementation rather than replacing individual components. This may allow us to more easily test each implementation, and make it obvious which code may differ between runtimes.
> 
> -ck
> 
> On Sun, Aug 25, 2019, at 11:34, Matt Sicker wrote:
>> It’s still super early in the JDK 14 development cycle (13 is in release
>> candidate phase as mentioned earlier), though one of the interesting new
>> features in development there is a standard API to access NVMe storage as a
>> ByteBuffer. Not only that, but there’s work being done to support long in
>> addition to int as the offset in ByteBuffer’s API, so we shouldn’t be
>> limited to a 2 GB buffer size max, either, which is useful for long running
>> logs without rotating.
>> 
>> What do you think? Would this work as a v14-specific variant of the
>> existing mmap appender, or an entirely new appender? I’m asking based on
>> two issues here: potential API changes that would warrant a second plugin,
>> or even simply the pain that is running multiple JDK versions in a single
>> project (already a pain point currently). I’d be disappointed if we have to
>> start offloading things like that into a separate repo entirely, but I’m
>> not really sure how to make this work well for both release purposes and
>> development ones.
>> -- 
>> Matt Sicker <bo...@gmail.com>
>> 

Re: [Java] New NVMe ByteBuffer API in Java 14, potential super fast appender?

[Carter Kozak <ck...@ckozak.net>]

JEP 352: Non-Volatile Mapped Byte Buffers: https://openjdk.java.net/jeps/352
This sounds interesting! I'm curious what kind of performance impact we could see in practice.

Regarding the repository configuration, I'm surprised we don't have tooling available to handle our use case, there must be projects which handle it seamlessly. A post from gradle https://blog.gradle.org/mrjars recommends separating these into different JARs in order to use and produce more standard tooling. I don't think that's quite what we want for log4j, however it may be helpful to anchor multi-version support in a single binding location, and request the current runtimes implementation rather than replacing individual components. This may allow us to more easily test each implementation, and make it obvious which code may differ between runtimes.

-ck

On Sun, Aug 25, 2019, at 11:34, Matt Sicker wrote:
> It’s still super early in the JDK 14 development cycle (13 is in release
> candidate phase as mentioned earlier), though one of the interesting new
> features in development there is a standard API to access NVMe storage as a
> ByteBuffer. Not only that, but there’s work being done to support long in
> addition to int as the offset in ByteBuffer’s API, so we shouldn’t be
> limited to a 2 GB buffer size max, either, which is useful for long running
> logs without rotating.
> 
> What do you think? Would this work as a v14-specific variant of the
> existing mmap appender, or an entirely new appender? I’m asking based on
> two issues here: potential API changes that would warrant a second plugin,
> or even simply the pain that is running multiple JDK versions in a single
> project (already a pain point currently). I’d be disappointed if we have to
> start offloading things like that into a separate repo entirely, but I’m
> not really sure how to make this work well for both release purposes and
> development ones.
> -- 
> Matt Sicker <bo...@gmail.com>
>