io_uring backend for NIO

[Martin Grigorov <mg...@apache.org>]

Hi devs,

Maybe you have also noticed some noise in the web about io_uring
<https://kernel.dk/io_uring.pdf> - the better epoll.

Pros:
- less syscalls
- no copying of data between userland and kernel

Cons:
- it seems rather unstable to me.
Many articles say that io_uring should be more stable since kernel v5.6.x
but in my experience there are many changes in each kernel versions

https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-will-revolutionize-programming-in-linux/
is a good article explaining the benefits.

Recently Netty project released 0.0.1-SNAPSHOT (
https://netty.io/news/2020/11/16/io_uring-0-0-1-Final.html) and
0.0.2-SNAPSHOT with some nice numbers:
- epoll

Speed: 80820 request/sec, 80820 response/sec
Requests: 2424626
Responses: 2424625

- io_uring

Speed: 267371 request/sec, 267370 response/sec
Requests: 8021137
Responses: 8021128

I've tried it with my HTTP2 comparison tests
(https://martin-grigorov.medium.com/comparing-the-performance-of-several-http2-web-servers-fb5f3787532a)

and the results were better but not so dramatic:

- epoll

-- x86_64: 24688 reqs/sec

-- aarch64: 25344

- uring

-- x86_64: 26731

-- aarch64: 28443


So I thought: What about introducing a Uring Protocol/Connector ?!
Since Netty uses JNI

to talk to io_uring C APIs it should look something like the AprConnector.

But soon after I realized that it would be much more reusable if implemented

as  java.nio.channels.spi.SelectorProvider. This way any project (not
just Tomcat) can

make use of it.


Here I wanted to poll what other devs think about this idea ?

Does that sound interesting to you ? Or we should leave it to the JDK developers

to do it for JDK XY (I guess this could be as early as Java 18 since 17 will

be a LTS and most probably the current epoll based impl won't be replaced so

soon).

I don't suggest to put it in Tomcat's code, so the dependency on JNI

should not be considered as a stopper.


I have some very early attempts implementing it with Netty's JNI code and

with Java Panama's jextract, but I've faced issues with both.


Please encourage/discourage me to continue digging in this! :-)


Regards,

Martin

Re: io_uring backend for NIO

[Rémy Maucherat <re...@apache.org>]

On Mon, Dec 7, 2020 at 3:12 PM Martin Grigorov <mg...@apache.org> wrote:

> Hi devs,
>
> Maybe you have also noticed some noise in the web about io_uring
> <https://kernel.dk/io_uring.pdf> - the better epoll.
>
> Pros:
> - less syscalls
> - no copying of data between userland and kernel
>
> Cons:
> - it seems rather unstable to me.
> Many articles say that io_uring should be more stable since kernel v5.6.x
> but in my experience there are many changes in each kernel versions
>
>
> https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-will-revolutionize-programming-in-linux/
> is a good article explaining the benefits.
>
> Recently Netty project released 0.0.1-SNAPSHOT (
> https://netty.io/news/2020/11/16/io_uring-0-0-1-Final.html) and
> 0.0.2-SNAPSHOT with some nice numbers:
> - epoll
>
> Speed: 80820 request/sec, 80820 response/sec
> Requests: 2424626
> Responses: 2424625
>
> - io_uring
>
> Speed: 267371 request/sec, 267370 response/sec
> Requests: 8021137
> Responses: 8021128
>
> I've tried it with my HTTP2 comparison tests
> (
> https://martin-grigorov.medium.com/comparing-the-performance-of-several-http2-web-servers-fb5f3787532a
> )
>
> and the results were better but not so dramatic:
>
> - epoll
>
> -- x86_64: 24688 reqs/sec
>
> -- aarch64: 25344
>
> - uring
>
> -- x86_64: 26731
>
> -- aarch64: 28443
>
>
> So I thought: What about introducing a Uring Protocol/Connector ?!
> Since Netty uses JNI
>
> to talk to io_uring C APIs it should look something like the AprConnector.
>
> But soon after I realized that it would be much more reusable if
> implemented
>
> as  java.nio.channels.spi.SelectorProvider. This way any project (not
> just Tomcat) can
>
> make use of it.
>
>
> Here I wanted to poll what other devs think about this idea ?
>
> Does that sound interesting to you ? Or we should leave it to the JDK
> developers
>
> to do it for JDK XY (I guess this could be as early as Java 18 since 17
> will
>
> be a LTS and most probably the current epoll based impl won't be replaced
> so
>
> soon).
>
> I don't suggest to put it in Tomcat's code, so the dependency on JNI
>
> should not be considered as a stopper.
>
>
> I have some very early attempts implementing it with Netty's JNI code and
>
> with Java Panama's jextract, but I've faced issues with both.
>
>
> Please encourage/discourage me to continue digging in this! :-)
>

I think I had read something about this, and then kind of forgot about it.
IMO to really help out people it has to happen in OpenJDK. I'm quite sure
it will work out since improvements have started to happen again in NIO
(bugfixes start in 11, actual improvements starting in 14). NIO2 would
obviously benefit as well.

It seems from your numbers there would be some measurable performance
improvements. Not 100% sure about real world though.

Rémy


>
>
> Regards,
>
> Martin
>