You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Martin Grigorov <mg...@apache.org> on 2020/09/28 14:33:25 UTC
ByteBuffer pooling
Hi,
I've profiled the memory allocation during load testing HTTP2:
https://pasteboard.co/Jtblqfl.png
As you can see there are a lot of ByteBuffer.allocate(int) calls.
org.apache.catalina.connector.Response#Response(int)
org.apache.catalina.connector.Request#inputBuffer
org.apache.coyote.http2.Http2AsyncParser#readFrame
org.apache.coyote.http2.Stream.StreamOutputBuffer#buffer
Has it been discussed in the past to use a pool of ByteBuffer instances ?
Netty provides such for its abstraction ByteBuf:
https://netty.io/wiki/using-as-a-generic-library.html
Here is a simple implementation:
https://github.com/jhg023/Pbbl/blob/39c749b9e65f4f8a840a07812559cf8830bd5eae/src/main/java/com/github/pbbl/AbstractBufferPool.java#L44
It's give() method could be optimized to not return the buffer to the pool
if the pool size is bigger than N, so it doesn't use huge list of buffers
which are used just once.
The big unknown to me is: where to return the buffers to the pool ?
For HTTP2 maybe this could be done after writing the buffer to the socket.
If the Request/Response is recycled then their Input/OutputBuffer are
reused and everything is OK. But if recycling is not in use then new
allocations are done for each new request.
I see that org.apache.tomcat.util.net.WriteBuffer does some pooling already.
What do you think?
Martin
Re: ByteBuffer pooling
Posted by Rémy Maucherat <re...@apache.org>.
On Mon, Sep 28, 2020 at 5:31 PM Martin Grigorov <mg...@apache.org>
wrote:
> On Mon, Sep 28, 2020 at 6:11 PM Rémy Maucherat <re...@apache.org> wrote:
>
> > On Mon, Sep 28, 2020 at 4:34 PM Martin Grigorov <mg...@apache.org>
> > wrote:
> >
> > > Hi,
> > >
> > > I've profiled the memory allocation during load testing HTTP2:
> > > https://pasteboard.co/Jtblqfl.png
> > >
> > > As you can see there are a lot of ByteBuffer.allocate(int) calls.
> > > org.apache.catalina.connector.Response#Response(int)
> > > org.apache.catalina.connector.Request#inputBuffer
> > > org.apache.coyote.http2.Http2AsyncParser#readFrame
> > > org.apache.coyote.http2.Stream.StreamOutputBuffer#buffer
> > >
> > > Has it been discussed in the past to use a pool of ByteBuffer
> instances ?
> > > Netty provides such for its abstraction ByteBuf:
> > > https://netty.io/wiki/using-as-a-generic-library.html
> > >
> > > Here is a simple implementation:
> > >
> > >
> >
> https://github.com/jhg023/Pbbl/blob/39c749b9e65f4f8a840a07812559cf8830bd5eae/src/main/java/com/github/pbbl/AbstractBufferPool.java#L44
> > > It's give() method could be optimized to not return the buffer to the
> > pool
> > > if the pool size is bigger than N, so it doesn't use huge list of
> buffers
> > > which are used just once.
> > >
> > > The big unknown to me is: where to return the buffers to the pool ?
> > > For HTTP2 maybe this could be done after writing the buffer to the
> > socket.
> > > If the Request/Response is recycled then their Input/OutputBuffer are
> > > reused and everything is OK. But if recycling is not in use then new
> > > allocations are done for each new request.
> > >
> > > I see that org.apache.tomcat.util.net.WriteBuffer does some pooling
> > > already.
> > >
> > > What do you think?
> > >
> >
> > I removed a lot of pooling in the connector with zero impact, so I don't
> > see the point.
> >
>
> ConcurrentDateFormat pools SimpleDateFormat instances and it does make a
> difference:
>
> https://medium.com/@martin.grigorov/compare-performance-of-javas-simpledateformat-against-datetimeformatter-31be58cadf1d
> !
> I'd give it a try for the HTTP2 Stream related allocations and let you know
> how it goes!
>
This is not a simple object, this involves parsing and computations. A BB
however is a very simple object, even if this somehow shows up in a
microbenchmark the actual impact will be negative (more memory is retained,
the life of the GC is harder, and most importantly contention on the
cache). I mean these were the optimizations I was doing in the 2000s, but
now it seems to work as the JDK people say.
There's an exception for direct BBs in theory, but in practice retaining
the memory is an even bigger issue.
Rémy
Re: ByteBuffer pooling
Posted by Martin Grigorov <mg...@apache.org>.
On Mon, Sep 28, 2020 at 6:11 PM Rémy Maucherat <re...@apache.org> wrote:
> On Mon, Sep 28, 2020 at 4:34 PM Martin Grigorov <mg...@apache.org>
> wrote:
>
> > Hi,
> >
> > I've profiled the memory allocation during load testing HTTP2:
> > https://pasteboard.co/Jtblqfl.png
> >
> > As you can see there are a lot of ByteBuffer.allocate(int) calls.
> > org.apache.catalina.connector.Response#Response(int)
> > org.apache.catalina.connector.Request#inputBuffer
> > org.apache.coyote.http2.Http2AsyncParser#readFrame
> > org.apache.coyote.http2.Stream.StreamOutputBuffer#buffer
> >
> > Has it been discussed in the past to use a pool of ByteBuffer instances ?
> > Netty provides such for its abstraction ByteBuf:
> > https://netty.io/wiki/using-as-a-generic-library.html
> >
> > Here is a simple implementation:
> >
> >
> https://github.com/jhg023/Pbbl/blob/39c749b9e65f4f8a840a07812559cf8830bd5eae/src/main/java/com/github/pbbl/AbstractBufferPool.java#L44
> > It's give() method could be optimized to not return the buffer to the
> pool
> > if the pool size is bigger than N, so it doesn't use huge list of buffers
> > which are used just once.
> >
> > The big unknown to me is: where to return the buffers to the pool ?
> > For HTTP2 maybe this could be done after writing the buffer to the
> socket.
> > If the Request/Response is recycled then their Input/OutputBuffer are
> > reused and everything is OK. But if recycling is not in use then new
> > allocations are done for each new request.
> >
> > I see that org.apache.tomcat.util.net.WriteBuffer does some pooling
> > already.
> >
> > What do you think?
> >
>
> I removed a lot of pooling in the connector with zero impact, so I don't
> see the point.
>
ConcurrentDateFormat pools SimpleDateFormat instances and it does make a
difference:
https://medium.com/@martin.grigorov/compare-performance-of-javas-simpledateformat-against-datetimeformatter-31be58cadf1d
!
I'd give it a try for the HTTP2 Stream related allocations and let you know
how it goes!
>
> Rémy
>
>
> >
> > Martin
> >
>
Re: ByteBuffer pooling
Posted by Rémy Maucherat <re...@apache.org>.
On Mon, Sep 28, 2020 at 4:34 PM Martin Grigorov <mg...@apache.org>
wrote:
> Hi,
>
> I've profiled the memory allocation during load testing HTTP2:
> https://pasteboard.co/Jtblqfl.png
>
> As you can see there are a lot of ByteBuffer.allocate(int) calls.
> org.apache.catalina.connector.Response#Response(int)
> org.apache.catalina.connector.Request#inputBuffer
> org.apache.coyote.http2.Http2AsyncParser#readFrame
> org.apache.coyote.http2.Stream.StreamOutputBuffer#buffer
>
> Has it been discussed in the past to use a pool of ByteBuffer instances ?
> Netty provides such for its abstraction ByteBuf:
> https://netty.io/wiki/using-as-a-generic-library.html
>
> Here is a simple implementation:
>
> https://github.com/jhg023/Pbbl/blob/39c749b9e65f4f8a840a07812559cf8830bd5eae/src/main/java/com/github/pbbl/AbstractBufferPool.java#L44
> It's give() method could be optimized to not return the buffer to the pool
> if the pool size is bigger than N, so it doesn't use huge list of buffers
> which are used just once.
>
> The big unknown to me is: where to return the buffers to the pool ?
> For HTTP2 maybe this could be done after writing the buffer to the socket.
> If the Request/Response is recycled then their Input/OutputBuffer are
> reused and everything is OK. But if recycling is not in use then new
> allocations are done for each new request.
>
> I see that org.apache.tomcat.util.net.WriteBuffer does some pooling
> already.
>
> What do you think?
>
I removed a lot of pooling in the connector with zero impact, so I don't
see the point.
Rémy
>
> Martin
>