PageCursorProviderImp::finishCleanup inneficiency

[nigro_franz <ni...@gmail.com>]

Hi folks,

I'm running some benchmarks on paging and I've noticed that
PageCursorProviderImpl::finishCleanup is performing Page::read on a cache
miss vs softCache just to call Page::delete, that will do something relevant
only on the large messages in the page.
I believe this could be improved by:
- marking the number of large messages on Page
- doing some quick check on Page::read to save creating PagedMessageImpl for
non large messages

Any idea to improve it?



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html

Re: PageCursorProviderImp::finishCleanup inneficiency

[Clebert Suconic <cl...@gmail.com>]

+1 on st least a flag to contain large messages.

This is because we delete large messages on ack however to be 100% sure we
wouldn’t miss anything (say in case of a crash) a double check is made.

Perhaps we could add a record to the beginning of the page with -1
(undefined) or N recorda as large and only check for large when there are
large messages.


This would probably break compatibility so we may need an extra check for
record type at the head.

We will need to update the record before the page is closed as we write to
tail and would update head when moving to a next page.

This will probably be an issue with slow / cloud storage as opposed to a
fast local disk.

On Tue, Dec 17, 2019 at 5:43 AM yw yw <wy...@gmail.com> wrote:

> This is a good improvement, I thought about it too. With 1, I didn't come
> up with good idea to implement it. With 2, it seems more like a simple and
> direct optimization on reading page :)
>
> nigro_franz <ni...@gmail.com> 于2019年12月17日周二 上午3:13写道：
>
> > Hi folks,
> >
> > I'm running some benchmarks on paging and I've noticed that
> > PageCursorProviderImpl::finishCleanup is performing Page::read on a cache
> > miss vs softCache just to call Page::delete, that will do something
> > relevant
> > only on the large messages in the page.
> > I believe this could be improved by:
> > - marking the number of large messages on Page
> > - doing some quick check on Page::read to save creating PagedMessageImpl
> > for
> > non large messages
> >
> > Any idea to improve it?
> >
> >
> >
> > --
> > Sent from:
> > http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
> >
>
-- 
Clebert Suconic

Re: PageCursorProviderImp::finishCleanup inneficiency

[yw yw <wy...@gmail.com>]

This is a good improvement, I thought about it too. With 1, I didn't come
up with good idea to implement it. With 2, it seems more like a simple and
direct optimization on reading page :)

nigro_franz <ni...@gmail.com> 于2019年12月17日周二 上午3:13写道：

> Hi folks,
>
> I'm running some benchmarks on paging and I've noticed that
> PageCursorProviderImpl::finishCleanup is performing Page::read on a cache
> miss vs softCache just to call Page::delete, that will do something
> relevant
> only on the large messages in the page.
> I believe this could be improved by:
> - marking the number of large messages on Page
> - doing some quick check on Page::read to save creating PagedMessageImpl
> for
> non large messages
>
> Any idea to improve it?
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
>