You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by nigro_franz <ni...@gmail.com> on 2016/06/08 14:52:00 UTC
sun.misc.Unsafe VS io.netty.util.internal.PlatformDependent VS null
Hi to everyone!
I've developed a memory mapped implementation of SequenceFile to be used
inside Artemis's Journal and i've encountered a big question: can i use
sun.misc.Unsafe?
I've noticed that between all the dependencies of Artemis there was Netty's
libs with its PlatformDependent class and i've finally choose to use it
instead of "naked" Unsafe,but i'm not completly satisfied by this solution
because PlatformDependent does not provide all the methods of Unsafe (e.g.
Unsafe::pageSize).
With such methods there are a lot of improvements and features that can be
developed, considering that in the future of Java there will be a place for
a safer Unsafe :)
There exists a good policy for all that cases in which something so "unsafe"
and "platform dependend" is needed?What the other devs think about it?
With Regards,
Francesco
--
View this message in context: http://activemq.2283324.n4.nabble.com/sun-misc-Unsafe-VS-io-netty-util-internal-PlatformDependent-VS-null-tp4712785.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: sun.misc.Unsafe VS io.netty.util.internal.PlatformDependent VS
null
Posted by nigro_franz <ni...@gmail.com>.
Hi!
Thanks for the good advice, i'm happy to let Unsafe improve further the
performances of this mapped implementation
Here https://issues.apache.org/jira/browse/ARTEMIS-508
<https://issues.apache.org/jira/browse/ARTEMIS-508> there is the JIRA
related to the memory mapped implementation of SequenceFile: i need to
squash the (too many) commits into one before doing any PR and obviosly
reimplement a "confined" version of PlatformDependent that let Android too
to use the mmap implementation (crossing the fingers, ART VM is pure evil).
I've built for now only 2 benchmarks (latency focused), but without using
JHM because of the coordinated omission issue, thus i've found JLBH (Java
Latency Benchmark Harness)
http://www.rationaljava.com/2016/04/a-series-of-posts-on-jlbh-java-latency.html
<http://www.rationaljava.com/2016/04/a-series-of-posts-on-jlbh-java-latency.html>
that is best suited for this kind of measurements IMHO.
The results of the benchmarks are pretty good against NIO (if you push data
in "burst" of less then 10% of your memory, for a default configured latest
ubuntu linux box): about 2 order of magnitude less in the 99 %tile latency
measurements.
With these benchs i could measure how much on different configured HW the
mmap Sequence File could "keep the pace" with a sustained load of messages,
thus producing something pretty similar to a real use case eg without any
"math benefits" derived by the backpressure of the OS during the page
faulting unavoidable pauses.
I've planned to solve the Page Faulting issue using a pure async
implementation, using a single producer single consumer wait-free ringBuffer
that feeds an event loop responsible to write the data on the MappedFile but
the benchs that i've built (around Journal and SequenceFile) show that the
higher layers (above the mmap impl of SequenceFile) slow even this pure-sync
implementation, so i'm not sure that the Journal could receive a perceptible
benefit from an async one considering that it will only reduce the hiccups
when a page fault of few microseconds occour (anyway it could be tuned with
Huge Pages and increasing the dirty ratio).
I'm very curious to test it against libaio and i will be very happy if you
are interested to try it with me, in order to be objective: my expectations
are that with burst of small messages the mmap impl will be faster (crossing
JNI has higher cost than call an intrinsified tiny copy with Unsafe, in my
experience) , the opposite while the size of the messages grows. If you need
other benchs (throughput focused with JHM) or need more info about the
implementation i'm ever available
--
View this message in context: http://activemq.2283324.n4.nabble.com/sun-misc-Unsafe-VS-io-netty-util-internal-PlatformDependent-VS-null-tp4712785p4712866.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: sun.misc.Unsafe VS io.netty.util.internal.PlatformDependent VS null
Posted by Clebert Suconic <cl...@gmail.com>.
Like with Artemis, there's the Artemis-native, where I isolate the use
of the native module.
Maybe you could create a project to just control MapFiles.. it would
be easier to maintain it I think.
On Thu, Jun 9, 2016 at 2:20 PM, Clebert Suconic
<cl...@gmail.com> wrote:
> I would isolate it on a library. Netty does have it but it's well
> controlled on a single place I think (easier to maintain it)
>
> I had played with memory mapped years ago and I've decided to use
> libaio instead. Did you run any micro benchmarks, have you made any
> progress? it seems like a nice idea if you found a breakthrough on
> making it performant.. I'm quite interested on it.
>
> On Wed, Jun 8, 2016 at 10:52 AM, nigro_franz <ni...@gmail.com> wrote:
>> Hi to everyone!
>>
>> I've developed a memory mapped implementation of SequenceFile to be used
>> inside Artemis's Journal and i've encountered a big question: can i use
>> sun.misc.Unsafe?
>>
>> I've noticed that between all the dependencies of Artemis there was Netty's
>> libs with its PlatformDependent class and i've finally choose to use it
>> instead of "naked" Unsafe,but i'm not completly satisfied by this solution
>> because PlatformDependent does not provide all the methods of Unsafe (e.g.
>> Unsafe::pageSize).
>> With such methods there are a lot of improvements and features that can be
>> developed, considering that in the future of Java there will be a place for
>> a safer Unsafe :)
>> There exists a good policy for all that cases in which something so "unsafe"
>> and "platform dependend" is needed?What the other devs think about it?
>>
>>
>> With Regards,
>> Francesco
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://activemq.2283324.n4.nabble.com/sun-misc-Unsafe-VS-io-netty-util-internal-PlatformDependent-VS-null-tp4712785.html
>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>
>
>
> --
> Clebert Suconic
--
Clebert Suconic
Re: sun.misc.Unsafe VS io.netty.util.internal.PlatformDependent VS null
Posted by Clebert Suconic <cl...@gmail.com>.
I would isolate it on a library. Netty does have it but it's well
controlled on a single place I think (easier to maintain it)
I had played with memory mapped years ago and I've decided to use
libaio instead. Did you run any micro benchmarks, have you made any
progress? it seems like a nice idea if you found a breakthrough on
making it performant.. I'm quite interested on it.
On Wed, Jun 8, 2016 at 10:52 AM, nigro_franz <ni...@gmail.com> wrote:
> Hi to everyone!
>
> I've developed a memory mapped implementation of SequenceFile to be used
> inside Artemis's Journal and i've encountered a big question: can i use
> sun.misc.Unsafe?
>
> I've noticed that between all the dependencies of Artemis there was Netty's
> libs with its PlatformDependent class and i've finally choose to use it
> instead of "naked" Unsafe,but i'm not completly satisfied by this solution
> because PlatformDependent does not provide all the methods of Unsafe (e.g.
> Unsafe::pageSize).
> With such methods there are a lot of improvements and features that can be
> developed, considering that in the future of Java there will be a place for
> a safer Unsafe :)
> There exists a good policy for all that cases in which something so "unsafe"
> and "platform dependend" is needed?What the other devs think about it?
>
>
> With Regards,
> Francesco
>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/sun-misc-Unsafe-VS-io-netty-util-internal-PlatformDependent-VS-null-tp4712785.html
> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
--
Clebert Suconic