[jira] [Commented] (ARTEMIS-508) Sequential File Improvement + Performance Tests

["clebert suconic (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARTEMIS-508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324586#comment-15324586 ] 

clebert suconic commented on ARTEMIS-508:
-----------------------------------------

It's worth checking this against this PR:

https://github.com/apache/activemq-artemis/pull/548


A lot of the journal is based on writing, batching writing, syncinc less often for multiple producers...

having this kind of thing might be worth checking against batched writes, and that PR might help it.


The syncs we need are only to guarantee the proper allocation of the files and to avoid close of the file when we are writing, we wouldn't need to sync during the actual write.

> Sequential File Improvement + Performance Tests
> -----------------------------------------------
>
>                 Key: ARTEMIS-508
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-508
>             Project: ActiveMQ Artemis
>          Issue Type: Improvement
>            Reporter: Francesco Nigro
>              Labels: feature, performance, test
>
> https://github.com/franz1981/activemq-artemis/tree/artemis-journal-performance/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/mapped
> I've implemented in the package org.apache.activemq.artemis.core.io.mapped a new memory mapped SequentialFile implementation to support fast write/read operations for OSs that cannot use libaio or need RAM access performances on standard Files.
> The implementation is not thread-safe (it is needed?) and needs more buffer checks (or a complex implementation).  I'm using the Netty's PlatformDependent class to perform bulk copy without safepoint's poll issues.
> The implementation is simple but it's good as a proof of concept to compare it against the others: i' ve added a coordinated-omission performance test to measure the latency of a directWrite + OS jitter.
> https://github.com/franz1981/activemq-artemis/blob/artemis-journal-performance/artemis-journal/src/test/java/org/apache/activemq/artemis/core/io/aio/SequentialFileBench.java
> The write tests show performances typical of memory mapped file: quoting from Peter Lawrey "for burst of up to 10% of the main memory, it can sustain rates of 1 - 3 GB/second written. e.g. A laptop with 8 GB of memory might handle bursts of 800 MB at a rate of 1 GB per second. A server with 64 GB of memory might handle a burst of 6.5 GB at a rate of 3 GB per second".
> I want to improve the quality of the implementation by:
> 1- enforcing the original SequenceFile contract 
> 2- replacing synchronized reads/writes with lock-free versions
> 3- replacing the EventExecutor with a Lock-Free (even wait-free) Array Queue + EventLoop's poller
> 4- addressing all the False-Sharing issues around all the AtomicLong instances used
> 5- reducing the garbage produced for the fast paths to 0
> [6- expose try methods to allow direct flow control]
> [7- replacing the Semaphore based rate limiter with a lock free one]
> That's are all proposals, what do you think about it? 
> Regards



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)