You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Alexey Scherbakov (Jira)" <ji...@apache.org> on 2025/09/02 14:47:00 UTC
[jira] [Comment Edited] (IGNITE-22835) Latency penalty for using disruptor threads
[ https://issues.apache.org/jira/browse/IGNITE-22835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18017696#comment-18017696 ]
Alexey Scherbakov edited comment on IGNITE-22835 at 9/2/25 2:46 PM:
--------------------------------------------------------------------
I've conducted new tests on a branch ignite-22825 (rev 5e1a391410920995650e2a73b9e019cee15352d9)
The scenario involves bare metal server and client.
Defaults
{noformat}
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 1 avgt 20 201.786 ± 9.149 us/op
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 avgt 20 217.094 ± 18.183 us/op
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 thrpt 20 19254.515 ± 795.112 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 thrpt 20 18977.851 ± 937.815 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 16 thrpt 20 56347.211 ± 3210.683 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 16 thrpt 20 55183.843 ± 2335.803 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 32 thrpt 20 86504.146 ± 6469.353 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 64 thrpt 20 111756.265 ± 10197.576 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 128 thrpt 20 116758.187 ± 12526.985 ops/s{noformat}
Shared event loop
{noformat}
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 1 avgt 20 185.268 ± 10.259 us/op
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 thrpt 20 19664.990 ± 890.717 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 16 thrpt 20 58425.282 ± 2904.932 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 32 thrpt 20 85003.399 ± 6795.118 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 64 thrpt 20 99424.795 ± 8263.485 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 64 thrpt 20 98579.741 ± 8528.646 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 128 thrpt 20 112725.852 ± 12242.326 ops/s{noformat}
Logit
{noformat}
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 thrpt 20 20590.542 ± 1135.181 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 16 thrpt 20 58533.771 ± 4342.358 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 32 thrpt 20 94998.876 ± 7756.843 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 64 thrpt 20 117384.260 ± 10187.656 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 128 thrpt 20 125908.100 ± 12779.266 ops/s{noformat}
Shared event loop + Logit
{noformat}
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 thrpt 20 22632.517 ± 985.868 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 16 thrpt 20 64362.742 ± 4222.572 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 32 thrpt 20 99072.524 ± 7338.908 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 64 thrpt 20 127911.652 ± 13472.827 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 128 thrpt 20 127059.057 ± 16563.789 ops/s{noformat}
was (Author: ascherbakov):
I've conducted new tests on a branch ignite-22825 (rev 5e1a391410920995650e2a73b9e019cee15352d9)
The scenario involves bare metal server and client.
Defaults
{noformat}
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 1 avgt 20 201.786 ± 9.149 us/op
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 avgt 20 217.094 ± 18.183 us/op
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 thrpt 20 19254.515 ± 795.112 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 thrpt 20 18977.851 ± 937.815 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 16 thrpt 20 56347.211 ± 3210.683 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 16 thrpt 20 55183.843 ± 2335.803 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 32 thrpt 20 86504.146 ± 6469.353 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 64 thrpt 20 111756.265 ± 10197.576 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 128 thrpt 20 116758.187 ± 12526.985 ops/s{noformat}
Shared event loop
{noformat}
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 1 avgt 20 185.268 ± 10.259 us/op
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 thrpt 20 19664.990 ± 890.717 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 16 thrpt 20 58425.282 ± 2904.932 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 32 thrpt 20 85003.399 ± 6795.118 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 64 thrpt 20 99424.795 ± 8263.485 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 64 thrpt 20 98579.741 ± 8528.646 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 128 thrpt 20 112725.852 ± 12242.326 ops/s{noformat}
Logit
{noformat}
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 thrpt 20 20590.542 ± 1135.181 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 16 thrpt 20 58533.771 ± 4342.358 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 32 thrpt 20 94998.876 ± 7756.843 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 64 thrpt 20 117384.260 ± 10187.656 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 128 thrpt 20 125908.100 ± 12779.266 ops/s{noformat}
Shared event loop + Logit
{noformat}
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 4 thrpt 20 22632.517 ± 985.868 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 16 thrpt 20 64362.742 ± 4222.572 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 32 thrpt 20 99072.524 ± 7338.908 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 64 thrpt 20 127911.652 ± 13472.827 ops/s
Benchmark (fsync) (offset) (partitionCount) (remote) (threads) Mode Cnt Score Error Units
ClientKvPutBenchmark.upsert false 0 32 true 128 thrpt 20 127059.057 ± 16563.789 ops/s{noformat}
> Latency penalty for using disruptor threads
> -------------------------------------------
>
> Key: IGNITE-22835
> URL: https://issues.apache.org/jira/browse/IGNITE-22835
> Project: Ignite
> Issue Type: Improvement
> Reporter: Vladislav Pyatkov
> Assignee: Alexey Scherbakov
> Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Motivation
> All three disruptors are part of the RAFT implementation. Each time a command has to be replicated, all the disruptors are used. This leads to a bad impact on the entire operation's latency.
> {noformat}
> onBeforeApplyCmd:%node_3344%JRaft-Request-Processor-3 0.2 5415400 5415600
> Here is hidden 6.1 us
> RaftNodeOnEvent:%node_3344%JRaft-NodeImpl-Disruptor_stripe_2-0 0.0 5421700 5421700
> --
> RaftNodeOnEvent:%node_3344%JRaft-NodeImpl-Disruptor_stripe_2-0 0.0 5421700 5421700
> Here is hidden 2.8 us
> LogManagerOnEvent:%node_3344%JRaft-LogManager-Disruptor_stripe_2-0 0.1 5424500 5424600
> --
> flushLog:%node_3344%JRaft-LogManager-Disruptor_stripe_2-0 18.7 5426400 5445100
> Here is hidden 2.9 us
> FSMCallerOnEvent:%node_3344%JRaft-FSMCaller-Disruptor_stripe_1-0 0.1 5448000 5448100
> {noformat}
> h3. Definition of done
> * We can try to use the disruptor wait policy, depending on the case.
> * Maybe we can reduce a number of the dusruptor queues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)