You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Kirill Gusakov (Jira)" <ji...@apache.org> on 2022/12/28 00:37:00 UTC
[jira] [Created] (IGNITE-18475) Huge performance drop with enabled sync write per log entry for RAFT logs
Kirill Gusakov created IGNITE-18475:
---------------------------------------
Summary: Huge performance drop with enabled sync write per log entry for RAFT logs
Key: IGNITE-18475
URL: https://issues.apache.org/jira/browse/IGNITE-18475
Project: Ignite
Issue Type: Task
Reporter: Kirill Gusakov
During the YCSB benchmark runs for ignite-3 beta1 we found out, that we have significant issues with performance for select/insert queries.
One of the root cause of these issues: write every log entry to rocksdb with enabled sync option (which leads to frequent fsync calls).
These issues can be reproduced by localised jmh benchmarks [SelectBenchmark|https://github.com/gridgain/apache-ignite-3/blob/4b9de922caa4aef97a5e8e159d5db76a3fc7a3ad/modules/runner/src/test/java/org/apache/ignite/internal/benchmark/SelectBenchmark.java#L39] and [InsertBenchmark|https://github.com/gridgain/apache-ignite-3/blob/4b9de922caa4aef97a5e8e159d5db76a3fc7a3ad/modules/runner/src/test/java/org/apache/ignite/internal/benchmark/InsertBenchmark.java#L29] with RaftOptions.sync=true/false:
* jdbc select queries: 115ms vs 4ms
* jdbc insert queries: 70ms vs 2.5ms
(These results received on MacBook Pro (16-inch, 2019) and it looks like macOS has slow fsync command in general, but runs on Ubuntu shows the huge different also (~26 times for insert test). So, your environment can show another, but still huge difference.)
Why select quries suffers from syncs even more, than inserts, described in https://issues.apache.org/jira/browse/IGNITE-18474.
Possible solutions for the issue:
* Doesn't sync every raft record in rocksdb by default, but it can break the raft invariants
* Investigate the inner parts of RocksDB (according syscall tracing, not every write with sync produce fsync syscall), maybe another strategies wll be suitable for our cases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)