You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Kirill Tkalenko (Jira)" <ji...@apache.org> on 2023/03/13 08:31:00 UTC
[jira] [Updated] (IGNITE-18475) Huge performance drop with enabled sync write per log entry for RAFT logs
[ https://issues.apache.org/jira/browse/IGNITE-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kirill Tkalenko updated IGNITE-18475:
-------------------------------------
Ignite Flags: (was: Docs Required,Release Notes Required)
> Huge performance drop with enabled sync write per log entry for RAFT logs
> -------------------------------------------------------------------------
>
> Key: IGNITE-18475
> URL: https://issues.apache.org/jira/browse/IGNITE-18475
> Project: Ignite
> Issue Type: Task
> Reporter: Kirill Gusakov
> Assignee: Ivan Bessonov
> Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During the YCSB benchmark runs for ignite-3 beta1 we found out, that we have significant issues with performance for select/insert queries.
> One of the root cause of these issues: write every log entry to rocksdb with enabled sync option (which leads to frequent fsync calls).
> These issues can be reproduced by localised jmh benchmarks [SelectBenchmark|https://github.com/gridgain/apache-ignite-3/blob/4b9de922caa4aef97a5e8e159d5db76a3fc7a3ad/modules/runner/src/test/java/org/apache/ignite/internal/benchmark/SelectBenchmark.java#L39] and [InsertBenchmark|https://github.com/gridgain/apache-ignite-3/blob/4b9de922caa4aef97a5e8e159d5db76a3fc7a3ad/modules/runner/src/test/java/org/apache/ignite/internal/benchmark/InsertBenchmark.java#L29] with RaftOptions.sync=true/false:
> * jdbc select queries: 115ms vs 4ms
> * jdbc insert queries: 70ms vs 2.5ms
> (These results received on MacBook Pro (16-inch, 2019) and it looks like macOS has slow fsync command in general, but runs on Ubuntu shows the huge different also (~26 times for insert test). So, your environment can show another, but still huge difference.)
> Why select queries suffers from syncs even more, than inserts, described in https://issues.apache.org/jira/browse/IGNITE-18474.
> Possible solutions for the issue:
> * Doesn't sync every raft record in rocksdb by default, but it can break the raft invariants
> * Investigate the inner parts of RocksDB (according syscall tracing, not every write with sync produce fsync syscall), maybe another strategies wll be suitable for our cases
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)