[jira] [Commented] (CASSANDRA-11383) SASI index build leads to massive OOM

["Pavel Yaskevich (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202027#comment-15202027 ] 

Pavel Yaskevich commented on CASSANDRA-11383:
---------------------------------------------

What is your memtable size? all of the index building is currently happening in fixed chunk sizes in memory, in case of flushing from memtable size of the every individual segment is going to be set to the size of memtable, so if you have big memtable and a bunch of indexes at this point it all depends on how fast can you flush and it looks like flush itself takes about 0.5 second.

> SASI index build leads to massive OOM
> -------------------------------------
>
>                 Key: CASSANDRA-11383
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11383
>             Project: Cassandra
>          Issue Type: Bug
>          Components: CQL
>         Environment: C* 3.4
>            Reporter: DOAN DuyHai
>         Attachments: system.log_sasi_build_oom
>
>
> 13 bare metal machines
> - 6 cores CPU (12 HT)
> - 64Gb RAM
> - 4 SSD in RAID0
>  JVM settings:
> - G1 GC
> - Xms32G, Xmx32G
> C* settings:
> - concurrent_compactors: 1
> - compaction_throughput_mb_per_sec: 256
> I created 9 SASI indices
>  - 8 indices with text field, NonTokenizingAnalyser,  PREFIX mode, case-insensitive
>  - 1 index with numeric field, SPARSE mode
>  After a while, the nodes just gone OOM.
>  I attach log files. You can see a lot of GC happening while index segments are flush to disk. At some point the node OOM ...
> /cc [~xedin]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)