You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Josh McKenzie (Jira)" <ji...@apache.org> on 2022/07/27 16:56:00 UTC
[jira] [Created] (CASSANDRA-17781) Secondary index building should not block compaction progress
Josh McKenzie created CASSANDRA-17781:
-----------------------------------------
Summary: Secondary index building should not block compaction progress
Key: CASSANDRA-17781
URL: https://issues.apache.org/jira/browse/CASSANDRA-17781
Project: Cassandra
Issue Type: Improvement
Components: Feature/2i Index
Reporter: Josh McKenzie
Assignee: Josh McKenzie
While CASSANDRA-11218 should also address this same problem, that ticket's a pretty big beautiful hammer and there's a fairly surgical modification we can make to the 2i building process to mitigate 2i building's ability to block compaction.
While 11218 approaches this from a "lots of compactions can starve 2i build and things never finish", what I'd like to address here is the inverse that building multiple secondary indices in parallel can occupy all available compactors to a point where we don't have any left to do actual compactions. Completing 2i build can take several hours and the backlog of compactions from that time may result in too many uncompacted sstables left to catch up.
A fairly simple fix is to break 2i building out to its own threadpool w/a hot proppable param you can use to change the # of threads for that build in the extreme cases where the default of 2 is indufficient.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org