You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jordan West (JIRA)" <ji...@apache.org> on 2018/05/14 19:43:00 UTC
[jira] [Comment Edited] (CASSANDRA-14055) Index redistribution
breaks SASI index
[ https://issues.apache.org/jira/browse/CASSANDRA-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369601#comment-16369601 ]
Jordan West edited comment on CASSANDRA-14055 at 5/14/18 7:42 PM:
------------------------------------------------------------------
[~lboutros]/[~jasobrown], some updates:
I have attached two new patches. One for trunk and one of 3.11. Unfortunately, the test changes in trunk don't work well on 3.11 so we can't have one patch. The primary changes in this patch are to change the order we iterate over the indexes to ensure we retain the newer instance of {{SSTableIndex}} and thus the newer {{SSTableReader}}. I also changed the code to clone the {{oldSSTables}} collection since its visible outside the {{View}} constructor.
||3.11||Trunk||
|[branch|https://github.com/jrwest/cassandra/tree/14055-jrwest-3.11]|[branch|https://github.com/jrwest/cassandra/tree/14055-jrwest-trunk]|
|[utests|https://circleci.com/gh/jrwest/cassandra/tree/14055-jrwest-3.11]|[utests|https://circleci.com/gh/jrwest/cassandra/tree/14055-jrwest-trunk]|
NOTE: same utests are failing on [trunk|https://circleci.com/gh/jrwest/cassandra/25] and I'm still working on getting dtests running with my CircleCI setup.
Also, I spoke with some colleagues including [~beobal] and [~krummas] about the use of {{sstableRef.globalCount()}} to determine when to delete the SASI index file. I've come to the conclusion that its use at all is wrong because it represents the number of references to the instance, not globally. Given index summary redistribution, this isn't a safe assumption. Looking back at the original SASI patches, I am not sure why it got merged this way. The [patches|https://github.com/xedin/sasi/blob/master/src/java/org/apache/cassandra/db/index/sasi/SSTableIndex.java#L120] used {{sstable.isMarkedCompacted()}} but the [merged code|https://github.com/apache/cassandra/commit/72790dc8e34826b39ac696b03025ae6b7b6beb2b#diff-4873bb6fcef158ff18d221571ef2ec7cR124] used {{sstableRef.globalCount()}}. Fixing this is a larger undertaking, so I propose we split that work into a separate ticket and focus this one on SASI's failure to account for index redistribution in the {{View}}. The work covered by the other ticket would entail either a) deleting the SASI index files as part of {{SSTableTidier}} or by moving {{SSTableIndex}} to use {{Ref}} and implementing a tidier specific to it.
was (Author: jrwest):
[~lboutros]/[~jasobrown], some updates:
I have attached two new patches. One for trunk and one of 3.11. Unfortunately, the test changes in trunk don't work well on 3.11 so we can't have one patch. The primary changes in this patch are to change the order we iterate over the indexes to ensure we retain the newer instance of {{SSTableIndex}} and thus the newer {{SSTableReader}}. I also changed the code to clone the {{oldSSTables}} collection since its visible outside the {{View}} constructor.
||3.11||Trunk||
|[branch|https://github.com/jrwest/cassandra/tree/14055-jrwest-3.11]|[branch|https://github.com/jrwest/cassandra/tree/14055-jrwest-trunk]|
|[utests|https://circleci.com/gh/jrwest/cassandra/24]|[utests|https://circleci.com/gh/jrwest/cassandra/26]|
NOTE: same utests are failing on [trunk|https://circleci.com/gh/jrwest/cassandra/25] and I'm still working on getting dtests running with my CircleCI setup.
Also, I spoke with some colleagues including [~beobal] and [~krummas] about the use of {{sstableRef.globalCount()}} to determine when to delete the SASI index file. I've come to the conclusion that its use at all is wrong because it represents the number of references to the instance, not globally. Given index summary redistribution, this isn't a safe assumption. Looking back at the original SASI patches, I am not sure why it got merged this way. The [patches|https://github.com/xedin/sasi/blob/master/src/java/org/apache/cassandra/db/index/sasi/SSTableIndex.java#L120] used {{sstable.isMarkedCompacted()}} but the [merged code|https://github.com/apache/cassandra/commit/72790dc8e34826b39ac696b03025ae6b7b6beb2b#diff-4873bb6fcef158ff18d221571ef2ec7cR124] used {{sstableRef.globalCount()}}. Fixing this is a larger undertaking, so I propose we split that work into a separate ticket and focus this one on SASI's failure to account for index redistribution in the {{View}}. The work covered by the other ticket would entail either a) deleting the SASI index files as part of {{SSTableTidier}} or by moving {{SSTableIndex}} to use {{Ref}} and implementing a tidier specific to it.
> Index redistribution breaks SASI index
> --------------------------------------
>
> Key: CASSANDRA-14055
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14055
> Project: Cassandra
> Issue Type: Bug
> Components: sasi
> Reporter: Ludovic Boutros
> Assignee: Jordan West
> Priority: Major
> Labels: patch, sasi
> Fix For: 3.11.x, 4.x
>
> Attachments: 14055-jrwest-3.11.patch, 14055-jrwest-trunk.patch, CASSANDRA-14055-jrwest.patch, CASSANDRA-14055.patch, CASSANDRA-14055.patch, CASSANDRA-14055.patch
>
>
> During index redistribution process, a new view is created.
> During this creation, old indexes should be released.
> But, new indexes are "attached" to the same SSTable as the old indexes.
> This leads to the deletion of the last SASI index file and breaks the index.
> The issue is in this function : [https://github.com/apache/cassandra/blob/9ee44db49b13d4b4c91c9d6332ce06a6e2abf944/src/java/org/apache/cassandra/index/sasi/conf/view/View.java#L62]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org