[jira] [Comment Edited] (CASSANDRA-17777) Skip sstable startup checks for dropped system tables

["Stefan Miklosovic (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-17777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571611#comment-17571611 ] 

Stefan Miklosovic edited comment on CASSANDRA-17777 at 7/26/22 8:46 PM:
------------------------------------------------------------------------

FYI there is also ColumnFamilyStore.scrubDataDirectories which is called during startup (not sure if it is before or after startup checks). If you are getting this issue, it seems like they are not detected by this method (or they would be if startup checks have not been failing?) Do you think it is somehow reasonable to remove such files as part of the scrub? My humble opinion is that we should not because if upgrade goes wrong and there is a need to downgrade, since scrub deleted these system tables it is too late.


was (Author: smiklosovic):
FYI there is also ColumnFamilyStore.scrubDataDirectories which is called where during startup (not sure if it is before or after startup checks). If you are getting this issue, it seems like they are not detected by this method (or they would be if startup checks have not been failing?) Do you think it is somehow reasonable to remove such files as part of the scrub? My humble opinion is that we should not because if upgrade goes wrong and there is a need to downgrade, since scrub deleted these system tables it is too late.

> Skip sstable startup checks for dropped system tables
> -----------------------------------------------------
>
>                 Key: CASSANDRA-17777
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-17777
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Local/Startup and Shutdown
>            Reporter: Josh McKenzie
>            Assignee: Josh McKenzie
>            Priority: Normal
>
> In CASSANDRA-8049 we changed our behavior to explicitly not startup while iterating over the whole data file directory before loading schema. In the case where you have (admittedly _very)_ old files left around from dropped system tables from old C* versions (think 1.0 era), you can end up with clusters that won't start due to what was, at the time, normal operations.
> We should change from hard killing nodes to instead warn operators they have leftovers of old system tables that need to be cleaned up. Ideally these files would be manually removed by operators before the upgrade to new versions however we can't always rely on this so killing nodes on startup for this specific case is suboptimal. We certainly still need to log the found files to give operators an indication of what needs to be cleaned up.
> While we can't load the schema to make sure all directories are safe, we _can_ know which tables exist in the system keyspace so we should be able to safely modify this startup check to log rather than kill for system tables only.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org