RE: Effectiveness of Scrub Operation vs SSTable previously marked in blacklist

[Michael Fong <mi...@ruckuswireless.com>]

Hi,

I have filed a jira ticket to keep tracked  @ https://issues.apache.org/jira/browse/CASSANDRA-11624

Thanks!

Sincerely,

Michael Fong

From: Marcus Eriksson [mailto:krummas@gmail.com]
Sent: Wednesday, March 23, 2016 10:47 PM
To: user@cassandra.apache.org
Subject: Re: Effectiveness of Scrub Operation vs SSTable previously marked in blacklist

yeah that is most likely a bug, could you file a ticket?

On Tue, Mar 22, 2016 at 4:36 AM, Michael Fong <mi...@ruckuswireless.com>> wrote:
Hi, all,

We recently encountered a scenario under Cassandra 2.0 deployment. Cassandra detected a corrupted sstable, and when we attempt to scrub the sstable (with all the associated sstables), the corrupted sstable was not included in the sstable list. This continues until we restart Cassandra and perform sstable again.

After we traced the Cassandra source code, we are a bit confused with the effectiveness of scrubbing and SStable being marked in blacklist in Cassandra 2.0+

It seems from previous version (Cassandra 1.2), the scrub operation would operate on a sstable regardless of it being previously marked. However, in Cassandra 2.0, the function flows seems changed.

Here is function flow that we traced in Cassandra 2.0 source code:


From org.apache.cassandra.db.compaction.CompactionManager

…
    public void performScrub(ColumnFamilyStore cfStore, final boolean skipCorrupted, final boolean checkData) throws InterruptedException, ExecutionException

    {

        performAllSSTableOperation(cfStore, new AllSSTablesOperation()

        {
…
    private void performAllSSTableOperation(final ColumnFamilyStore cfs, final AllSSTablesOperation operation) throws InterruptedException, ExecutionException
    {
        final Iterable<SSTableReader> sstables = cfs.markAllCompacting();
…

org.apache.cassandra.db. ColumnFamilyStore
…

    public Iterable<SSTableReader> markAllCompacting()

    {

        Callable<Iterable<SSTableReader>> callable = new Callable<Iterable<SSTableReader>>()

        {

            public Iterable<SSTableReader> call() throws Exception

            {

                assert data.getCompacting().isEmpty() : data.getCompacting();

                Iterable<SSTableReader> sstables = Lists.newArrayList(AbstractCompactionStrategy.filterSuspectSSTables(getSSTables()));

                if (Iterables.isEmpty(sstables))

                    return null;
…

If it is true, would this flow – marking corrupted sstable in blacklist, defeat the original purpose of scrub operation?  Thanks in advanced!


Sincerely,

Michael Fong