You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Jeremy Jongsma <je...@barchart.com> on 2014/06/10 16:31:05 UTC
Frequent secondary index sstable corruption
I'm in the process of migrating data over to cassandra for several of our
apps, and a few of the schemas use secondary indexes. Four times in the
last couple months I've run into a corrupted sstable belonging to a
secondary index, but have never seen this on any other sstables. When it
happens, any query against the secondary index just hangs until the node is
fixed. It's making me a bit nervous about using secondary indexes in
production.
This has usually happened after a bulk data import, so I am wondering if
the "firehose method" of dumping initial data into cassandra (write
consistency = any) is causing some sort of write concurrency issue when it
comes to secondary indexes. Has anyone else experienced this?
The cluster is running 1.2.16 on 4x EC2 m1.large instances.
Re: Frequent secondary index sstable corruption
Posted by Tyler Hobbs <ty...@datastax.com>.
If you've been dropping and recreating tables with the same name, you might
be seeing this: https://issues.apache.org/jira/browse/CASSANDRA-6525
On Tue, Jun 10, 2014 at 12:19 PM, Robert Coli <rc...@eventbrite.com> wrote:
> On Tue, Jun 10, 2014 at 7:31 AM, Jeremy Jongsma <je...@barchart.com>
> wrote:
>
>> I'm in the process of migrating data over to cassandra for several of our
>> apps, and a few of the schemas use secondary indexes. Four times in the
>> last couple months I've run into a corrupted sstable belonging to a
>> secondary index, but have never seen this on any other sstables. When it
>> happens, any query against the secondary index just hangs until the node is
>> fixed. It's making me a bit nervous about using secondary indexes in
>> production.
>>
>
>
> http://mail-archives.apache.org/mod_mbox/incubator-cassandra-user/201405.mbox/%3CCAEDUwd1i2BwJ-PAFE1qhjQFZ=qz2vA_vXWo_jDYCmS8EvkBSLQ@mail.gmail.com%3E
>
> I don't know if this particular issue is known and/or fixed upstream, but
> FWIW/FYI!
>
> =Rob
>
--
Tyler Hobbs
DataStax <http://datastax.com/>
Re: Frequent secondary index sstable corruption
Posted by Robert Coli <rc...@eventbrite.com>.
On Tue, Jun 10, 2014 at 7:31 AM, Jeremy Jongsma <je...@barchart.com> wrote:
> I'm in the process of migrating data over to cassandra for several of our
> apps, and a few of the schemas use secondary indexes. Four times in the
> last couple months I've run into a corrupted sstable belonging to a
> secondary index, but have never seen this on any other sstables. When it
> happens, any query against the secondary index just hangs until the node is
> fixed. It's making me a bit nervous about using secondary indexes in
> production.
>
http://mail-archives.apache.org/mod_mbox/incubator-cassandra-user/201405.mbox/%3CCAEDUwd1i2BwJ-PAFE1qhjQFZ=qz2vA_vXWo_jDYCmS8EvkBSLQ@mail.gmail.com%3E
I don't know if this particular issue is known and/or fixed upstream, but
FWIW/FYI!
=Rob