You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by mahesh rajamani <ra...@gmail.com> on 2014/02/14 15:45:48 UTC
Expired column showing up
Hi,
I am using Cassandra 2.0.2 version. On a wide row (approx. 10000 columns),
I expire few column by setting TTL as 1 second. At times these columns show
up during slice query.
When I have this issue, running count and get commands for that row using
Cassandra cli it gives different column counts.
But once I run flush and compact, the issue goes off and expired columns
don't show up.
Can someone provide some help on this issue.
--
Regards,
Mahesh Rajamani
Re: Expired column showing up
Posted by Jacob Rhoden <ja...@me.com>.
It is my understanding that rows with TTLs don't mix well with rows that don't have TTLs. ie they should all have TTL or all not have TTL.
That said if you can create a small java class (test case) that demonstrates the problem, I'm happy to try it out on 2.0.5. This code can be attached to a jira ticket if needed.
______________________________
Sent from iPhone
> On 15 Feb 2014, at 1:45 am, mahesh rajamani <ra...@gmail.com> wrote:
>
> Hi,
>
> I am using Cassandra 2.0.2 version. On a wide row (approx. 10000 columns),
> I expire few column by setting TTL as 1 second. At times these columns show
> up during slice query.
>
> When I have this issue, running count and get commands for that row using
> Cassandra cli it gives different column counts.
>
> But once I run flush and compact, the issue goes off and expired columns
> don't show up.
>
> Can someone provide some help on this issue.
>
> --
> Regards,
> Mahesh Rajamani
Re: Expired column showing up
Posted by Jacob Rhoden <ja...@me.com>.
It is my understanding that rows with TTLs don't mix well with rows that don't have TTLs. ie they should all have TTL or all not have TTL.
That said if you can create a small java class (test case) that demonstrates the problem, I'm happy to try it out on 2.0.5. This code can be attached to a jira ticket if needed.
______________________________
Sent from iPhone
> On 15 Feb 2014, at 1:45 am, mahesh rajamani <ra...@gmail.com> wrote:
>
> Hi,
>
> I am using Cassandra 2.0.2 version. On a wide row (approx. 10000 columns),
> I expire few column by setting TTL as 1 second. At times these columns show
> up during slice query.
>
> When I have this issue, running count and get commands for that row using
> Cassandra cli it gives different column counts.
>
> But once I run flush and compact, the issue goes off and expired columns
> don't show up.
>
> Can someone provide some help on this issue.
>
> --
> Regards,
> Mahesh Rajamani
Re: Expired column showing up
Posted by mahesh rajamani <ra...@gmail.com>.
I upgraded the Cassandra to 2.0.5, these issues did not occur so far.
Thanks
Mahesh
On Mon, Feb 17, 2014 at 1:43 PM, mahesh rajamani
<ra...@gmail.com>wrote:
> Christian,
>
> There are 2 use cases which are failing, and both looks to be similar
> issue, basically happens in column family set with TTL.
>
> case 1) I manage index for specific data as single row in a column family.
> I set TTL to 1 second if the data need to be removed from the index row.
> Under some scenario the get and count for the row key gives different
> column counts. In the application if I do get I get correct set of
> columns(expired columns don't return), but if I do slice query and read 100
> columns at a time, the columns set with TTL returns. I am not able to
> understand, what is starting this issue.
>
> case 2) I have column family for managing locks, In this case I insert
> a column with by default TTL as 15 seconds. If the transaction completes
> before I remove the column by again setting TTL to 1 second.
>
> In this case when running flush the flush hangs with following Assertion
> exception.
>
> ERROR [FlushWriter:1] 2014-02-17 11:49:29,349 CassandraDaemon.java (line
> 187) Exception in thread Thread[FlushWriter:1,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:198)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:186)
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:360)
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:315)
> at
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
>
>
> Thanks
> Mahesh
>
>
>
> On Mon, Feb 17, 2014 at 12:43 PM, horschi <ho...@gmail.com> wrote:
>
>> Hi Mahesh,
>>
>> the problem is that every column is only tombstoned for as long as the
>> original column was valid.
>>
>> So if the last update was only valid for 1 sec, then the tombstone will
>> also be valid for 1 second! If the previous was valid for a longer time,
>> then this old value might reappear.
>>
>> Maybe you can explain why you are doing this?
>>
>> kind regards,
>> Christian
>>
>>
>>
>> On Mon, Feb 17, 2014 at 6:18 PM, mahesh rajamani <
>> rajamani.mahesh@gmail.com> wrote:
>>
>>> Christain,
>>>
>>> Yes. Is it a problem? Can you explain what happens in this scenario?
>>>
>>> Thanks
>>> Mahesh
>>>
>>>
>>> On Fri, Feb 14, 2014 at 3:07 PM, horschi <ho...@gmail.com> wrote:
>>>
>>>> Hi Mahesh,
>>>>
>>>> is it possible you are creating columns with a long TTL, then update
>>>> these columns with a smaller TTL?
>>>>
>>>> kind regards,
>>>> Christian
>>>>
>>>>
>>>> On Fri, Feb 14, 2014 at 3:45 PM, mahesh rajamani <
>>>> rajamani.mahesh@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am using Cassandra 2.0.2 version. On a wide row (approx. 10000
>>>>> columns), I expire few column by setting TTL as 1 second. At times these
>>>>> columns show up during slice query.
>>>>>
>>>>> When I have this issue, running count and get commands for that row
>>>>> using Cassandra cli it gives different column counts.
>>>>>
>>>>> But once I run flush and compact, the issue goes off and expired
>>>>> columns don't show up.
>>>>>
>>>>> Can someone provide some help on this issue.
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Mahesh Rajamani
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Mahesh Rajamani
>>>
>>
>>
>
>
> --
> Regards,
> Mahesh Rajamani
>
--
Regards,
Mahesh Rajamani
Re: Expired column showing up
Posted by mahesh rajamani <ra...@gmail.com>.
Christian,
There are 2 use cases which are failing, and both looks to be similar
issue, basically happens in column family set with TTL.
case 1) I manage index for specific data as single row in a column family.
I set TTL to 1 second if the data need to be removed from the index row.
Under some scenario the get and count for the row key gives different
column counts. In the application if I do get I get correct set of
columns(expired columns don't return), but if I do slice query and read 100
columns at a time, the columns set with TTL returns. I am not able to
understand, what is starting this issue.
case 2) I have column family for managing locks, In this case I insert
a column with by default TTL as 15 seconds. If the transaction completes
before I remove the column by again setting TTL to 1 second.
In this case when running flush the flush hangs with following Assertion
exception.
ERROR [FlushWriter:1] 2014-02-17 11:49:29,349 CassandraDaemon.java (line
187) Exception in thread Thread[FlushWriter:1,5,main]
java.lang.AssertionError
at
org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:198)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:186)
at
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:360)
at
org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:315)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Thanks
Mahesh
On Mon, Feb 17, 2014 at 12:43 PM, horschi <ho...@gmail.com> wrote:
> Hi Mahesh,
>
> the problem is that every column is only tombstoned for as long as the
> original column was valid.
>
> So if the last update was only valid for 1 sec, then the tombstone will
> also be valid for 1 second! If the previous was valid for a longer time,
> then this old value might reappear.
>
> Maybe you can explain why you are doing this?
>
> kind regards,
> Christian
>
>
>
> On Mon, Feb 17, 2014 at 6:18 PM, mahesh rajamani <
> rajamani.mahesh@gmail.com> wrote:
>
>> Christain,
>>
>> Yes. Is it a problem? Can you explain what happens in this scenario?
>>
>> Thanks
>> Mahesh
>>
>>
>> On Fri, Feb 14, 2014 at 3:07 PM, horschi <ho...@gmail.com> wrote:
>>
>>> Hi Mahesh,
>>>
>>> is it possible you are creating columns with a long TTL, then update
>>> these columns with a smaller TTL?
>>>
>>> kind regards,
>>> Christian
>>>
>>>
>>> On Fri, Feb 14, 2014 at 3:45 PM, mahesh rajamani <
>>> rajamani.mahesh@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am using Cassandra 2.0.2 version. On a wide row (approx. 10000
>>>> columns), I expire few column by setting TTL as 1 second. At times these
>>>> columns show up during slice query.
>>>>
>>>> When I have this issue, running count and get commands for that row
>>>> using Cassandra cli it gives different column counts.
>>>>
>>>> But once I run flush and compact, the issue goes off and expired
>>>> columns don't show up.
>>>>
>>>> Can someone provide some help on this issue.
>>>>
>>>> --
>>>> Regards,
>>>> Mahesh Rajamani
>>>>
>>>
>>>
>>
>>
>> --
>> Regards,
>> Mahesh Rajamani
>>
>
>
--
Regards,
Mahesh Rajamani
Re: Expired column showing up
Posted by horschi <ho...@gmail.com>.
Hi Mahesh,
the problem is that every column is only tombstoned for as long as the
original column was valid.
So if the last update was only valid for 1 sec, then the tombstone will
also be valid for 1 second! If the previous was valid for a longer time,
then this old value might reappear.
Maybe you can explain why you are doing this?
kind regards,
Christian
On Mon, Feb 17, 2014 at 6:18 PM, mahesh rajamani
<ra...@gmail.com>wrote:
> Christain,
>
> Yes. Is it a problem? Can you explain what happens in this scenario?
>
> Thanks
> Mahesh
>
>
> On Fri, Feb 14, 2014 at 3:07 PM, horschi <ho...@gmail.com> wrote:
>
>> Hi Mahesh,
>>
>> is it possible you are creating columns with a long TTL, then update
>> these columns with a smaller TTL?
>>
>> kind regards,
>> Christian
>>
>>
>> On Fri, Feb 14, 2014 at 3:45 PM, mahesh rajamani <
>> rajamani.mahesh@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am using Cassandra 2.0.2 version. On a wide row (approx. 10000
>>> columns), I expire few column by setting TTL as 1 second. At times these
>>> columns show up during slice query.
>>>
>>> When I have this issue, running count and get commands for that row
>>> using Cassandra cli it gives different column counts.
>>>
>>> But once I run flush and compact, the issue goes off and expired columns
>>> don't show up.
>>>
>>> Can someone provide some help on this issue.
>>>
>>> --
>>> Regards,
>>> Mahesh Rajamani
>>>
>>
>>
>
>
> --
> Regards,
> Mahesh Rajamani
>
Re: Expired column showing up
Posted by mahesh rajamani <ra...@gmail.com>.
Christain,
Yes. Is it a problem? Can you explain what happens in this scenario?
Thanks
Mahesh
On Fri, Feb 14, 2014 at 3:07 PM, horschi <ho...@gmail.com> wrote:
> Hi Mahesh,
>
> is it possible you are creating columns with a long TTL, then update these
> columns with a smaller TTL?
>
> kind regards,
> Christian
>
>
> On Fri, Feb 14, 2014 at 3:45 PM, mahesh rajamani <
> rajamani.mahesh@gmail.com> wrote:
>
>> Hi,
>>
>> I am using Cassandra 2.0.2 version. On a wide row (approx. 10000
>> columns), I expire few column by setting TTL as 1 second. At times these
>> columns show up during slice query.
>>
>> When I have this issue, running count and get commands for that row using
>> Cassandra cli it gives different column counts.
>>
>> But once I run flush and compact, the issue goes off and expired columns
>> don't show up.
>>
>> Can someone provide some help on this issue.
>>
>> --
>> Regards,
>> Mahesh Rajamani
>>
>
>
--
Regards,
Mahesh Rajamani
Re: Expired column showing up
Posted by horschi <ho...@gmail.com>.
Hi Mahesh,
is it possible you are creating columns with a long TTL, then update these
columns with a smaller TTL?
kind regards,
Christian
On Fri, Feb 14, 2014 at 3:45 PM, mahesh rajamani
<ra...@gmail.com>wrote:
> Hi,
>
> I am using Cassandra 2.0.2 version. On a wide row (approx. 10000 columns),
> I expire few column by setting TTL as 1 second. At times these columns show
> up during slice query.
>
> When I have this issue, running count and get commands for that row using
> Cassandra cli it gives different column counts.
>
> But once I run flush and compact, the issue goes off and expired columns
> don't show up.
>
> Can someone provide some help on this issue.
>
> --
> Regards,
> Mahesh Rajamani
>
Re: Expired column showing up
Posted by Edward Capriolo <ed...@gmail.com>.
You should upgrade. Cassandra 2.0.2 is not the latest version. If you still
have the problem report a bug.
On Fri, Feb 14, 2014 at 12:50 PM, Yogi Nerella <yn...@gmail.com>wrote:
> I am just learning, I don't know answer to your question, but What is the
> use case for TTL as 1 second?
>
>
>
>
> On Fri, Feb 14, 2014 at 6:45 AM, mahesh rajamani <
> rajamani.mahesh@gmail.com> wrote:
>
>> Hi,
>>
>> I am using Cassandra 2.0.2 version. On a wide row (approx. 10000
>> columns), I expire few column by setting TTL as 1 second. At times these
>> columns show up during slice query.
>>
>> When I have this issue, running count and get commands for that row using
>> Cassandra cli it gives different column counts.
>>
>> But once I run flush and compact, the issue goes off and expired columns
>> don't show up.
>>
>> Can someone provide some help on this issue.
>>
>> --
>> Regards,
>> Mahesh Rajamani
>>
>
>
Re: Expired column showing up
Posted by Yogi Nerella <yn...@gmail.com>.
I am just learning, I don't know answer to your question, but What is the
use case for TTL as 1 second?
On Fri, Feb 14, 2014 at 6:45 AM, mahesh rajamani
<ra...@gmail.com>wrote:
> Hi,
>
> I am using Cassandra 2.0.2 version. On a wide row (approx. 10000 columns),
> I expire few column by setting TTL as 1 second. At times these columns show
> up during slice query.
>
> When I have this issue, running count and get commands for that row using
> Cassandra cli it gives different column counts.
>
> But once I run flush and compact, the issue goes off and expired columns
> don't show up.
>
> Can someone provide some help on this issue.
>
> --
> Regards,
> Mahesh Rajamani
>