You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Robin Verlangen <ro...@us2.nl> on 2012/09/10 11:47:31 UTC
Long-life TTL and extending TTL
Hi there,
I'm working on a project that might want to set TTL to roughly 7 years.
However it might occur that the TTL should be reduced or extended. Is there
any way of updating the TTL without being in need of rewriting the data
back again? This would cause way to much overhead for this.
If not, is running a Map/Reduce task on the whole data set the "best"
option or should I think in a difference approach for this challenge?
My last question is regarding to a long term TTL, does this have any
negative impact on the cluster? Maybe during compaction, repair,
reading/writing?
Best regards,
Robin Verlangen
*Software engineer*
*
*
W http://www.robinverlangen.nl
E robin@us2.nl
Disclaimer: The information contained in this message and attachments is
intended solely for the attention and use of the named addressee and may be
confidential. If you are not the intended recipient, you are reminded that
the information remains the property of the sender. You must not use,
disclose, distribute, copy, print or rely on this e-mail. If you have
received this message in error, please contact the sender immediately and
irrevocably delete this message and any copies.
Re: Long-life TTL and extending TTL
Posted by Sylvain Lebresne <sy...@datastax.com>.
> Is there any way of updating the TTL without being in need of rewriting the data back again?
No, there isn't.
> If not, is running a Map/Reduce task on the whole data set the "best" option
If the TTL is made rather infrequently and on a large percentage of
the data, which seems to be your case, then that's likely the option I
would use.
> My last question is regarding to a long term TTL, does this have any
> negative impact on the cluster?
The main drawback of TTL is that each column has a 8 bytes overhead (a
tad more details here:
http://www.datastax.com/dev/blog/whats-new-cassandra-07-expiring-columns).
It's not a big deal imo, especially now that we have compression, but
it's worth knowing. There is nothing specific to having "long term"
TTL in any case.
--
Sylvain
Re: Long-life TTL and extending TTL
Posted by Robin Verlangen <ro...@us2.nl>.
OK, that's an option indeed. However due to the amount of records it would
also involve bucketing which makes it not the simplest option. Further
more, there are lots of manual indexes referring to the keys of the actual
events: all those indexes should also be updated.
Best regards,
Robin Verlangen
*Software engineer*
*
*
W http://www.robinverlangen.nl
E robin@us2.nl
Disclaimer: The information contained in this message and attachments is
intended solely for the attention and use of the named addressee and may be
confidential. If you are not the intended recipient, you are reminded that
the information remains the property of the sender. You must not use,
disclose, distribute, copy, print or rely on this e-mail. If you have
received this message in error, please contact the sender immediately and
irrevocably delete this message and any copies.
2012/9/10 Oleg Dulin <ol...@gmail.com>
> **
>
> You should create an index where you store references to your records. You
> can use composite column names where column name=composite(timestamp,key)
>
>
> then you would get a slice of all columns where timestamp part of the
> composite is >= TTL in the past, and then iterate through them and delete
> the items.
>
>
> Regards,
>
> Oleg
>
>
>
> On 2012-09-10 09:47:31 +0000, Robin Verlangen said:
>
>
> Hi there,
>
>
> I'm working on a project that might want to set TTL to roughly 7 years.
> However it might occur that the TTL should be reduced or extended. Is there
> any way of updating the TTL without being in need of rewriting the data
> back again? This would cause way to much overhead for this.
>
>
> If not, is running a Map/Reduce task on the whole data set the "best"
> option or should I think in a difference approach for this challenge?
>
>
> My last question is regarding to a long term TTL, does this have any
> negative impact on the cluster? Maybe during compaction, repair,
> reading/writing?
>
>
> Best regards,
>
>
> Robin Verlangen
>
> Software engineer
>
>
> W http://www.robinverlangen.nl
>
> E robin@us2.nl
>
>
> Disclaimer: The information contained in this message and attachments is
> intended solely for the attention and use of the named addressee and may be
> confidential. If you are not the intended recipient, you are reminded that
> the information remains the property of the sender. You must not use,
> disclose, distribute, copy, print or rely on this e-mail. If you have
> received this message in error, please contact the sender immediately and
> irrevocably delete this message and any copies.
>
>
>
>
>
Re: Long-life TTL and extending TTL
Posted by Oleg Dulin <ol...@gmail.com>.
You should create an index where you store references to your records.
You can use composite column names where column
name=composite(timestamp,key)
then you would get a slice of all columns where timestamp part of the
composite is >= TTL in the past, and then iterate through them and
delete the items.
Regards,
Oleg
On 2012-09-10 09:47:31 +0000, Robin Verlangen said:
> Hi there,
>
> I'm working on a project that might want to set TTL to roughly 7 years.
> However it might occur that the TTL should be reduced or extended. Is
> there any way of updating the TTL without being in need of rewriting
> the data back again? This would cause way to much overhead for this.
>
> If not, is running a Map/Reduce task on the whole data set the "best"
> option or should I think in a difference approach for this challenge?
>
> My last question is regarding to a long term TTL, does this have any
> negative impact on the cluster? Maybe during compaction, repair,
> reading/writing?
>
> Best regards,
>
> Robin Verlangen
> Software engineer
>
> W http://www.robinverlangen.nl
> E robin@us2.nl
>
> Disclaimer: The information contained in this message and attachments
> is intended solely for the attention and use of the named addressee and
> may be confidential. If you are not the intended recipient, you are
> reminded that the information remains the property of the sender. You
> must not use, disclose, distribute, copy, print or rely on this e-mail.
> If you have received this message in error, please contact the sender
> immediately and irrevocably delete this message and any copies.
Re: Long-life TTL and extending TTL
Posted by Robin Verlangen <ro...@us2.nl>.
Hi Sergey,
That's exactly what I mean. I really hope that this will get released soon!
Best regards,
Robin Verlangen
*Software engineer*
*
*
W http://www.robinverlangen.nl
E robin@us2.nl
Disclaimer: The information contained in this message and attachments is
intended solely for the attention and use of the named addressee and may be
confidential. If you are not the intended recipient, you are reminded that
the information remains the property of the sender. You must not use,
disclose, distribute, copy, print or rely on this e-mail. If you have
received this message in error, please contact the sender immediately and
irrevocably delete this message and any copies.
2012/9/15 Sergey Tryuber <st...@gmail.com>
> Hi Robin,
>
> We have the same headache and hope that
> https://issues.apache.org/jira/browse/CASSANDRA-3974 will be a balsam.
>
>
> On 10 September 2012 13:47, Robin Verlangen <ro...@us2.nl> wrote:
>
>> Hi there,
>>
>> I'm working on a project that might want to set TTL to roughly 7 years.
>> However it might occur that the TTL should be reduced or extended. Is there
>> any way of updating the TTL without being in need of rewriting the data
>> back again? This would cause way to much overhead for this.
>>
>> If not, is running a Map/Reduce task on the whole data set the "best"
>> option or should I think in a difference approach for this challenge?
>>
>> My last question is regarding to a long term TTL, does this have any
>> negative impact on the cluster? Maybe during compaction, repair,
>> reading/writing?
>>
>> Best regards,
>>
>> Robin Verlangen
>> *Software engineer*
>> *
>> *
>> W http://www.robinverlangen.nl
>> E robin@us2.nl
>>
>> Disclaimer: The information contained in this message and attachments is
>> intended solely for the attention and use of the named addressee and may be
>> confidential. If you are not the intended recipient, you are reminded that
>> the information remains the property of the sender. You must not use,
>> disclose, distribute, copy, print or rely on this e-mail. If you have
>> received this message in error, please contact the sender immediately and
>> irrevocably delete this message and any copies.
>>
>>
>
Re: Long-life TTL and extending TTL
Posted by Sergey Tryuber <st...@gmail.com>.
Hi Robin,
We have the same headache and hope that
https://issues.apache.org/jira/browse/CASSANDRA-3974 will be a balsam.
On 10 September 2012 13:47, Robin Verlangen <ro...@us2.nl> wrote:
> Hi there,
>
> I'm working on a project that might want to set TTL to roughly 7 years.
> However it might occur that the TTL should be reduced or extended. Is there
> any way of updating the TTL without being in need of rewriting the data
> back again? This would cause way to much overhead for this.
>
> If not, is running a Map/Reduce task on the whole data set the "best"
> option or should I think in a difference approach for this challenge?
>
> My last question is regarding to a long term TTL, does this have any
> negative impact on the cluster? Maybe during compaction, repair,
> reading/writing?
>
> Best regards,
>
> Robin Verlangen
> *Software engineer*
> *
> *
> W http://www.robinverlangen.nl
> E robin@us2.nl
>
> Disclaimer: The information contained in this message and attachments is
> intended solely for the attention and use of the named addressee and may be
> confidential. If you are not the intended recipient, you are reminded that
> the information remains the property of the sender. You must not use,
> disclose, distribute, copy, print or rely on this e-mail. If you have
> received this message in error, please contact the sender immediately and
> irrevocably delete this message and any copies.
>
>