TTL rows with 2i

[Eduardo Alonso <ed...@stratio.com>]

Hi,

We have been investigating how to include in our 2i implementation the
ability to index TTL expirable Cells in Cassandra 3.x.

Reading comments in o.a.c.index.Index.Indexer.removeRows it seems that this
method is called when a compaction detects that a cell has expired.

I dont know if this is correct, so the question is:

How does the non-columnFamily based 2i get notified when a row ttl has
expired?

I have checked the only two calls to Indexer.removeRows in
o.a.c.index.SecondaryIndexManager.IndexGCTransaction.commit in the
followings releases and it seems that has not changed at all.

Sam Tunnicliffe, any ideas about this??

Regards


Eduardo Alonso
Vía de las dos Castillas, 33, Ática 4, 3ª Planta
28224 Pozuelo de Alarcón, Madrid
Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
<https://twitter.com/StratioBD>*

Re: TTL rows with 2i

[Eduardo Alonso <ed...@stratio.com>]

Thank you, sam, for the fast clarification.
Regards
El 9/3/2016 7:03 p. m., "Sam Tunnicliffe" <sa...@beobal.com> escribió:

> The problem with row level ttls in this regard is that the scope of a
> particular compaction may not include all versions of a given row. So just
> because a primary key liveness denoting an expired row may be written to
> the new SSTable, it doesn't necessarily mean that an index should purge its
> entry/entries for that row. That said, it should probably be the
> responsibility of the index implementation to manage that, but at the
> moment the {{onPrimaryKeyLivenessInfo}} event during compaction doesn't
> cause the registered indexes to be notified. I've opened
> https://issues.apache.org/jira/browse/CASSANDRA-11329 for this.
>
>
> On Wed, Mar 9, 2016 at 4:05 PM, Eduardo Alonso <ed...@stratio.com>
> wrote:
>
> > Hi,
> >
> > We have been investigating how to include in our 2i implementation the
> > ability to index TTL expirable Cells in Cassandra 3.x.
> >
> > Reading comments in o.a.c.index.Index.Indexer.removeRows it seems that
> this
> > method is called when a compaction detects that a cell has expired.
> >
> > I dont know if this is correct, so the question is:
> >
> > How does the non-columnFamily based 2i get notified when a row ttl has
> > expired?
> >
> > I have checked the only two calls to Indexer.removeRows in
> > o.a.c.index.SecondaryIndexManager.IndexGCTransaction.commit in the
> > followings releases and it seems that has not changed at all.
> >
> > Sam Tunnicliffe, any ideas about this??
> >
> > Regards
> >
> >
> > Eduardo Alonso
> > Vía de las dos Castillas, 33, Ática 4, 3ª Planta
> > 28224 Pozuelo de Alarcón, Madrid
> > Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
> > <https://twitter.com/StratioBD>*
> >
>

Re: TTL rows with 2i

[Sam Tunnicliffe <sa...@beobal.com>]

The problem with row level ttls in this regard is that the scope of a
particular compaction may not include all versions of a given row. So just
because a primary key liveness denoting an expired row may be written to
the new SSTable, it doesn't necessarily mean that an index should purge its
entry/entries for that row. That said, it should probably be the
responsibility of the index implementation to manage that, but at the
moment the {{onPrimaryKeyLivenessInfo}} event during compaction doesn't
cause the registered indexes to be notified. I've opened
https://issues.apache.org/jira/browse/CASSANDRA-11329 for this.


On Wed, Mar 9, 2016 at 4:05 PM, Eduardo Alonso <ed...@stratio.com>
wrote:

> Hi,
>
> We have been investigating how to include in our 2i implementation the
> ability to index TTL expirable Cells in Cassandra 3.x.
>
> Reading comments in o.a.c.index.Index.Indexer.removeRows it seems that this
> method is called when a compaction detects that a cell has expired.
>
> I dont know if this is correct, so the question is:
>
> How does the non-columnFamily based 2i get notified when a row ttl has
> expired?
>
> I have checked the only two calls to Indexer.removeRows in
> o.a.c.index.SecondaryIndexManager.IndexGCTransaction.commit in the
> followings releases and it seems that has not changed at all.
>
> Sam Tunnicliffe, any ideas about this??
>
> Regards
>
>
> Eduardo Alonso
> Vía de las dos Castillas, 33, Ática 4, 3ª Planta
> 28224 Pozuelo de Alarcón, Madrid
> Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
> <https://twitter.com/StratioBD>*
>