Column TTL misbehavior?

[Javier Canillas <ja...@gmail.com>]

I have been working on Aquiles for some time, and some of the people that is
actually using my client to consume Cassandra's services (lastest version on
Cassandra 0.7.0) has followed me an issue. Básically, they were trying to
submit a new column with a TTL value of 852282524, and everytime they tried
to insert int, the following Thrift error was going back to them:

Thrift.TApplicationException: Internal error processing insert
at Apache.Cassandra.Cassandra.Client.recv_insert() in
D:\apache-cassandra-0.7.0-beta3\interface\gen-csharp\Apache\Cassandra\Cassandra.cs:line
485
at Apache.Cassandra.Cassandra.Client.insert(Byte[] key, ColumnParent
column_parent, Column column, ConsistencyLevel consistency_level) in
D:\apache-cassandra-0.7.0-beta3\interface\gen-csharp\Apache\Cassandra\Cassandra.cs:line
463

And the log within Cassandra was:

ERROR [pool-1-thread-45] 2011-01-16 19:03:35,514 Cassandra.java (line 2960)
Internal error processing insert
java.lang.AssertionError: -2147462157
at org.apache.cassandra.db.ExpiringColumn.<init>(ExpiringColumn.java:57)
at org.apache.cassandra.db.ExpiringColumn.<init>(ExpiringColumn.java:50)
at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:167)
at org.apache.cassandra.db.RowMutation.add(RowMutation.java:151)
at
org.apache.cassandra.thrift.CassandraServer.insert(CassandraServer.java:343)
at
org.apache.cassandra.thrift.Cassandra$Processor$insert.process(Cassandra.java:2952)
at
org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2555)
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

I know that a TTL value of 9864 days is way too long, but after stepping
into the code, I found out the following:

public ExpiringColumn(ByteBuffer name, ByteBuffer value, long timestamp, int
timeToLive)
    {
      this(name, value, timestamp, timeToLive, (int)
(System.currentTimeMillis() / 1000) + timeToLive);
    }

So, the constructor of ExpiringColumn was basically converting a long into
an int and then adding the TTL value for that column. this value is the
placed on localExpirationTime which is used to check to see if the Column is
mark for deletation. This last variable is defined as integer.

So, after doing some math and rechecking the code, I think I have come
across a serious problem which means that, as time passes, the maximum
posible TTL value that might be set to an Expiring column must be each
second lower than before.

This is a huge limitation, as for now (using System.currentTimeMillis() from
my machine) the maximum posible TTL value is 9861 days (851990400 seconds)
now, but tomorrow it will be 9860 days and so on. So each day we came closer
to this issue.

I think that a good solution is changing the localExpirationTime from int to
long and, avoid converting a long value to its integer representation.

Just give me your thoughts about this.

Javier Canillas

Re: Column TTL misbehavior?

[Javier Canillas <ja...@gmail.com>]

Avoid this subject, giving it a second thought, There is no reason to put a
column a TTL value that goes past a year, and this problem will only appear
after the year 2038.

I need to stop drinking coffee.

On Wed, Jan 19, 2011 at 7:08 PM, Javier Canillas
<ja...@gmail.com>wrote:

> I have been working on Aquiles for some time, and some of the people that
> is actually using my client to consume Cassandra's services (lastest
> version on Cassandra 0.7.0) has followed me an issue. Básically, they were
> trying to submit a new column with a TTL value of 852282524, and everytime
> they tried to insert int, the following Thrift error was going back to them:
>
> Thrift.TApplicationException: Internal error processing insert
> at Apache.Cassandra.Cassandra.Client.recv_insert() in
> D:\apache-cassandra-0.7.0-beta3\interface\gen-csharp\Apache\Cassandra\Cassandra.cs:line
> 485
> at Apache.Cassandra.Cassandra.Client.insert(Byte[] key, ColumnParent
> column_parent, Column column, ConsistencyLevel consistency_level) in
> D:\apache-cassandra-0.7.0-beta3\interface\gen-csharp\Apache\Cassandra\Cassandra.cs:line
> 463
>
> And the log within Cassandra was:
>
> ERROR [pool-1-thread-45] 2011-01-16 19:03:35,514 Cassandra.java (line 2960)
> Internal error processing insert
> java.lang.AssertionError: -2147462157
> at org.apache.cassandra.db.ExpiringColumn.<init>(ExpiringColumn.java:57)
> at org.apache.cassandra.db.ExpiringColumn.<init>(ExpiringColumn.java:50)
> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:167)
> at org.apache.cassandra.db.RowMutation.add(RowMutation.java:151)
> at
> org.apache.cassandra.thrift.CassandraServer.insert(CassandraServer.java:343)
> at
> org.apache.cassandra.thrift.Cassandra$Processor$insert.process(Cassandra.java:2952)
> at
> org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2555)
> at
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
>
> I know that a TTL value of 9864 days is way too long, but after stepping
> into the code, I found out the following:
>
> public ExpiringColumn(ByteBuffer name, ByteBuffer value, long timestamp,
> int timeToLive)
>     {
>       this(name, value, timestamp, timeToLive, (int)
> (System.currentTimeMillis() / 1000) + timeToLive);
>     }
>
> So, the constructor of ExpiringColumn was basically converting a long into
> an int and then adding the TTL value for that column. this value is the
> placed on localExpirationTime which is used to check to see if the Column is
> mark for deletation. This last variable is defined as integer.
>
> So, after doing some math and rechecking the code, I think I have come
> across a serious problem which means that, as time passes, the maximum
> posible TTL value that might be set to an Expiring column must be each
> second lower than before.
>
> This is a huge limitation, as for now (using System.currentTimeMillis()
> from my machine) the maximum posible TTL value is 9861 days (851990400
> seconds) now, but tomorrow it will be 9860 days and so on. So each day we
> came closer to this issue.
>
> I think that a good solution is changing the localExpirationTime from int
> to long and, avoid converting a long value to its integer representation.
>
> Just give me your thoughts about this.
>
> Javier Canillas
>
>
>
>
>
>
>