missing a sub column

[Jeesoo Shin <bs...@gmail.com>]

Hi,
I'm using Hector 0.8.0-2 & Cassandra 0.8.6 on EC2 XLarge systems.
I have a Supercolumn with 10 sub columns.
Most of times it works, but occasionally I find last sub column is
missing completely.

I checked our program that it's successfully inserting all 10 sub
columns into HColumn
just before mutator.insert or mutator.addInsertion (call execute at count 50).
To check if Cassandra is losing the data, I made a small program to
read commitlog file.
Well, I found logs with only 9 sub columns.

Now I'm lost... I have no idea where and why a sub column is missing.
Could there be a problem in thrift transition? (which happens to work
most of times.. )
ANY IDEAS? plz.



one strange thing I notice in commitlog,
4 - total length
8 - unknown
2 - length of keyspace
# - keyspace
2 - length of row key of column family
# - row key of column family
9 - unknown : 8,9th is column family id.
16 - unknown
(if supercolumn)
2 - length of super column name
# - super column name
14 - unknown
2 - number of batch

In local system, using TC, number of batch displays number of mutator.
when mutator.insert, it's 1
when mutator.addInsertion(50 or less), it's 50 or less
I presume 0 is delete.
The strange part is I see much larger number of batch,  149, 201,~ up to 1024.
Does hector merge insert for optimization? Could this optimization
part lose a sub column?
I don't think cassandra will check last commit, merge, +1 batch#,..



I'm very very lost. :(
any comment will help.
Thank you.