You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Philippe <wa...@gmail.com> on 2011/08/07 23:27:31 UTC
Replicate on write stage errors
hello,
I've got new errors showing up in my cassandra log file since I starting
testing batch mutates (and it failed).
I have done a rolling restart and they are not disappearing.
How can I fix this ? What is this really saying about my data and my cluster
?
Thanks
ERROR [ReplicateOnWriteStage:35] 2011-08-07 23:22:39,147
AbstractCassandraDaemon.java (line 139) Fatal exception in thread
Thread[ReplicateOnWriteStage:35,5,main]
java.lang.RuntimeException: java.lang.IllegalArgumentException: ColumnFamily
ColumnFamily(PUBLIC_MONTHLY_11 [SuperColumn(310260
[00000000000200:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd, 2,
6}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
6}]@1312752159034!-9223372036854775808,00000000000201:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
2, -378}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
-378}]@1312752159034!-9223372036854775808,00000000000300:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
17, 12124}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 28,
14097},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 15,
3737}]@1312752159034!-9223372036854775808,00000000000301:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
17, -917234}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 28,
-1148333},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 15,
-287859}]@1312752159034!-9223372036854775808,]),]) already has modifications
in this mutation: ColumnFamily(PUBLIC_MONTHLY_11 [SuperColumn(31026
[00000000000200:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd, 13,
387}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 8,
1991},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 6,
237}]@1312752159034!-9223372036854775808,00000000000201:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
13, -31485}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 8,
-138769},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 6,
-18209}]@1312752159034!-9223372036854775808,00000000000300:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
2, 8}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
8}]@1312752159034!-9223372036854775808,00000000000301:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
2, -708}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
-708}]@1312752159034!-9223372036854775808,]),])
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalArgumentException: ColumnFamily
ColumnFamily(PUBLIC_MONTHLY_11 [SuperColumn(310260
[00000000000200:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd, 2,
6}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
6}]@1312752159034!-9223372036854775808,00000000000201:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
2, -378}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
-378}]@1312752159034!-9223372036854775808,00000000000300:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
17, 12124}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 28,
14097},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 15,
3737}]@1312752159034!-9223372036854775808,00000000000301:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
17, -917234}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 28,
-1148333},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 15,
-287859}]@1312752159034!-9223372036854775808,]),]) already has modifications
in this mutation: ColumnFamily(PUBLIC_MONTHLY_11 [SuperColumn(31026
[00000000000200:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd, 13,
387}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 8,
1991},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 6,
237}]@1312752159034!-9223372036854775808,00000000000201:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
13, -31485}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 8,
-138769},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 6,
-18209}]@1312752159034!-9223372036854775808,00000000000300:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
2, 8}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
8}]@1312752159034!-9223372036854775808,00000000000301:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
2, -708}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
-708}]@1312752159034!-9223372036854775808,]),])
at org.apache.cassandra.db.RowMutation.add(RowMutation.java:123)
at
org.apache.cassandra.db.CounterMutation.makeReplicationMutation(CounterMutation.java:120)
at
org.apache.cassandra.service.StorageProxy$5$1.runMayThrow(StorageProxy.java:455)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
Re: Replicate on write stage errors
Posted by Sylvain Lebresne <sy...@datastax.com>.
On Sun, Aug 7, 2011 at 11:27 PM, Philippe <wa...@gmail.com> wrote:
> hello,
> I've got new errors showing up in my cassandra log file since I starting
> testing batch mutates (and it failed).
> I have done a rolling restart and they are not disappearing.
> How can I fix this ?
It's a known bug:
https://issues.apache.org/jira/browse/CASSANDRA-2949
You can fix by applying the patch attached to that issue or waiting
for 0.8.3. Good news being, it is
under a vote, and unless a big regression if found, it should be
released very soon (tomorrow, maybe
today).
--
Sylvain
> What is this really saying about my data and my cluster ?
> Thanks
> ERROR [ReplicateOnWriteStage:35] 2011-08-07 23:22:39,147
> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
> Thread[ReplicateOnWriteStage:35,5,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException: ColumnFamily
> ColumnFamily(PUBLIC_MONTHLY_11 [SuperColumn(310260
> [00000000000200:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd, 2,
> 6}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
> 6}]@1312752159034!-9223372036854775808,00000000000201:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 2, -378}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
> -378}]@1312752159034!-9223372036854775808,00000000000300:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 17, 12124}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 28,
> 14097},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 15,
> 3737}]@1312752159034!-9223372036854775808,00000000000301:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 17, -917234}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 28,
> -1148333},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 15,
> -287859}]@1312752159034!-9223372036854775808,]),]) already has modifications
> in this mutation: ColumnFamily(PUBLIC_MONTHLY_11 [SuperColumn(31026
> [00000000000200:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd, 13,
> 387}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 8,
> 1991},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 6,
> 237}]@1312752159034!-9223372036854775808,00000000000201:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 13, -31485}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 8,
> -138769},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 6,
> -18209}]@1312752159034!-9223372036854775808,00000000000300:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 2, 8}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
> 8}]@1312752159034!-9223372036854775808,00000000000301:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 2, -708}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
> -708}]@1312752159034!-9223372036854775808,]),])
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException: ColumnFamily
> ColumnFamily(PUBLIC_MONTHLY_11 [SuperColumn(310260
> [00000000000200:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd, 2,
> 6}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
> 6}]@1312752159034!-9223372036854775808,00000000000201:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 2, -378}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
> -378}]@1312752159034!-9223372036854775808,00000000000300:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 17, 12124}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 28,
> 14097},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 15,
> 3737}]@1312752159034!-9223372036854775808,00000000000301:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 17, -917234}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 28,
> -1148333},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 15,
> -287859}]@1312752159034!-9223372036854775808,]),]) already has modifications
> in this mutation: ColumnFamily(PUBLIC_MONTHLY_11 [SuperColumn(31026
> [00000000000200:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd, 13,
> 387}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 8,
> 1991},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 6,
> 237}]@1312752159034!-9223372036854775808,00000000000201:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 13, -31485}*,{224ceb80-b5fb-11e0-0000-848783ceb9bf, 8,
> -138769},{e0db20e0-b5ff-11e0-0000-01687c4831ff, 6,
> -18209}]@1312752159034!-9223372036854775808,00000000000300:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 2, 8}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
> 8}]@1312752159034!-9223372036854775808,00000000000301:false:[{223a4de0-b5fb-11e0-0000-826f85850cbd,
> 2, -708}*,{e0db20e0-b5ff-11e0-0000-01687c4831ff, 2,
> -708}]@1312752159034!-9223372036854775808,]),])
> at org.apache.cassandra.db.RowMutation.add(RowMutation.java:123)
> at
> org.apache.cassandra.db.CounterMutation.makeReplicationMutation(CounterMutation.java:120)
> at
> org.apache.cassandra.service.StorageProxy$5$1.runMayThrow(StorageProxy.java:455)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
>