You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "ZhaoYang (JIRA)" <ji...@apache.org> on 2016/08/09 15:24:20 UTC
[jira] [Commented] (CASSANDRA-12152) Unknown exception caught while
attempting to update MaterializedView: AssertionError: Flags = 128
[ https://issues.apache.org/jira/browse/CASSANDRA-12152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413712#comment-15413712 ]
ZhaoYang commented on CASSANDRA-12152:
--------------------------------------
it seems that the row flag is being read as 'static', but Cassandra is expecting 'Row' or 'Tombstone' type.
> Unknown exception caught while attempting to update MaterializedView: AssertionError: Flags = 128
> -------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-12152
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12152
> Project: Cassandra
> Issue Type: Bug
> Reporter: Nilson Pontello
>
> I have a single DC with 3 cassandra nodes. After a restart today, none of them were capable of processing the commitlog while starting up. The exception doesn't contains enough information about what is going on, please check bellow:
> {code}
> ERROR [SharedPool-Worker-21] 2016-07-08 12:42:12,866 Keyspace.java:521 - Unknown exception caught while attempting to update MaterializedView! data_monitor.user_timeline
> java.lang.AssertionError: Flags = 128
> at org.apache.cassandra.db.ClusteringPrefix$Deserializer.prepare(ClusteringPrefix.java:421) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.prepareNext(UnfilteredDeserializer.java:172) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.hasNext(UnfilteredDeserializer.java:153) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.handlePreSliceData(SSTableIterator.java:96) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:141) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:354) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:229) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:93) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:25) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:70) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.withSSTablesIterated(SinglePartitionReadCommand.java:637) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:586) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:463) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:325) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:366) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.ReadCommand.executeInternal(ReadCommand.java:397) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.service.pager.AbstractQueryPager.fetchPageInternal(AbstractQueryPager.java:77) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.service.pager.SinglePartitionPager.fetchPageInternal(SinglePartitionPager.java:34) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.view.View.readLocalRows(View.java:565) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.view.View.getTemporalRowSet(View.java:621) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:136) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:516) [apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:409) [apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.commitlog.CommitLogReplayer$MutationInitiator$1.runMayThrow(CommitLogReplayer.java:157) [apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) [apache-cassandra-3.5.jar:3.5]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_91]
> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) [apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [apache-cassandra-3.5.jar:3.5]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> WARN [SharedPool-Worker-21] 2016-07-08 12:42:12,867 AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread Thread[SharedPool-Worker-21,5,main]: {}
> java.lang.AssertionError: Flags = 128
> at org.apache.cassandra.db.ClusteringPrefix$Deserializer.prepare(ClusteringPrefix.java:421) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.prepareNext(UnfilteredDeserializer.java:172) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.hasNext(UnfilteredDeserializer.java:153) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.handlePreSliceData(SSTableIterator.java:96) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:141) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:354) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:229) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:93) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:25) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:70) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.withSSTablesIterated(SinglePartitionReadCommand.java:637) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:586) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:463) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:325) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:366) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.ReadCommand.executeInternal(ReadCommand.java:397) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.service.pager.AbstractQueryPager.fetchPageInternal(AbstractQueryPager.java:77) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.service.pager.SinglePartitionPager.fetchPageInternal(SinglePartitionPager.java:34) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.view.View.readLocalRows(View.java:565) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.view.View.getTemporalRowSet(View.java:621) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:136) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:516) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:409) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.commitlog.CommitLogReplayer$MutationInitiator$1.runMayThrow(CommitLogReplayer.java:157) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.5.jar:3.5]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_91]
> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [apache-cassandra-3.5.jar:3.5]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> ERROR [main] 2016-07-08 12:42:12,875 CassandraDaemon.java:697 - Exception encountered during startup
> java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError: Flags = 128
> at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:384) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.commitlog.CommitLogReplayer.replayMutation(CommitLogReplayer.java:592) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.commitlog.CommitLogReplayer.replaySyncSection(CommitLogReplayer.java:521) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:407) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:236) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:192) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:172) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:283) [apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551) [apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:680) [apache-cassandra-3.5.jar:3.5]
> Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError: Flags = 128
> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.get(AbstractLocalAwareExecutorService.java:200) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:380) ~[apache-cassandra-3.5.jar:3.5]
> ... 9 common frames omitted
> Caused by: java.lang.AssertionError: Flags = 128
> at org.apache.cassandra.db.ClusteringPrefix$Deserializer.prepare(ClusteringPrefix.java:421) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.prepareNext(UnfilteredDeserializer.java:172) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.hasNext(UnfilteredDeserializer.java:153) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.handlePreSliceData(SSTableIterator.java:96) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:141) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:354) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:229) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:93) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:25) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:70) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.withSSTablesIterated(SinglePartitionReadCommand.java:637) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:586) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:463) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:325) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:366) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.ReadCommand.executeInternal(ReadCommand.java:397) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.service.pager.AbstractQueryPager.fetchPageInternal(AbstractQueryPager.java:77) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.service.pager.SinglePartitionPager.fetchPageInternal(SinglePartitionPager.java:34) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.view.View.readLocalRows(View.java:565) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.view.View.getTemporalRowSet(View.java:621) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:136) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:516) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.Keyspace.applyFromCommitLog(Keyspace.java:409) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.db.commitlog.CommitLogReplayer$MutationInitiator$1.runMayThrow(CommitLogReplayer.java:157) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.5.jar:3.5]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_91]
> at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[apache-cassandra-3.5.jar:3.5]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) ~[apache-cassandra-3.5.jar:3.5]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
> {code}
> I have played with *commit_failure_policy* hoping that this could be ignored but it wasn't. I don't mind loosing this materialized view, I can create it over again if needed.
> I am opening this bug because the error message is not informative and I have no clues of what should be done in order to put my cluster live again...
> Also, my system administrator did stop cassandra without running the bellow commands:
> {code}
> nodetool disablethrift
> nodetool disablegossip
> nodetool drain
> {code}
> I am not sure if this could have affected the pending commitlogs somehow.
> Any help will be appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)