You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Mark Bryant <MB...@sumo-digital.com> on 2018/10/24 12:03:17 UTC

java.lang.AssertionError: Memory was freed during index rebuild

Has anyone seen this error or might have an idea what is causing this?

Version: 3.5

java.lang.AssertionError: Memory was freed
        at org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:103)
        at org.apache.cassandra.io.util.Memory.getLong(Memory.java:260)
        at org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:235)
        at org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBufferMmap(CompressedRandomAccessReader.java:170)
        at org.apache.cassandra.io.util.RandomAccessReader.reBuffer(RandomAccessReader.java:111)
        at org.apache.cassandra.io.util.RebufferingInputStream.read(RebufferingInputStream.java:88)
        at org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:66)
        at org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:400)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithVIntLength(ByteBufferUtil.java:338)
        at org.apache.cassandra.db.marshal.AbstractType.readValue(AbstractType.java:391)
        at org.apache.cassandra.db.ClusteringPrefix$Deserializer.deserializeOne(ClusteringPrefix.java:480)
        at org.apache.cassandra.db.ClusteringPrefix$Deserializer.deserializeAll(ClusteringPrefix.java:486)
        at org.apache.cassandra.db.ClusteringPrefix$Deserializer.deserializeNextClustering(ClusteringPrefix.java:502)
        at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.readNext(UnfilteredDeserializer.java:210)
        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.computeNext(SSTableIterator.java:125)
        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:149)
        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:354)
        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:229)
        at org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32)
        at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:93)
        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:25)
        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
        at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
        at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112)
        at org.apache.cassandra.index.SecondaryIndexManager.indexPartition(SecondaryIndexManager.java:549)
        at org.apache.cassandra.db.Keyspace.indexPartition(Keyspace.java:570)
        at org.apache.cassandra.index.internal.CollatedViewIndexBuilder.build(CollatedViewIndexBuilder.java:70)
        at org.apache.cassandra.db.compaction.CompactionManager$12.run(CompactionManager.java:1472)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

I ran `nodetool rebuild_index -- keyspace table index` on 6 nodes of a 6 node cluster. After several hours of processing, it failed on 3 of the nodes with the following additional call stacks:

java.lang.AssertionError: Memory was freed
        at org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:103)
        at org.apache.cassandra.io.util.Memory.getLong(Memory.java:260)
        at org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:235)
        at org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBufferMmap(CompressedRandomAccessReader.java:170)
        at org.apache.cassandra.io.util.RandomAccessReader.reBuffer(RandomAccessReader.java:111)
        at org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:144)
        at org.apache.cassandra.utils.vint.VIntCoding.readUnsignedVInt(VIntCoding.java:63)
        at org.apache.cassandra.io.util.RebufferingInputStream.readUnsignedVInt(RebufferingInputStream.java:209)
        at org.apache.cassandra.db.Columns$Serializer.deserializeSubset(Columns.java:494)
        at org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:429)
        at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.readNext(UnfilteredDeserializer.java:211)
        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.computeNext(SSTableIterator.java:125)
        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:149)
        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:354)
        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:229)
        at org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32)
        at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:93)
        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:25)
        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
        at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
        at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112)
        at org.apache.cassandra.index.SecondaryIndexManager.indexPartition(SecondaryIndexManager.java:549)
        at org.apache.cassandra.db.Keyspace.indexPartition(Keyspace.java:570)
        at org.apache.cassandra.index.internal.CollatedViewIndexBuilder.build(CollatedViewIndexBuilder.java:70)
        at org.apache.cassandra.db.compaction.CompactionManager$12.run(CompactionManager.java:1472)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: Memory was freed
        at org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:103)
        at org.apache.cassandra.io.util.Memory.getLong(Memory.java:260)
        at org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:235)
        at org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBufferMmap(CompressedRandomAccessReader.java:170)
        at org.apache.cassandra.io.util.RandomAccessReader.reBuffer(RandomAccessReader.java:111)
        at org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:144)
        at org.apache.cassandra.utils.vint.VIntCoding.readUnsignedVInt(VIntCoding.java:73)
        at org.apache.cassandra.io.util.RebufferingInputStream.readUnsignedVInt(RebufferingInputStream.java:209)
        at org.apache.cassandra.db.SerializationHeader.readTimestamp(SerializationHeader.java:205)
        at org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:420)
        at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.readNext(UnfilteredDeserializer.java:211)
        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.computeNext(SSTableIterator.java:125)
        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:149)
        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:354)
        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:229)
        at org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32)
        at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:93)
        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:25)
        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
        at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
        at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112)
        at org.apache.cassandra.index.SecondaryIndexManager.indexPartition(SecondaryIndexManager.java:549)
        at org.apache.cassandra.db.Keyspace.indexPartition(Keyspace.java:570)
        at org.apache.cassandra.index.internal.CollatedViewIndexBuilder.build(CollatedViewIndexBuilder.java:70)
        at org.apache.cassandra.db.compaction.CompactionManager$12.run(CompactionManager.java:1472)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Is something timing out and freeing up the memory?
Mark Bryant
[Sumo-Logo]     [Tel] +44 (0)114 2426766
[Web]www.sumo-digital.com
[Address]32 Jessops Riverside, 800 Brightside Lane, Sheffield, S9 2RX, United Kingdom
[------------------------------------------------------------------------------------------------------------------------------]
[Facebook]<https://www.facebook.com/SumoDigital> [Twitter] <https://twitter.com/sumodigitalltd>  [LinkedIn] <https://www.linkedin.com/company/sumo-digital>
The contents of this email are confidential and may not be copied, disclosed to other parties or used by anyone other than the intended recipient. If you received this email in error, please contact Sumo Digital Ltd by email: info@sumo-digital.com or by calling +44 (0)114 2426766. Quote the name of the sender, your email address and then delete this email from your system.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org


Re: java.lang.AssertionError: Memory was freed during index rebuild

Posted by Jeff Jirsa <jj...@gmail.com>.
3.5 is probably not a version you should be using in production in 2018 - it was a feature release and has had no bug fixes for years. Going up to 3.11.3 will likely fix many serious bugs you’re not noticing, and maybe the bug below you are noticing 


-- 
Jeff Jirsa


> On Oct 24, 2018, at 5:03 AM, Mark Bryant <MB...@sumo-digital.com> wrote:
> 
> Has anyone seen this error or might have an idea what is causing this?
> 
> Version: 3.5
> 
> java.lang.AssertionError: Memory was freed
>        at org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:103)
>        at org.apache.cassandra.io.util.Memory.getLong(Memory.java:260)
>        at org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:235)
>        at org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBufferMmap(CompressedRandomAccessReader.java:170)
>        at org.apache.cassandra.io.util.RandomAccessReader.reBuffer(RandomAccessReader.java:111)
>        at org.apache.cassandra.io.util.RebufferingInputStream.read(RebufferingInputStream.java:88)
>        at org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:66)
>        at org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:60)
>        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:400)
>        at org.apache.cassandra.utils.ByteBufferUtil.readWithVIntLength(ByteBufferUtil.java:338)
>        at org.apache.cassandra.db.marshal.AbstractType.readValue(AbstractType.java:391)
>        at org.apache.cassandra.db.ClusteringPrefix$Deserializer.deserializeOne(ClusteringPrefix.java:480)
>        at org.apache.cassandra.db.ClusteringPrefix$Deserializer.deserializeAll(ClusteringPrefix.java:486)
>        at org.apache.cassandra.db.ClusteringPrefix$Deserializer.deserializeNextClustering(ClusteringPrefix.java:502)
>        at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.readNext(UnfilteredDeserializer.java:210)
>        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.computeNext(SSTableIterator.java:125)
>        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:149)
>        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:354)
>        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:229)
>        at org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32)
>        at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
>        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:93)
>        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:25)
>        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>        at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
>        at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
>        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
>        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>        at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112)
>        at org.apache.cassandra.index.SecondaryIndexManager.indexPartition(SecondaryIndexManager.java:549)
>        at org.apache.cassandra.db.Keyspace.indexPartition(Keyspace.java:570)
>        at org.apache.cassandra.index.internal.CollatedViewIndexBuilder.build(CollatedViewIndexBuilder.java:70)
>        at org.apache.cassandra.db.compaction.CompactionManager$12.run(CompactionManager.java:1472)
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>        at java.lang.Thread.run(Thread.java:748)
> 
> I ran `nodetool rebuild_index -- keyspace table index` on 6 nodes of a 6 node cluster. After several hours of processing, it failed on 3 of the nodes with the following additional call stacks:
> 
> java.lang.AssertionError: Memory was freed
>        at org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:103)
>        at org.apache.cassandra.io.util.Memory.getLong(Memory.java:260)
>        at org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:235)
>        at org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBufferMmap(CompressedRandomAccessReader.java:170)
>        at org.apache.cassandra.io.util.RandomAccessReader.reBuffer(RandomAccessReader.java:111)
>        at org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:144)
>        at org.apache.cassandra.utils.vint.VIntCoding.readUnsignedVInt(VIntCoding.java:63)
>        at org.apache.cassandra.io.util.RebufferingInputStream.readUnsignedVInt(RebufferingInputStream.java:209)
>        at org.apache.cassandra.db.Columns$Serializer.deserializeSubset(Columns.java:494)
>        at org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:429)
>        at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.readNext(UnfilteredDeserializer.java:211)
>        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.computeNext(SSTableIterator.java:125)
>        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:149)
>        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:354)
>        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:229)
>        at org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32)
>        at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
>        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:93)
>        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:25)
>        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>        at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
>        at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
>        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
>        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>        at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112)
>        at org.apache.cassandra.index.SecondaryIndexManager.indexPartition(SecondaryIndexManager.java:549)
>        at org.apache.cassandra.db.Keyspace.indexPartition(Keyspace.java:570)
>        at org.apache.cassandra.index.internal.CollatedViewIndexBuilder.build(CollatedViewIndexBuilder.java:70)
>        at org.apache.cassandra.db.compaction.CompactionManager$12.run(CompactionManager.java:1472)
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>        at java.lang.Thread.run(Thread.java:748)
> 
> java.lang.AssertionError: Memory was freed
>        at org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:103)
>        at org.apache.cassandra.io.util.Memory.getLong(Memory.java:260)
>        at org.apache.cassandra.io.compress.CompressionMetadata.chunkFor(CompressionMetadata.java:235)
>        at org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBufferMmap(CompressedRandomAccessReader.java:170)
>        at org.apache.cassandra.io.util.RandomAccessReader.reBuffer(RandomAccessReader.java:111)
>        at org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:144)
>        at org.apache.cassandra.utils.vint.VIntCoding.readUnsignedVInt(VIntCoding.java:73)
>        at org.apache.cassandra.io.util.RebufferingInputStream.readUnsignedVInt(RebufferingInputStream.java:209)
>        at org.apache.cassandra.db.SerializationHeader.readTimestamp(SerializationHeader.java:205)
>        at org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:420)
>        at org.apache.cassandra.db.UnfilteredDeserializer$CurrentDeserializer.readNext(UnfilteredDeserializer.java:211)
>        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.computeNext(SSTableIterator.java:125)
>        at org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:149)
>        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:354)
>        at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:229)
>        at org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:32)
>        at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
>        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:93)
>        at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:25)
>        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>        at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
>        at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
>        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
>        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:419)
>        at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:279)
>        at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
>        at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:112)
>        at org.apache.cassandra.index.SecondaryIndexManager.indexPartition(SecondaryIndexManager.java:549)
>        at org.apache.cassandra.db.Keyspace.indexPartition(Keyspace.java:570)
>        at org.apache.cassandra.index.internal.CollatedViewIndexBuilder.build(CollatedViewIndexBuilder.java:70)
>        at org.apache.cassandra.db.compaction.CompactionManager$12.run(CompactionManager.java:1472)
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>        at java.lang.Thread.run(Thread.java:748)
> 
> Is something timing out and freeing up the memory?
> Mark Bryant
> [Sumo-Logo]     [Tel] +44 (0)114 2426766
> [Web]www.sumo-digital.com
> [Address]32 Jessops Riverside, 800 Brightside Lane, Sheffield, S9 2RX, United Kingdom
> [------------------------------------------------------------------------------------------------------------------------------]
> [Facebook]<https://www.facebook.com/SumoDigital> [Twitter] <https://twitter.com/sumodigitalltd>  [LinkedIn] <https://www.linkedin.com/company/sumo-digital>
> The contents of this email are confidential and may not be copied, disclosed to other parties or used by anyone other than the intended recipient. If you received this email in error, please contact Sumo Digital Ltd by email: info@sumo-digital.com or by calling +44 (0)114 2426766. Quote the name of the sender, your email address and then delete this email from your system.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org