You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Imri Zvik (JIRA)" <ji...@apache.org> on 2015/02/01 19:58:35 UTC
[jira] [Commented] (CASSANDRA-8210) "java.lang.AssertionError:
Memory was freed" exception in CompactionExecutor
[ https://issues.apache.org/jira/browse/CASSANDRA-8210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14300409#comment-14300409 ]
Imri Zvik commented on CASSANDRA-8210:
--------------------------------------
I am getting a similar error when I run cleanup (using 2.0.12):
[root@cassandra005 ~]# nodetool cleanup
Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.AssertionError: Memory was freed
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:234)
at org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:272)
at org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1115)
at org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2177)
at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: Memory was freed
at org.apache.cassandra.io.util.Memory.checkPosition(Memory.java:259)
at org.apache.cassandra.io.util.Memory.getInt(Memory.java:211)
at org.apache.cassandra.io.sstable.IndexSummary.getIndex(IndexSummary.java:79)
at org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:84)
at org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:58)
at org.apache.cassandra.io.sstable.SSTableReader.getIndexScanPosition(SSTableReader.java:602)
at org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:947)
at org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:910)
at org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:819)
at org.apache.cassandra.db.ColumnFamilyStore.getExpectedCompactedFileSize(ColumnFamilyStore.java:1088)
at org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompaction(CompactionManager.java:564)
at org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:63)
at org.apache.cassandra.db.compaction.CompactionManager$5.perform(CompactionManager.java:281)
at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:225)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
... 3 more
> "java.lang.AssertionError: Memory was freed" exception in CompactionExecutor
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-8210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8210
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: DSE 4.5.2, Cassandra 2.0.10, OEL 6.5, kernel 3.8.13-44.el6uek.x86_64, 128Gb of RAM, swap disabled, JRE 1.7.0_67-b01
> Reporter: Nikolai Grigoriev
> Priority: Minor
> Attachments: cassandra-env.sh, cassandra.yaml, occurence frequency.png, system.log.gz
>
>
> I have just got this problem on multiple nodes. Cassandra 2.0.10 (DSE 4.5.2). After looking through the history I have found that it was actually happening on all nodes since the start of large compaction process (I've loaded tons of data in the system and then turned off all load to let it compact the data).
> {code}
> ERROR [CompactionExecutor:1196] 2014-10-28 17:14:50,124 CassandraDaemon.java (line 199) Exception in thread Thread[CompactionExecutor:1196,1,main]
> java.lang.AssertionError: Memory was freed
> at org.apache.cassandra.io.util.Memory.checkPosition(Memory.java:259)
> at org.apache.cassandra.io.util.Memory.getInt(Memory.java:211)
> at org.apache.cassandra.io.sstable.IndexSummary.getIndex(IndexSummary.java:79)
> at org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:84)
> at org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:58)
> at org.apache.cassandra.io.sstable.SSTableReader.getSampleIndexesForRanges(SSTableReader.java:692)
> at org.apache.cassandra.io.sstable.SSTableReader.estimatedKeysForRanges(SSTableReader.java:663)
> at org.apache.cassandra.db.compaction.AbstractCompactionStrategy.worthDroppingTombstones(AbstractCompactionStrategy.java:328)
> at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.findDroppableSSTable(LeveledCompactionStrategy.java:354)
> at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getMaximalTask(LeveledCompactionStrategy.java:125)
> at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:113)
> at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:192)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)