You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Ryan Daum (JIRA)" <ji...@apache.org> on 2010/01/13 20:12:54 UTC
[jira] Created: (CASSANDRA-694) Failure to flush commit log
Failure to flush commit log
---------------------------
Key: CASSANDRA-694
URL: https://issues.apache.org/jira/browse/CASSANDRA-694
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.5
Environment: Linux 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 GNU/Linux, ec2 small instance
Reporter: Ryan Daum
The following exception occurs consistently on at least node (note did not occur on other same-configured nodes) during startup:
INFO - Replaying /var/lib/cassandra/commitlog/CommitLog-1262855754427.log, /var/lib/cassandra/commitlog/CommitLog-1262832689989.log, /var/lib/cassandra/commitlog/CommitLog-1262885833186.log, /var/lib/cassandra/commitlog/CommitLog-1262900845019.log, /var/lib/cassandra/commitlog/CommitLog-1262913267844.log, /var/lib/cassandra/commitlog/CommitLog-1262927898170.log, /var/lib/cassandra/commitlog/CommitLog-1262961421039.log, /var/lib/cassandra/commitlog/CommitLog-1262977175175.log, /var/lib/cassandra/commitlog/CommitLog-1262989588783.log, /var/lib/cassandra/commitlog/CommitLog-1263000573676.log, /var/lib/cassandra/commitlog/CommitLog-1263013691393.log, /var/lib/cassandra/commitlog/CommitLog-1263044706108.log, /var/lib/cassandra/commitlog/CommitLog-1263060004191.log, /var/lib/cassandra/commitlog/CommitLog-1263071446342.log, /var/lib/cassandra/commitlog/CommitLog-1263082950154.log, /var/lib/cassandra/commitlog/CommitLog-1263095400814.log, /var/lib/cassandra/commitlog/CommitLog-1263118331046.log, /var/lib/cassandra/commitlog/CommitLog-1263143402963.log, /var/lib/cassandra/commitlog/CommitLog-1263155294308.log, /var/lib/cassandra/commitlog/CommitLog-1263166154352.log, /var/lib/cassandra/commitlog/CommitLog-1263178359247.log, /var/lib/cassandra/commitlog/CommitLog-1263202112017.log, /var/lib/cassandra/commitlog/CommitLog-1263230932274.log, /var/lib/cassandra/commitlog/CommitLog-1263250726505.log, /var/lib/cassandra/commitlog/CommitLog-1263264159438.log, /var/lib/cassandra/commitlog/CommitLog-1263289964249.log, /var/lib/cassandra/commitlog/CommitLog-1263317974387.log, /var/lib/cassandra/commitlog/CommitLog-1263331989090.log, /var/lib/cassandra/commitlog/CommitLog-1263344147667.log, /var/lib/cassandra/commitlog/CommitLog-1263359751527.log, /var/lib/cassandra/commitlog/CommitLog-1263395707008.log, /var/lib/cassandra/commitlog/CommitLog-1263397833524.log, /var/lib/cassandra/commitlog/CommitLog-1263398736183.log, /var/lib/cassandra/commitlog/CommitLog-1263399753707.log, /var/lib/cassandra/commitlog/CommitLog-1263401667504.log, /var/lib/cassandra/commitlog/CommitLog-1263404640782.log, /var/lib/cassandra/commitlog/CommitLog-1263405827234.log, /var/lib/cassandra/commitlog/CommitLog-1263406901115.log
INFO - LocationInfo has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(LocationInfo)@25934689
INFO - HintsColumnFamily has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(HintsColumnFamily)@4766820
INFO - AdXRequestStatistics has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(AdXRequestStatistics)@21521158
INFO - TokenGoogleIDCF has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(TokenGoogleIDCF)@22889075
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)
Caused by: java.lang.AssertionError: Blocking serialized executor is not yet implemented
at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
at org.apache.cassandra.db.Table.flush(Table.java:464)
at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:397)
at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:65)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:90)
at org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:135)
... 5 more
And the same exception occurs intermittently on other node (running) nodes during 'nodeprobe flush':
root@domU-12-31-38-00-26-31:~# nodeprobe -host localhost -port 8080 flush Logger
Exception in thread "main" java.lang.AssertionError: Blocking serialized executor is not yet implemented
at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
at org.apache.cassandra.service.StorageService.forceTableFlush(StorageService.java:984)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-694) Failure to flush commit log
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-694:
-------------------------------------
Attachment: 694-trunk.txt
version for trunk w/ more comments & a unit test. will backport when i commit to 0.5.
> Failure to flush commit log
> ---------------------------
>
> Key: CASSANDRA-694
> URL: https://issues.apache.org/jira/browse/CASSANDRA-694
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Linux 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 GNU/Linux, ec2 small instance
> Reporter: Ryan Daum
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 694-0.5.txt, 694-trunk.txt
>
>
> The following exception occurs consistently on at least node (note did not occur on other same-configured nodes) during startup:
> INFO - Replaying /var/lib/cassandra/commitlog/CommitLog-1262855754427.log, /var/lib/cassandra/commitlog/CommitLog-1262832689989.log, /var/lib/cassandra/commitlog/CommitLog-1262885833186.log, /var/lib/cassandra/commitlog/CommitLog-1262900845019.log, /var/lib/cassandra/commitlog/CommitLog-1262913267844.log, /var/lib/cassandra/commitlog/CommitLog-1262927898170.log, /var/lib/cassandra/commitlog/CommitLog-1262961421039.log, /var/lib/cassandra/commitlog/CommitLog-1262977175175.log, /var/lib/cassandra/commitlog/CommitLog-1262989588783.log, /var/lib/cassandra/commitlog/CommitLog-1263000573676.log, /var/lib/cassandra/commitlog/CommitLog-1263013691393.log, /var/lib/cassandra/commitlog/CommitLog-1263044706108.log, /var/lib/cassandra/commitlog/CommitLog-1263060004191.log, /var/lib/cassandra/commitlog/CommitLog-1263071446342.log, /var/lib/cassandra/commitlog/CommitLog-1263082950154.log, /var/lib/cassandra/commitlog/CommitLog-1263095400814.log, /var/lib/cassandra/commitlog/CommitLog-1263118331046.log, /var/lib/cassandra/commitlog/CommitLog-1263143402963.log, /var/lib/cassandra/commitlog/CommitLog-1263155294308.log, /var/lib/cassandra/commitlog/CommitLog-1263166154352.log, /var/lib/cassandra/commitlog/CommitLog-1263178359247.log, /var/lib/cassandra/commitlog/CommitLog-1263202112017.log, /var/lib/cassandra/commitlog/CommitLog-1263230932274.log, /var/lib/cassandra/commitlog/CommitLog-1263250726505.log, /var/lib/cassandra/commitlog/CommitLog-1263264159438.log, /var/lib/cassandra/commitlog/CommitLog-1263289964249.log, /var/lib/cassandra/commitlog/CommitLog-1263317974387.log, /var/lib/cassandra/commitlog/CommitLog-1263331989090.log, /var/lib/cassandra/commitlog/CommitLog-1263344147667.log, /var/lib/cassandra/commitlog/CommitLog-1263359751527.log, /var/lib/cassandra/commitlog/CommitLog-1263395707008.log, /var/lib/cassandra/commitlog/CommitLog-1263397833524.log, /var/lib/cassandra/commitlog/CommitLog-1263398736183.log, /var/lib/cassandra/commitlog/CommitLog-1263399753707.log, /var/lib/cassandra/commitlog/CommitLog-1263401667504.log, /var/lib/cassandra/commitlog/CommitLog-1263404640782.log, /var/lib/cassandra/commitlog/CommitLog-1263405827234.log, /var/lib/cassandra/commitlog/CommitLog-1263406901115.log
> INFO - LocationInfo has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(LocationInfo)@25934689
> INFO - HintsColumnFamily has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(HintsColumnFamily)@4766820
> INFO - AdXRequestStatistics has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(AdXRequestStatistics)@21521158
> INFO - TokenGoogleIDCF has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(TokenGoogleIDCF)@22889075
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)
> Caused by: java.lang.AssertionError: Blocking serialized executor is not yet implemented
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
> at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
> at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
> at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
> at org.apache.cassandra.db.Table.flush(Table.java:464)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:397)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:65)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:90)
> at org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:135)
> ... 5 more
> And the same exception occurs intermittently on other node (running) nodes during 'nodeprobe flush':
> root@domU-12-31-38-00-26-31:~# nodeprobe -host localhost -port 8080 flush Logger
> Exception in thread "main" java.lang.AssertionError: Blocking serialized executor is not yet implemented
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
> at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
> at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
> at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
> at org.apache.cassandra.service.StorageService.forceTableFlush(StorageService.java:984)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
> at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
> at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
> at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359)
> at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
> at sun.rmi.transport.Transport$1.run(Transport.java:159)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-694) Failure to flush commit log
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799938#action_12799938 ]
Jonathan Ellis commented on CASSANDRA-694:
------------------------------------------
from irc:
rdaum> that patch works
> Failure to flush commit log
> ---------------------------
>
> Key: CASSANDRA-694
> URL: https://issues.apache.org/jira/browse/CASSANDRA-694
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Linux 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 GNU/Linux, ec2 small instance
> Reporter: Ryan Daum
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 694-0.5.txt, 694-trunk.txt
>
>
> The following exception occurs consistently on at least node (note did not occur on other same-configured nodes) during startup:
> INFO - Replaying /var/lib/cassandra/commitlog/CommitLog-1262855754427.log, /var/lib/cassandra/commitlog/CommitLog-1262832689989.log, /var/lib/cassandra/commitlog/CommitLog-1262885833186.log, /var/lib/cassandra/commitlog/CommitLog-1262900845019.log, /var/lib/cassandra/commitlog/CommitLog-1262913267844.log, /var/lib/cassandra/commitlog/CommitLog-1262927898170.log, /var/lib/cassandra/commitlog/CommitLog-1262961421039.log, /var/lib/cassandra/commitlog/CommitLog-1262977175175.log, /var/lib/cassandra/commitlog/CommitLog-1262989588783.log, /var/lib/cassandra/commitlog/CommitLog-1263000573676.log, /var/lib/cassandra/commitlog/CommitLog-1263013691393.log, /var/lib/cassandra/commitlog/CommitLog-1263044706108.log, /var/lib/cassandra/commitlog/CommitLog-1263060004191.log, /var/lib/cassandra/commitlog/CommitLog-1263071446342.log, /var/lib/cassandra/commitlog/CommitLog-1263082950154.log, /var/lib/cassandra/commitlog/CommitLog-1263095400814.log, /var/lib/cassandra/commitlog/CommitLog-1263118331046.log, /var/lib/cassandra/commitlog/CommitLog-1263143402963.log, /var/lib/cassandra/commitlog/CommitLog-1263155294308.log, /var/lib/cassandra/commitlog/CommitLog-1263166154352.log, /var/lib/cassandra/commitlog/CommitLog-1263178359247.log, /var/lib/cassandra/commitlog/CommitLog-1263202112017.log, /var/lib/cassandra/commitlog/CommitLog-1263230932274.log, /var/lib/cassandra/commitlog/CommitLog-1263250726505.log, /var/lib/cassandra/commitlog/CommitLog-1263264159438.log, /var/lib/cassandra/commitlog/CommitLog-1263289964249.log, /var/lib/cassandra/commitlog/CommitLog-1263317974387.log, /var/lib/cassandra/commitlog/CommitLog-1263331989090.log, /var/lib/cassandra/commitlog/CommitLog-1263344147667.log, /var/lib/cassandra/commitlog/CommitLog-1263359751527.log, /var/lib/cassandra/commitlog/CommitLog-1263395707008.log, /var/lib/cassandra/commitlog/CommitLog-1263397833524.log, /var/lib/cassandra/commitlog/CommitLog-1263398736183.log, /var/lib/cassandra/commitlog/CommitLog-1263399753707.log, /var/lib/cassandra/commitlog/CommitLog-1263401667504.log, /var/lib/cassandra/commitlog/CommitLog-1263404640782.log, /var/lib/cassandra/commitlog/CommitLog-1263405827234.log, /var/lib/cassandra/commitlog/CommitLog-1263406901115.log
> INFO - LocationInfo has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(LocationInfo)@25934689
> INFO - HintsColumnFamily has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(HintsColumnFamily)@4766820
> INFO - AdXRequestStatistics has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(AdXRequestStatistics)@21521158
> INFO - TokenGoogleIDCF has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(TokenGoogleIDCF)@22889075
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)
> Caused by: java.lang.AssertionError: Blocking serialized executor is not yet implemented
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
> at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
> at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
> at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
> at org.apache.cassandra.db.Table.flush(Table.java:464)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:397)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:65)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:90)
> at org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:135)
> ... 5 more
> And the same exception occurs intermittently on other node (running) nodes during 'nodeprobe flush':
> root@domU-12-31-38-00-26-31:~# nodeprobe -host localhost -port 8080 flush Logger
> Exception in thread "main" java.lang.AssertionError: Blocking serialized executor is not yet implemented
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
> at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
> at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
> at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
> at org.apache.cassandra.service.StorageService.forceTableFlush(StorageService.java:984)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
> at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
> at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
> at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359)
> at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
> at sun.rmi.transport.Transport$1.run(Transport.java:159)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-694) Failure to flush commit log
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-694:
-------------------------------------
Attachment: 694-0.5.txt
assumption that all single-threaded executors have an unbounded queue is no longer valid. this patch provides a policy for dealing with single thread executors w/ a full queue.
> Failure to flush commit log
> ---------------------------
>
> Key: CASSANDRA-694
> URL: https://issues.apache.org/jira/browse/CASSANDRA-694
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Linux 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 GNU/Linux, ec2 small instance
> Reporter: Ryan Daum
> Attachments: 694-0.5.txt
>
>
> The following exception occurs consistently on at least node (note did not occur on other same-configured nodes) during startup:
> INFO - Replaying /var/lib/cassandra/commitlog/CommitLog-1262855754427.log, /var/lib/cassandra/commitlog/CommitLog-1262832689989.log, /var/lib/cassandra/commitlog/CommitLog-1262885833186.log, /var/lib/cassandra/commitlog/CommitLog-1262900845019.log, /var/lib/cassandra/commitlog/CommitLog-1262913267844.log, /var/lib/cassandra/commitlog/CommitLog-1262927898170.log, /var/lib/cassandra/commitlog/CommitLog-1262961421039.log, /var/lib/cassandra/commitlog/CommitLog-1262977175175.log, /var/lib/cassandra/commitlog/CommitLog-1262989588783.log, /var/lib/cassandra/commitlog/CommitLog-1263000573676.log, /var/lib/cassandra/commitlog/CommitLog-1263013691393.log, /var/lib/cassandra/commitlog/CommitLog-1263044706108.log, /var/lib/cassandra/commitlog/CommitLog-1263060004191.log, /var/lib/cassandra/commitlog/CommitLog-1263071446342.log, /var/lib/cassandra/commitlog/CommitLog-1263082950154.log, /var/lib/cassandra/commitlog/CommitLog-1263095400814.log, /var/lib/cassandra/commitlog/CommitLog-1263118331046.log, /var/lib/cassandra/commitlog/CommitLog-1263143402963.log, /var/lib/cassandra/commitlog/CommitLog-1263155294308.log, /var/lib/cassandra/commitlog/CommitLog-1263166154352.log, /var/lib/cassandra/commitlog/CommitLog-1263178359247.log, /var/lib/cassandra/commitlog/CommitLog-1263202112017.log, /var/lib/cassandra/commitlog/CommitLog-1263230932274.log, /var/lib/cassandra/commitlog/CommitLog-1263250726505.log, /var/lib/cassandra/commitlog/CommitLog-1263264159438.log, /var/lib/cassandra/commitlog/CommitLog-1263289964249.log, /var/lib/cassandra/commitlog/CommitLog-1263317974387.log, /var/lib/cassandra/commitlog/CommitLog-1263331989090.log, /var/lib/cassandra/commitlog/CommitLog-1263344147667.log, /var/lib/cassandra/commitlog/CommitLog-1263359751527.log, /var/lib/cassandra/commitlog/CommitLog-1263395707008.log, /var/lib/cassandra/commitlog/CommitLog-1263397833524.log, /var/lib/cassandra/commitlog/CommitLog-1263398736183.log, /var/lib/cassandra/commitlog/CommitLog-1263399753707.log, /var/lib/cassandra/commitlog/CommitLog-1263401667504.log, /var/lib/cassandra/commitlog/CommitLog-1263404640782.log, /var/lib/cassandra/commitlog/CommitLog-1263405827234.log, /var/lib/cassandra/commitlog/CommitLog-1263406901115.log
> INFO - LocationInfo has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(LocationInfo)@25934689
> INFO - HintsColumnFamily has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(HintsColumnFamily)@4766820
> INFO - AdXRequestStatistics has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(AdXRequestStatistics)@21521158
> INFO - TokenGoogleIDCF has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(TokenGoogleIDCF)@22889075
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)
> Caused by: java.lang.AssertionError: Blocking serialized executor is not yet implemented
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
> at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
> at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
> at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
> at org.apache.cassandra.db.Table.flush(Table.java:464)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:397)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:65)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:90)
> at org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:135)
> ... 5 more
> And the same exception occurs intermittently on other node (running) nodes during 'nodeprobe flush':
> root@domU-12-31-38-00-26-31:~# nodeprobe -host localhost -port 8080 flush Logger
> Exception in thread "main" java.lang.AssertionError: Blocking serialized executor is not yet implemented
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
> at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
> at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
> at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
> at org.apache.cassandra.service.StorageService.forceTableFlush(StorageService.java:984)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
> at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
> at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
> at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359)
> at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
> at sun.rmi.transport.Transport$1.run(Transport.java:159)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-694) Failure to flush commit log
Posted by "Ryan Daum (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799886#action_12799886 ]
Ryan Daum commented on CASSANDRA-694:
-------------------------------------
Note that the error occurs with build off trunk (svn rev 898899) as well.
> Failure to flush commit log
> ---------------------------
>
> Key: CASSANDRA-694
> URL: https://issues.apache.org/jira/browse/CASSANDRA-694
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: Linux 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 GNU/Linux, ec2 small instance
> Reporter: Ryan Daum
>
> The following exception occurs consistently on at least node (note did not occur on other same-configured nodes) during startup:
> INFO - Replaying /var/lib/cassandra/commitlog/CommitLog-1262855754427.log, /var/lib/cassandra/commitlog/CommitLog-1262832689989.log, /var/lib/cassandra/commitlog/CommitLog-1262885833186.log, /var/lib/cassandra/commitlog/CommitLog-1262900845019.log, /var/lib/cassandra/commitlog/CommitLog-1262913267844.log, /var/lib/cassandra/commitlog/CommitLog-1262927898170.log, /var/lib/cassandra/commitlog/CommitLog-1262961421039.log, /var/lib/cassandra/commitlog/CommitLog-1262977175175.log, /var/lib/cassandra/commitlog/CommitLog-1262989588783.log, /var/lib/cassandra/commitlog/CommitLog-1263000573676.log, /var/lib/cassandra/commitlog/CommitLog-1263013691393.log, /var/lib/cassandra/commitlog/CommitLog-1263044706108.log, /var/lib/cassandra/commitlog/CommitLog-1263060004191.log, /var/lib/cassandra/commitlog/CommitLog-1263071446342.log, /var/lib/cassandra/commitlog/CommitLog-1263082950154.log, /var/lib/cassandra/commitlog/CommitLog-1263095400814.log, /var/lib/cassandra/commitlog/CommitLog-1263118331046.log, /var/lib/cassandra/commitlog/CommitLog-1263143402963.log, /var/lib/cassandra/commitlog/CommitLog-1263155294308.log, /var/lib/cassandra/commitlog/CommitLog-1263166154352.log, /var/lib/cassandra/commitlog/CommitLog-1263178359247.log, /var/lib/cassandra/commitlog/CommitLog-1263202112017.log, /var/lib/cassandra/commitlog/CommitLog-1263230932274.log, /var/lib/cassandra/commitlog/CommitLog-1263250726505.log, /var/lib/cassandra/commitlog/CommitLog-1263264159438.log, /var/lib/cassandra/commitlog/CommitLog-1263289964249.log, /var/lib/cassandra/commitlog/CommitLog-1263317974387.log, /var/lib/cassandra/commitlog/CommitLog-1263331989090.log, /var/lib/cassandra/commitlog/CommitLog-1263344147667.log, /var/lib/cassandra/commitlog/CommitLog-1263359751527.log, /var/lib/cassandra/commitlog/CommitLog-1263395707008.log, /var/lib/cassandra/commitlog/CommitLog-1263397833524.log, /var/lib/cassandra/commitlog/CommitLog-1263398736183.log, /var/lib/cassandra/commitlog/CommitLog-1263399753707.log, /var/lib/cassandra/commitlog/CommitLog-1263401667504.log, /var/lib/cassandra/commitlog/CommitLog-1263404640782.log, /var/lib/cassandra/commitlog/CommitLog-1263405827234.log, /var/lib/cassandra/commitlog/CommitLog-1263406901115.log
> INFO - LocationInfo has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(LocationInfo)@25934689
> INFO - HintsColumnFamily has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(HintsColumnFamily)@4766820
> INFO - AdXRequestStatistics has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(AdXRequestStatistics)@21521158
> INFO - TokenGoogleIDCF has reached its threshold; switching in a fresh Memtable
> INFO - Enqueuing flush of Memtable(TokenGoogleIDCF)@22889075
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)
> Caused by: java.lang.AssertionError: Blocking serialized executor is not yet implemented
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
> at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
> at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
> at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
> at org.apache.cassandra.db.Table.flush(Table.java:464)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:397)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:65)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:90)
> at org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:135)
> ... 5 more
> And the same exception occurs intermittently on other node (running) nodes during 'nodeprobe flush':
> root@domU-12-31-38-00-26-31:~# nodeprobe -host localhost -port 8080 flush Logger
> Exception in thread "main" java.lang.AssertionError: Blocking serialized executor is not yet implemented
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
> at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
> at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
> at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
> at org.apache.cassandra.service.StorageService.forceTableFlush(StorageService.java:984)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
> at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
> at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
> at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359)
> at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
> at sun.rmi.transport.Transport$1.run(Transport.java:159)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.