You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jackson Chung (JIRA)" <ji...@apache.org> on 2012/05/17 20:30:08 UTC
[jira] [Created] (CASSANDRA-4255) concurrent modif ex when repair
is run on LCS
Jackson Chung created CASSANDRA-4255:
----------------------------------------
Summary: concurrent modif ex when repair is run on LCS
Key: CASSANDRA-4255
URL: https://issues.apache.org/jira/browse/CASSANDRA-4255
Project: Cassandra
Issue Type: Bug
Reporter: Jackson Chung
came across this, will try to figure a way to systematically reprod this. But the problem is the sstable list in the manifest is changing as the repair is triggered:
{panel}
Exception in thread "main" java.util.ConcurrentModificationException
at java.util.AbstractList$Itr.checkForComodification(Unknown Source)
at java.util.AbstractList$Itr.next(Unknown Source)
at org.apache.cassandra.io.sstable.SSTable.getTotalBytes(SSTable.java:250)
at org.apache.cassandra.db.compaction.LeveledManifest.getEstimatedTasks(LeveledManifest.java:435)
at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getEstimatedRemainingTasks(LeveledCompactionStrategy.java:128)
at org.apache.cassandra.db.compaction.CompactionManager.getPendingTasks(CompactionManager.java:1063)
at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
at com.sun.jmx.mbeanserver.PerInterface.getAttribute(Unknown Source)
at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(Unknown Source)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown Source)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.access$200(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown Source)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
at sun.rmi.transport.Transport$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{panel}
maybe we could change the list to a copyOnArrayList? just a suggestion, haven't investigated much yet:
{code:title=LeveledManifest.java}
generations[i] = new ArrayList<SSTableReader>();
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4255) concurrent modif ex when repair
is run on LCS
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280066#comment-13280066 ]
Sylvain Lebresne commented on CASSANDRA-4255:
---------------------------------------------
+1
> concurrent modif ex when repair is run on LCS
> ---------------------------------------------
>
> Key: CASSANDRA-4255
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4255
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Jackson Chung
> Assignee: Jonathan Ellis
> Priority: Minor
> Labels: compaction
> Fix For: 1.0.11, 1.1.1
>
> Attachments: 4255.txt
>
>
> came across this, will try to figure a way to systematically reprod this. But the problem is the sstable list in the manifest is changing as the repair is triggered:
> {panel}
> Exception in thread "main" java.util.ConcurrentModificationException
> at java.util.AbstractList$Itr.checkForComodification(Unknown Source)
> at java.util.AbstractList$Itr.next(Unknown Source)
> at org.apache.cassandra.io.sstable.SSTable.getTotalBytes(SSTable.java:250)
> at org.apache.cassandra.db.compaction.LeveledManifest.getEstimatedTasks(LeveledManifest.java:435)
> at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getEstimatedRemainingTasks(LeveledCompactionStrategy.java:128)
> at org.apache.cassandra.db.compaction.CompactionManager.getPendingTasks(CompactionManager.java:1063)
> at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
> at com.sun.jmx.mbeanserver.PerInterface.getAttribute(Unknown Source)
> at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(Unknown Source)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown Source)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.access$200(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
> at sun.rmi.transport.Transport$1.run(Unknown Source)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {panel}
> maybe we could change the list to a copyOnArrayList? just a suggestion, haven't investigated much yet:
> {code:title=LeveledManifest.java}
> generations[i] = new ArrayList<SSTableReader>();
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4255) concurrent modif ex when repair
is run on LCS
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4255:
--------------------------------------
Priority: Minor (was: Major)
Looks like this only affects getEstimatedTasks, not actually repair.
> concurrent modif ex when repair is run on LCS
> ---------------------------------------------
>
> Key: CASSANDRA-4255
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4255
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Jackson Chung
> Assignee: Jonathan Ellis
> Priority: Minor
> Labels: compaction
> Fix For: 1.0.11, 1.1.1
>
> Attachments: 4255.txt
>
>
> came across this, will try to figure a way to systematically reprod this. But the problem is the sstable list in the manifest is changing as the repair is triggered:
> {panel}
> Exception in thread "main" java.util.ConcurrentModificationException
> at java.util.AbstractList$Itr.checkForComodification(Unknown Source)
> at java.util.AbstractList$Itr.next(Unknown Source)
> at org.apache.cassandra.io.sstable.SSTable.getTotalBytes(SSTable.java:250)
> at org.apache.cassandra.db.compaction.LeveledManifest.getEstimatedTasks(LeveledManifest.java:435)
> at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getEstimatedRemainingTasks(LeveledCompactionStrategy.java:128)
> at org.apache.cassandra.db.compaction.CompactionManager.getPendingTasks(CompactionManager.java:1063)
> at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
> at com.sun.jmx.mbeanserver.PerInterface.getAttribute(Unknown Source)
> at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(Unknown Source)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown Source)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.access$200(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
> at sun.rmi.transport.Transport$1.run(Unknown Source)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {panel}
> maybe we could change the list to a copyOnArrayList? just a suggestion, haven't investigated much yet:
> {code:title=LeveledManifest.java}
> generations[i] = new ArrayList<SSTableReader>();
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4255) concurrent modif ex when repair
is run on LCS
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4255:
--------------------------------------
Attachment: 4255.txt
patch attached to synchronize getEstimatedTasks. copyOnWriteArrayList would probably work too, but we already synchronize other accesses (to prevent cross-level races) so that's simplest here.
> concurrent modif ex when repair is run on LCS
> ---------------------------------------------
>
> Key: CASSANDRA-4255
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4255
> Project: Cassandra
> Issue Type: Bug
> Reporter: Jackson Chung
> Attachments: 4255.txt
>
>
> came across this, will try to figure a way to systematically reprod this. But the problem is the sstable list in the manifest is changing as the repair is triggered:
> {panel}
> Exception in thread "main" java.util.ConcurrentModificationException
> at java.util.AbstractList$Itr.checkForComodification(Unknown Source)
> at java.util.AbstractList$Itr.next(Unknown Source)
> at org.apache.cassandra.io.sstable.SSTable.getTotalBytes(SSTable.java:250)
> at org.apache.cassandra.db.compaction.LeveledManifest.getEstimatedTasks(LeveledManifest.java:435)
> at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getEstimatedRemainingTasks(LeveledCompactionStrategy.java:128)
> at org.apache.cassandra.db.compaction.CompactionManager.getPendingTasks(CompactionManager.java:1063)
> at sun.reflect.GeneratedMethodAccessor73.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
> at com.sun.jmx.mbeanserver.PerInterface.getAttribute(Unknown Source)
> at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(Unknown Source)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown Source)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.access$200(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
> at sun.rmi.transport.Transport$1.run(Unknown Source)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {panel}
> maybe we could change the list to a copyOnArrayList? just a suggestion, haven't investigated much yet:
> {code:title=LeveledManifest.java}
> generations[i] = new ArrayList<SSTableReader>();
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira