You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Mark Robson (JIRA)" <ji...@apache.org> on 2009/05/08 14:55:45 UTC
[jira] Created: (CASSANDRA-156) error reading key until first use
of the HTTP interface
error reading key until first use of the HTTP interface
-------------------------------------------------------
Key: CASSANDRA-156
URL: https://issues.apache.org/jira/browse/CASSANDRA-156
Project: Cassandra
Issue Type: Bug
Affects Versions: trunk
Environment: java version "1.6.0_13" Linux tst04o 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
Reporter: Mark Robson
Priority: Minor
After startup, but before the first access to the HTTP interface, thrift command get_slice returns the following error:
./Cassandra-remote -h tst04o:9160 get_slice Messages 305 base 0 1
Traceback (most recent call last):
File "./Cassandra-remote", line 96, in ?
pp.pprint(client.get_slice(args[0],args[1],args[2],eval(args[3]),eval(args[4]),))
File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 213, in get_slice
return self.recv_get_slice()
File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 233, in recv_get_slice
raise x
thrift.Thrift.TApplicationException: Internal error processing get_slice
Error message on the log file:
ERROR [pool-1-thread-1] 2009-05-08 14:49:36,977 Cassandra.java (line 823) Internal error processing get_slice
java.lang.RuntimeException: error reading key 305
at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:256)
at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:363)
at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:112)
at org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:191)
at org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:817)
at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:805)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.util.concurrent.TimeoutException: Operation timed out.
at org.apache.cassandra.net.AsyncResult.get(AsyncResult.java:95)
at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:252)
... 9 more
After first access to the HTTP interface, the get_slice method now succeeds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-156) error reading key until first use
of the HTTP interface
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708619#action_12708619 ]
Jonathan Ellis commented on CASSANDRA-156:
------------------------------------------
MarkR42: Ok, I reverted the patch and the bug has returned - that's good enough for me :)
committed
> error reading key until first use of the HTTP interface
> -------------------------------------------------------
>
> Key: CASSANDRA-156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-156
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: trunk
> Environment: java version "1.6.0_13" Linux tst04o 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
> Reporter: Mark Robson
> Assignee: Jonathan Ellis
> Priority: Minor
> Attachments: 156.patch
>
>
> After startup, but before the first access to the HTTP interface, thrift command get_slice returns the following error:
> ./Cassandra-remote -h tst04o:9160 get_slice Messages 305 base 0 1
> Traceback (most recent call last):
> File "./Cassandra-remote", line 96, in ?
> pp.pprint(client.get_slice(args[0],args[1],args[2],eval(args[3]),eval(args[4]),))
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 213, in get_slice
> return self.recv_get_slice()
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 233, in recv_get_slice
> raise x
> thrift.Thrift.TApplicationException: Internal error processing get_slice
> Error message on the log file:
> ERROR [pool-1-thread-1] 2009-05-08 14:49:36,977 Cassandra.java (line 823) Internal error processing get_slice
> java.lang.RuntimeException: error reading key 305
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:256)
> at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:363)
> at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:112)
> at org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:191)
> at org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:817)
> at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:805)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.concurrent.TimeoutException: Operation timed out.
> at org.apache.cassandra.net.AsyncResult.get(AsyncResult.java:95)
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:252)
> ... 9 more
> After first access to the HTTP interface, the get_slice method now succeeds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-156) error reading key until first use
of the HTTP interface
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-156:
-------------------------------------
Attachment: 156.patch
> error reading key until first use of the HTTP interface
> -------------------------------------------------------
>
> Key: CASSANDRA-156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-156
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: trunk
> Environment: java version "1.6.0_13" Linux tst04o 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
> Reporter: Mark Robson
> Priority: Minor
> Attachments: 156.patch
>
>
> After startup, but before the first access to the HTTP interface, thrift command get_slice returns the following error:
> ./Cassandra-remote -h tst04o:9160 get_slice Messages 305 base 0 1
> Traceback (most recent call last):
> File "./Cassandra-remote", line 96, in ?
> pp.pprint(client.get_slice(args[0],args[1],args[2],eval(args[3]),eval(args[4]),))
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 213, in get_slice
> return self.recv_get_slice()
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 233, in recv_get_slice
> raise x
> thrift.Thrift.TApplicationException: Internal error processing get_slice
> Error message on the log file:
> ERROR [pool-1-thread-1] 2009-05-08 14:49:36,977 Cassandra.java (line 823) Internal error processing get_slice
> java.lang.RuntimeException: error reading key 305
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:256)
> at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:363)
> at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:112)
> at org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:191)
> at org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:817)
> at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:805)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.concurrent.TimeoutException: Operation timed out.
> at org.apache.cassandra.net.AsyncResult.get(AsyncResult.java:95)
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:252)
> ... 9 more
> After first access to the HTTP interface, the get_slice method now succeeds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-156) error reading key until first use
of the HTTP interface
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708851#action_12708851 ]
Hudson commented on CASSANDRA-156:
----------------------------------
Integrated in Cassandra #73 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/73/])
another workaround for register/select wonkiness. patch by jbellis; tested by Mark Robson for
> error reading key until first use of the HTTP interface
> -------------------------------------------------------
>
> Key: CASSANDRA-156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-156
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: trunk
> Environment: java version "1.6.0_13" Linux tst04o 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
> Reporter: Mark Robson
> Assignee: Jonathan Ellis
> Priority: Minor
> Attachments: 156.patch
>
>
> After startup, but before the first access to the HTTP interface, thrift command get_slice returns the following error:
> ./Cassandra-remote -h tst04o:9160 get_slice Messages 305 base 0 1
> Traceback (most recent call last):
> File "./Cassandra-remote", line 96, in ?
> pp.pprint(client.get_slice(args[0],args[1],args[2],eval(args[3]),eval(args[4]),))
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 213, in get_slice
> return self.recv_get_slice()
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 233, in recv_get_slice
> raise x
> thrift.Thrift.TApplicationException: Internal error processing get_slice
> Error message on the log file:
> ERROR [pool-1-thread-1] 2009-05-08 14:49:36,977 Cassandra.java (line 823) Internal error processing get_slice
> java.lang.RuntimeException: error reading key 305
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:256)
> at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:363)
> at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:112)
> at org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:191)
> at org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:817)
> at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:805)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.concurrent.TimeoutException: Operation timed out.
> at org.apache.cassandra.net.AsyncResult.get(AsyncResult.java:95)
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:252)
> ... 9 more
> After first access to the HTTP interface, the get_slice method now succeeds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-156) error reading key until first use
of the HTTP interface
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707428#action_12707428 ]
Jonathan Ellis commented on CASSANDRA-156:
------------------------------------------
in the future, please add thread dumps as attachments, they're a bit long :)
this part is the key:
"MESSAGE-SERIALIZER-POOL:1" prio=10 tid=0x00002aaafc06a400 nid=0x3387 waiting for monitor entry [0x000000004268d000..0x000000004268dd00]
java.lang.Thread.State: BLOCKED (on object monitor)
at sun.nio.ch.SelectorImpl.register(SelectorImpl.java:115)
- waiting to lock <0x00002aaab2397368> (a java.util.Collections$UnmodifiableSet)
at java.nio.channels.spi.AbstractSelectableChannel.register(AbstractSelectableChannel.java:180)
- locked <0x00002aaab26c39c0> (a java.lang.Object)
at org.apache.cassandra.net.SelectorManager.register(SelectorManager.java:79)
at org.apache.cassandra.net.TcpConnection.<init>(TcpConnection.java:91)
at org.apache.cassandra.net.TcpConnectionManager.getConnection(TcpConnectionManager.java:64)
at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:313)
at org.apache.cassandra.net.MessageSerializationTask.run(MessageSerializationTask.java:66)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
that says that register() is blocking on a lock held by the SelectorManager.select():
selector.select(100);
this is a bug in the jdk or your os. (I'm not sure how to narrow it down further.) The semantics of select(100) are,
* @param timeout If positive, block for up to <tt>timeout</tt>
* milliseconds, more or less, while waiting for a
* channel to become ready; if zero, block indefinitely;
* must not be negative
so each 100ms register() calls should be able to go through but here you are getting stuck indefinitely anyway.
we ran into this in CASSANDRA-97 too, there we were able to re-order things so that all the register()s happen before the first select() call but in this case that doesn't seem possible.
Try the attached patch and see if that helps. If it doesn't, try changing select(100) to select(). (still post-patch-apply.)
> error reading key until first use of the HTTP interface
> -------------------------------------------------------
>
> Key: CASSANDRA-156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-156
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: trunk
> Environment: java version "1.6.0_13" Linux tst04o 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
> Reporter: Mark Robson
> Priority: Minor
> Attachments: 156.patch
>
>
> After startup, but before the first access to the HTTP interface, thrift command get_slice returns the following error:
> ./Cassandra-remote -h tst04o:9160 get_slice Messages 305 base 0 1
> Traceback (most recent call last):
> File "./Cassandra-remote", line 96, in ?
> pp.pprint(client.get_slice(args[0],args[1],args[2],eval(args[3]),eval(args[4]),))
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 213, in get_slice
> return self.recv_get_slice()
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 233, in recv_get_slice
> raise x
> thrift.Thrift.TApplicationException: Internal error processing get_slice
> Error message on the log file:
> ERROR [pool-1-thread-1] 2009-05-08 14:49:36,977 Cassandra.java (line 823) Internal error processing get_slice
> java.lang.RuntimeException: error reading key 305
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:256)
> at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:363)
> at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:112)
> at org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:191)
> at org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:817)
> at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:805)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.concurrent.TimeoutException: Operation timed out.
> at org.apache.cassandra.net.AsyncResult.get(AsyncResult.java:95)
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:252)
> ... 9 more
> After first access to the HTTP interface, the get_slice method now succeeds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-156) error reading key until first use
of the HTTP interface
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-156.
--------------------------------------
Resolution: Fixed
Assignee: Jonathan Ellis
> error reading key until first use of the HTTP interface
> -------------------------------------------------------
>
> Key: CASSANDRA-156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-156
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: trunk
> Environment: java version "1.6.0_13" Linux tst04o 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
> Reporter: Mark Robson
> Assignee: Jonathan Ellis
> Priority: Minor
> Attachments: 156.patch
>
>
> After startup, but before the first access to the HTTP interface, thrift command get_slice returns the following error:
> ./Cassandra-remote -h tst04o:9160 get_slice Messages 305 base 0 1
> Traceback (most recent call last):
> File "./Cassandra-remote", line 96, in ?
> pp.pprint(client.get_slice(args[0],args[1],args[2],eval(args[3]),eval(args[4]),))
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 213, in get_slice
> return self.recv_get_slice()
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 233, in recv_get_slice
> raise x
> thrift.Thrift.TApplicationException: Internal error processing get_slice
> Error message on the log file:
> ERROR [pool-1-thread-1] 2009-05-08 14:49:36,977 Cassandra.java (line 823) Internal error processing get_slice
> java.lang.RuntimeException: error reading key 305
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:256)
> at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:363)
> at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:112)
> at org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:191)
> at org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:817)
> at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:805)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.concurrent.TimeoutException: Operation timed out.
> at org.apache.cassandra.net.AsyncResult.get(AsyncResult.java:95)
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:252)
> ... 9 more
> After first access to the HTTP interface, the get_slice method now succeeds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-156) error reading key until first use
of the HTTP interface
Posted by "Mark Robson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707407#action_12707407 ]
Mark Robson commented on CASSANDRA-156:
---------------------------------------
Thread dump as requested
Full thread dump Java HotSpot(TM) 64-Bit Server VM (11.3-b02 mixed mode):
"pool-1-thread-2" prio=10 tid=0x00002aaafc0e6800 nid=0x33bb waiting on condition [0x0000000041804000..0x0000000041804b00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab24e2680> (a java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:422)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:857)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"pool-1-thread-1" prio=10 tid=0x00002aaafc0dc000 nid=0x33ba waiting on condition [0x0000000041703000..0x0000000041703a80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab24e2680> (a java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:422)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:857)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"GMFD:1" prio=10 tid=0x00002aaafc0da000 nid=0x33b9 waiting on condition [0x00000000457be000..0x00000000457bea00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2422968> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"Timer-1" prio=10 tid=0x00002aaafc0d8800 nid=0x33b8 in Object.wait() [0x00000000456bd000..0x00000000456bdd80]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab23fcdf8> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00002aaab23fcdf8> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"Timer thread for monitoring AnalyticsContext" daemon prio=10 tid=0x00002aaafc0d7000 nid=0x33b7 in Object.wait() [0x00000000455bc000..0x00000000455bcd00]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab23b7480> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00002aaab23b7480> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"UDP Selector Manager" prio=10 tid=0x00002aaafc0d5800 nid=0x33b6 runnable [0x00000000454bb000..0x00000000454bbc80]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00002aaab23a0398> (a sun.nio.ch.Util$1)
- locked <0x00002aaab23a0380> (a java.util.Collections$UnmodifiableSet)
- locked <0x00002aaab23a0020> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at org.apache.cassandra.net.SelectorManager.run(SelectorManager.java:92)
"TCP Selector Manager" prio=10 tid=0x00002aaafc0d0800 nid=0x33b5 runnable [0x0000000041f54000..0x0000000041f54c00]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00002aaab2397380> (a sun.nio.ch.Util$1)
- locked <0x00002aaab2397368> (a java.util.Collections$UnmodifiableSet)
- locked <0x00002aaab2396fe8> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at org.apache.cassandra.net.SelectorManager.run(SelectorManager.java:92)
"HINTED-HANDOFF-POOL:1" prio=10 tid=0x00002aaafc0bf400 nid=0x33b4 waiting on condition [0x00000000453ba000..0x00000000453bab80]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab22f6748> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MINOR-COMPACTION-POOL:1" prio=10 tid=0x00002aaafc0be400 nid=0x33b3 waiting on condition [0x00000000452b9000..0x00000000452b9b00]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab22dc660> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MEMTABLE-POOL-HintsColumnFamily10:1" prio=10 tid=0x00002aaafc0bb000 nid=0x33b2 waiting on condition [0x00000000451b8000..0x00000000451b8a80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab226f360> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MEMTABLE-POOL-StandardByTime29:1" prio=10 tid=0x00002aaafc0b9800 nid=0x33b1 waiting on condition [0x00000000450b7000..0x00000000450b7a00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab226ae38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MEMTABLE-POOL-TableMetadata8:1" prio=10 tid=0x00002aaafc0b8000 nid=0x33b0 waiting on condition [0x0000000044fb6000..0x0000000044fb6d80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2266918> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MEMTABLE-POOL-StandardByTime17:1" prio=10 tid=0x00002aaafc0b3000 nid=0x33af waiting on condition [0x0000000044eb5000..0x0000000044eb5d00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2262400> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MEMTABLE-POOL-LocationInfo6:1" prio=10 tid=0x00002aaafc0b1800 nid=0x33ae waiting on condition [0x0000000044db4000..0x0000000044db4c80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab225df20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MEMTABLE-POOL-base5:1" prio=10 tid=0x00002aaafc0afc00 nid=0x33ad waiting on condition [0x0000000044cb3000..0x0000000044cb3c00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2259ae8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MEMTABLE-POOL-Super24:1" prio=10 tid=0x00002aaafc0ae000 nid=0x33ac waiting on condition [0x0000000044bb2000..0x0000000044bb2b80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2255680> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MEMTABLE-POOL-extra3:1" prio=10 tid=0x00002aaafc0ac800 nid=0x33ab waiting on condition [0x0000000044ab1000..0x0000000044ab1b00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2251188> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MEMTABLE-POOL-Super12:1" prio=10 tid=0x00002aaafc0abc00 nid=0x33aa waiting on condition [0x00000000449b0000..0x00000000449b0a80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab224cd20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MEMTABLE-POOL-RecycleColumnFamily1:1" prio=10 tid=0x00002aaafc0a8400 nid=0x33a9 waiting on condition [0x00000000448af000..0x00000000448afa00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab223e860> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"HTTP-REQUEST:1" prio=10 tid=0x00002aaafc0a1400 nid=0x33a8 waiting on condition [0x00000000447ae000..0x00000000447aed80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2159ed8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MAP-REDUCE-STAGE:4" prio=10 tid=0x00002aaafc09f800 nid=0x33a7 waiting on condition [0x00000000446ad000..0x00000000446add00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2157b70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MAP-REDUCE-STAGE:3" prio=10 tid=0x00002aaafc09e000 nid=0x33a6 waiting on condition [0x00000000445ac000..0x00000000445acc80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2157b70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MAP-REDUCE-STAGE:2" prio=10 tid=0x00002aaafc09c400 nid=0x33a5 waiting on condition [0x00000000444ab000..0x00000000444abc00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2157b70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MAP-REDUCE-STAGE:1" prio=10 tid=0x00002aaafc09a800 nid=0x33a4 waiting on condition [0x00000000443aa000..0x00000000443aab80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2157b70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-READ-STAGE:8" prio=10 tid=0x00002aaafc098c00 nid=0x33a3 waiting on condition [0x00000000442a9000..0x00000000442a9b00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2154ca0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-READ-STAGE:7" prio=10 tid=0x00002aaafc097400 nid=0x33a2 waiting on condition [0x00000000441a8000..0x00000000441a8a80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2154ca0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-READ-STAGE:6" prio=10 tid=0x00002aaafc095800 nid=0x33a1 waiting on condition [0x00000000440a7000..0x00000000440a7a00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2154ca0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-READ-STAGE:5" prio=10 tid=0x00002aaafc093c00 nid=0x33a0 waiting on condition [0x0000000043fa6000..0x0000000043fa6d80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2154ca0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-READ-STAGE:4" prio=10 tid=0x00002aaafc092000 nid=0x339f waiting on condition [0x0000000043ea5000..0x0000000043ea5d00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2154ca0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-READ-STAGE:3" prio=10 tid=0x00002aaafc090c00 nid=0x339e waiting on condition [0x0000000043da4000..0x0000000043da4c80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2154ca0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-READ-STAGE:2" prio=10 tid=0x00002aaafc08f800 nid=0x339d waiting on condition [0x0000000043ca3000..0x0000000043ca3c00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2154ca0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-READ-STAGE:1" prio=10 tid=0x00002aaafc08e400 nid=0x339c waiting on condition [0x0000000043ba2000..0x0000000043ba2b80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2154ca0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-MUTATION-STAGE:4" prio=10 tid=0x00002aaafc08c800 nid=0x339b waiting on condition [0x0000000043aa1000..0x0000000043aa1b00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2152838> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-MUTATION-STAGE:3" prio=10 tid=0x00002aaafc08ac00 nid=0x339a waiting on condition [0x00000000439a0000..0x00000000439a0a80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2152838> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-MUTATION-STAGE:2" prio=10 tid=0x00002aaafc089400 nid=0x3399 waiting on condition [0x000000004389f000..0x000000004389fa00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2152838> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"ROW-MUTATION-STAGE:1" prio=10 tid=0x00002aaafc087800 nid=0x3398 waiting on condition [0x000000004379e000..0x000000004379ed80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2152838> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"CONSISTENCY-MANAGER:4" prio=10 tid=0x00002aaafc085c00 nid=0x3397 waiting on condition [0x000000004369d000..0x000000004369dd00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab21505d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"CONSISTENCY-MANAGER:3" prio=10 tid=0x00002aaafc084400 nid=0x3396 waiting on condition [0x000000004359c000..0x000000004359cc80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab21505d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"CONSISTENCY-MANAGER:2" prio=10 tid=0x00002aaafc082800 nid=0x3395 waiting on condition [0x000000004349b000..0x000000004349bc00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab21505d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"CONSISTENCY-MANAGER:1" prio=10 tid=0x00002aaafc080c00 nid=0x3394 waiting on condition [0x000000004339a000..0x000000004339ab80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab21505d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"RESPONSE-STAGE:4" prio=10 tid=0x00002aaafc07e800 nid=0x3393 waiting on condition [0x0000000043299000..0x0000000043299b00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20f6e28> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"RESPONSE-STAGE:3" prio=10 tid=0x00002aaafc07d000 nid=0x3392 waiting on condition [0x0000000043198000..0x0000000043198a80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20f6e28> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"RESPONSE-STAGE:2" prio=10 tid=0x00002aaafc07b400 nid=0x3391 waiting on condition [0x0000000043097000..0x0000000043097a00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20f6e28> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"RESPONSE-STAGE:1" prio=10 tid=0x00002aaafc079800 nid=0x3390 waiting on condition [0x0000000042f96000..0x0000000042f96d80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20f6e28> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGE-STREAMING-POOL:1" prio=10 tid=0x00002aaafc077c00 nid=0x338f waiting on condition [0x0000000042e95000..0x0000000042e95d00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20ec248> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGE-DESERIALIZER-POOL:4" prio=10 tid=0x00002aaafc076000 nid=0x338e waiting on condition [0x0000000042d94000..0x0000000042d94c80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20e9e70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGE-DESERIALIZER-POOL:3" prio=10 tid=0x00002aaafc074400 nid=0x338d waiting on condition [0x0000000042c93000..0x0000000042c93c00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20e9e70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGE-DESERIALIZER-POOL:2" prio=10 tid=0x00002aaafc072c00 nid=0x338c waiting on condition [0x0000000042b92000..0x0000000042b92b80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20e9e70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGE-DESERIALIZER-POOL:1" prio=10 tid=0x00002aaafc071000 nid=0x338b waiting on condition [0x0000000042a91000..0x0000000042a91b00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20e9e70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGE-SERIALIZER-POOL:4" prio=10 tid=0x00002aaafc06f400 nid=0x338a waiting on condition [0x0000000042990000..0x0000000042990a80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20e7bd0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGE-SERIALIZER-POOL:3" prio=10 tid=0x00002aaafc06d800 nid=0x3389 waiting on condition [0x000000004288f000..0x000000004288fa00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20e7bd0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGE-SERIALIZER-POOL:2" prio=10 tid=0x00002aaafc06c000 nid=0x3388 waiting on condition [0x000000004278e000..0x000000004278ed80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab26a45e8> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
at org.apache.cassandra.net.TcpConnectionManager.getConnection(TcpConnectionManager.java:59)
at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:313)
at org.apache.cassandra.net.MessageSerializationTask.run(MessageSerializationTask.java:66)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
"MESSAGE-SERIALIZER-POOL:1" prio=10 tid=0x00002aaafc06a400 nid=0x3387 waiting for monitor entry [0x000000004268d000..0x000000004268dd00]
java.lang.Thread.State: BLOCKED (on object monitor)
at sun.nio.ch.SelectorImpl.register(SelectorImpl.java:115)
- waiting to lock <0x00002aaab2397368> (a java.util.Collections$UnmodifiableSet)
at java.nio.channels.spi.AbstractSelectableChannel.register(AbstractSelectableChannel.java:180)
- locked <0x00002aaab26c39c0> (a java.lang.Object)
at org.apache.cassandra.net.SelectorManager.register(SelectorManager.java:79)
at org.apache.cassandra.net.TcpConnection.<init>(TcpConnection.java:91)
at org.apache.cassandra.net.TcpConnectionManager.getConnection(TcpConnectionManager.java:64)
at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:313)
at org.apache.cassandra.net.MessageSerializationTask.run(MessageSerializationTask.java:66)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
"MESSAGING-SERVICE-POOL:4" prio=10 tid=0x00002aaafc060000 nid=0x3386 waiting on condition [0x000000004258c000..0x000000004258cc80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20e58d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGING-SERVICE-POOL:3" prio=10 tid=0x00002aaafc05e400 nid=0x3385 waiting on condition [0x000000004248b000..0x000000004248bc00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20e58d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGING-SERVICE-POOL:2" prio=10 tid=0x00002aaafc05c800 nid=0x3384 waiting on condition [0x000000004238a000..0x000000004238ab80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20e58d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MESSAGING-SERVICE-POOL:1" prio=10 tid=0x00002aaafc05b400 nid=0x3383 waiting on condition [0x0000000042289000..0x0000000042289b00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20e58d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"CACHETABLE-TIMER-2" daemon prio=10 tid=0x00002aaafc062000 nid=0x3382 in Object.wait() [0x0000000042188000..0x0000000042188a80]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab20e4fc8> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00002aaab20e4fc8> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"CACHETABLE-TIMER-1" daemon prio=10 tid=0x00002aaafc060c00 nid=0x3381 in Object.wait() [0x0000000042087000..0x0000000042087a00]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab20e44f8> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00002aaab20e44f8> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"LOAD-BALANCER-STAGE:1" prio=10 tid=0x00002aaafc04e800 nid=0x3380 waiting on condition [0x0000000041602000..0x0000000041602d80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab2077370> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"LB-TARGET:1" prio=10 tid=0x00002aaafc050000 nid=0x337f waiting on condition [0x0000000041501000..0x0000000041501d00]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab20715b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"BOOT-STRAPPER:1" prio=10 tid=0x00002aaafc053800 nid=0x337e waiting on condition [0x0000000041400000..0x0000000041400c80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab205cbf8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"Timer-0" prio=10 tid=0x00002aaafc052400 nid=0x337d in Object.wait() [0x00000000412ff000..0x00000000412ffc00]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab205bb28> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:509)
- locked <0x00002aaab205bb28> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"FILEUTILS-DELETE-POOL:1" prio=10 tid=0x00002aaafc04c000 nid=0x337c waiting on condition [0x00000000411fe000..0x00000000411feb80]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00002aaab29d07e0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"RMI TCP Accept-0" daemon prio=10 tid=0x0000000057080c00 nid=0x337a runnable [0x0000000040e43000..0x0000000040e43a80]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x00002aaab29d0978> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:34)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
at java.lang.Thread.run(Thread.java:619)
"RMI TCP Accept-8080" daemon prio=10 tid=0x000000005706f800 nid=0x3379 runnable [0x00000000410fd000..0x00000000410fda00]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x00002aaab29d0b88> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
at java.lang.Thread.run(Thread.java:619)
"RMI TCP Accept-0" daemon prio=10 tid=0x0000000057065c00 nid=0x3378 runnable [0x0000000040d42000..0x0000000040d42d80]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x00002aaab29d0d90> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
at java.lang.Thread.run(Thread.java:619)
"Low Memory Detector" daemon prio=10 tid=0x0000000056d65400 nid=0x3377 runnable [0x0000000000000000..0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"CompilerThread1" daemon prio=10 tid=0x0000000056d63000 nid=0x3376 waiting on condition [0x0000000000000000..0x0000000041d51480]
java.lang.Thread.State: RUNNABLE
"CompilerThread0" daemon prio=10 tid=0x0000000056d5f000 nid=0x3375 waiting on condition [0x0000000000000000..0x0000000041c503b0]
java.lang.Thread.State: RUNNABLE
"JDWP Event Helper Thread" daemon prio=10 tid=0x0000000056d49c00 nid=0x3374 runnable [0x0000000000000000..0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"JDWP Transport Listener: dt_socket" daemon prio=10 tid=0x0000000056d46400 nid=0x3373 runnable [0x0000000000000000..0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x0000000056d38000 nid=0x3372 runnable [0x0000000000000000..0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Surrogate Locker Thread (CMS)" daemon prio=10 tid=0x0000000056d36400 nid=0x3371 waiting on condition [0x0000000000000000..0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0x0000000056d13800 nid=0x3370 in Object.wait() [0x00000000405df000..0x00000000405dfd80]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab29d1220> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x00002aaab29d1220> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0x0000000056d11c00 nid=0x336f in Object.wait() [0x00000000404de000..0x00000000404ded00]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaab29d1258> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x00002aaab29d1258> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0x0000000056c91400 nid=0x336c runnable [0x000000004080b000..0x000000004080bec0]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x00002aaab24be1c8> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:118)
at org.apache.thrift.transport.TServerSocket.acceptImpl(TServerSocket.java:34)
at org.apache.thrift.transport.TServerTransport.accept(TServerTransport.java:31)
at org.apache.thrift.server.TThreadPoolServer.serve(TThreadPoolServer.java:183)
at org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:97)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:133)
"VM Thread" prio=10 tid=0x0000000056d0c400 nid=0x336e runnable
"Concurrent Mark-Sweep GC Thread" prio=10 tid=0x0000000056cb0c00 nid=0x336d runnable
"VM Periodic Task Thread" prio=10 tid=0x0000000057083000 nid=0x337b waiting on condition
JNI global references: 2577
Heap
def new generation total 19136K, used 18899K [0x00002aaab1930000, 0x00002aaab2df0000, 0x00002aaab2df0000)
eden space 17024K, 99% used [0x00002aaab1930000, 0x00002aaab29bff78, 0x00002aaab29d0000)
from space 2112K, 91% used [0x00002aaab29d0000, 0x00002aaab2bb4e40, 0x00002aaab2be0000)
to space 2112K, 0% used [0x00002aaab2be0000, 0x00002aaab2be0000, 0x00002aaab2df0000)
concurrent mark-sweep generation total 109824K, used 0K [0x00002aaab2df0000, 0x00002aaab9930000, 0x00002aaaf1930000)
concurrent-mark-sweep perm gen total 21248K, used 13606K [0x00002aaaf1930000, 0x00002aaaf2df0000, 0x00002aaaf6d30000)
> error reading key until first use of the HTTP interface
> -------------------------------------------------------
>
> Key: CASSANDRA-156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-156
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: trunk
> Environment: java version "1.6.0_13" Linux tst04o 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
> Reporter: Mark Robson
> Priority: Minor
>
> After startup, but before the first access to the HTTP interface, thrift command get_slice returns the following error:
> ./Cassandra-remote -h tst04o:9160 get_slice Messages 305 base 0 1
> Traceback (most recent call last):
> File "./Cassandra-remote", line 96, in ?
> pp.pprint(client.get_slice(args[0],args[1],args[2],eval(args[3]),eval(args[4]),))
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 213, in get_slice
> return self.recv_get_slice()
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 233, in recv_get_slice
> raise x
> thrift.Thrift.TApplicationException: Internal error processing get_slice
> Error message on the log file:
> ERROR [pool-1-thread-1] 2009-05-08 14:49:36,977 Cassandra.java (line 823) Internal error processing get_slice
> java.lang.RuntimeException: error reading key 305
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:256)
> at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:363)
> at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:112)
> at org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:191)
> at org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:817)
> at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:805)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.concurrent.TimeoutException: Operation timed out.
> at org.apache.cassandra.net.AsyncResult.get(AsyncResult.java:95)
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:252)
> ... 9 more
> After first access to the HTTP interface, the get_slice method now succeeds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-156) error reading key until first use
of the HTTP interface
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707363#action_12707363 ]
Jonathan Ellis commented on CASSANDRA-156:
------------------------------------------
how many nodes are you running? can you reproduce this at will on a fresh start after wiping /var/cassandra/* ?
> error reading key until first use of the HTTP interface
> -------------------------------------------------------
>
> Key: CASSANDRA-156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-156
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: trunk
> Environment: java version "1.6.0_13" Linux tst04o 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
> Reporter: Mark Robson
> Priority: Minor
>
> After startup, but before the first access to the HTTP interface, thrift command get_slice returns the following error:
> ./Cassandra-remote -h tst04o:9160 get_slice Messages 305 base 0 1
> Traceback (most recent call last):
> File "./Cassandra-remote", line 96, in ?
> pp.pprint(client.get_slice(args[0],args[1],args[2],eval(args[3]),eval(args[4]),))
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 213, in get_slice
> return self.recv_get_slice()
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 233, in recv_get_slice
> raise x
> thrift.Thrift.TApplicationException: Internal error processing get_slice
> Error message on the log file:
> ERROR [pool-1-thread-1] 2009-05-08 14:49:36,977 Cassandra.java (line 823) Internal error processing get_slice
> java.lang.RuntimeException: error reading key 305
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:256)
> at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:363)
> at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:112)
> at org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:191)
> at org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:817)
> at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:805)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.concurrent.TimeoutException: Operation timed out.
> at org.apache.cassandra.net.AsyncResult.get(AsyncResult.java:95)
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:252)
> ... 9 more
> After first access to the HTTP interface, the get_slice method now succeeds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-156) error reading key until first use
of the HTTP interface
Posted by "Mark Robson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708613#action_12708613 ]
Mark Robson commented on CASSANDRA-156:
---------------------------------------
156.patch looks like it fixes it, although I've also made a new build off trunk, so something in there could have done instead.
Anyway it seems to have gone away ( the problem) after I used the patched version.
> error reading key until first use of the HTTP interface
> -------------------------------------------------------
>
> Key: CASSANDRA-156
> URL: https://issues.apache.org/jira/browse/CASSANDRA-156
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: trunk
> Environment: java version "1.6.0_13" Linux tst04o 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:10:25 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
> Reporter: Mark Robson
> Priority: Minor
> Attachments: 156.patch
>
>
> After startup, but before the first access to the HTTP interface, thrift command get_slice returns the following error:
> ./Cassandra-remote -h tst04o:9160 get_slice Messages 305 base 0 1
> Traceback (most recent call last):
> File "./Cassandra-remote", line 96, in ?
> pp.pprint(client.get_slice(args[0],args[1],args[2],eval(args[3]),eval(args[4]),))
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 213, in get_slice
> return self.recv_get_slice()
> File "/opt/mailcontrol/gen-py/org/apache/cassandra/Cassandra.py", line 233, in recv_get_slice
> raise x
> thrift.Thrift.TApplicationException: Internal error processing get_slice
> Error message on the log file:
> ERROR [pool-1-thread-1] 2009-05-08 14:49:36,977 Cassandra.java (line 823) Internal error processing get_slice
> java.lang.RuntimeException: error reading key 305
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:256)
> at org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:363)
> at org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:112)
> at org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:191)
> at org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:817)
> at org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:805)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.concurrent.TimeoutException: Operation timed out.
> at org.apache.cassandra.net.AsyncResult.get(AsyncResult.java:95)
> at org.apache.cassandra.service.StorageProxy.weakReadRemote(StorageProxy.java:252)
> ... 9 more
> After first access to the HTTP interface, the get_slice method now succeeds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.