You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Tyler Hobbs (JIRA)" <ji...@apache.org> on 2014/04/08 23:41:16 UTC
[jira] [Commented] (CASSANDRA-7007) Streaming Errors due to
RejectedExecutionException
[ https://issues.apache.org/jira/browse/CASSANDRA-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963478#comment-13963478 ]
Tyler Hobbs commented on CASSANDRA-7007:
----------------------------------------
+1
> Streaming Errors due to RejectedExecutionException
> --------------------------------------------------
>
> Key: CASSANDRA-7007
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7007
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Tyler Hobbs
> Assignee: Yuki Morishita
> Priority: Blocker
> Fix For: 2.0.7
>
> Attachments: 7007-2.0.txt
>
>
> With the same repair setup as CASSANDRA-6984, I'm now seeing repair failures due to some streaming tasks hitting RejectedExecutionException:
> {noformat}
> ERROR [STREAM-OUT-/127.0.0.3] 2014-04-08 13:50:15,424 StreamSession.java (line 420) [Stream #9f64e230-bf4e-11e3-b994-43eb3a328df9] Streaming error occurred
> java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@1eb6b48a rejected from java.util.concurrent.ScheduledThreadPoolExecutor@6d06dfc6[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
> at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
> at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
> at java.util.concurrent.Executors$DelegatedScheduledExecutorService.schedule(Executors.java:689)
> at org.apache.cassandra.streaming.StreamTransferTask.scheduleTimeout(StreamTransferTask.java:125)
> at org.apache.cassandra.streaming.StreamSession.fileSent(StreamSession.java:468)
> at org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:60)
> at org.apache.cassandra.streaming.messages.OutgoingFileMessage$1.serialize(OutgoingFileMessage.java:42)
> at org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:45)
> at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:383)
> at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:355)
> at java.lang.Thread.run(Thread.java:724)
> INFO [STREAM-OUT-/127.0.0.3] 2014-04-08 13:50:15,431 StreamResultFuture.java (line 186) [Stream #9f64e230-bf4e-11e3-b994-43eb3a328df9] Session with /127.0.0.3 is complete
> WARN [STREAM-OUT-/127.0.0.3] 2014-04-08 13:50:15,431 StreamResultFuture.java (line 215) [Stream #9f64e230-bf4e-11e3-b994-43eb3a328df9] Stream failed
> ERROR [NonPeriodicTasks:1] 2014-04-08 13:50:15,449 CassandraDaemon.java (line 198) Exception in thread Thread[NonPeriodicTasks:1,5,main]
> java.lang.RuntimeException: Outgoing stream handler has been closed
> at org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:170)
> at org.apache.cassandra.streaming.StreamSession.maybeCompleted(StreamSession.java:620)
> at org.apache.cassandra.streaming.StreamSession.taskCompleted(StreamSession.java:566)
> at org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:120)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> {noformat}
> Since the ThreadPoolExecutor says its state is TERMINATED, I'm guessing this is due to a race condition where the timeout executor can be shutdown too early.
--
This message was sent by Atlassian JIRA
(v6.2#6252)