You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Patrik Modesto (JIRA)" <ji...@apache.org> on 2011/02/08 07:57:57 UTC
[jira] Created: (CASSANDRA-2134) Error in ThreadPoolExecutor
Error in ThreadPoolExecutor
---------------------------
Key: CASSANDRA-2134
URL: https://issues.apache.org/jira/browse/CASSANDRA-2134
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.1
Environment: Linux 2.6.32-bpo.4edois1-openvz-amd64 #1 SMP x86_64 GNU/Linux
Reporter: Patrik Modesto
On my two-node test setup I get repeatedly following error:
The 10.0.18.129 server log:
{noformat}
INFO 14:10:37,707 Node /10.0.18.99 has restarted, now UP again
INFO 14:10:37,708 Checking remote schema before delivering hints
INFO 14:10:37,708 Sleeping 45506ms to stagger hint delivery
INFO 14:10:37,709 Node /10.0.18.99 state jump to normal
INFO 14:11:23,215 Started hinted handoff for endpoint /10.0.18.99
ERROR 14:11:23,884 Error in ThreadPoolExecutor
java.lang.RuntimeException: java.lang.IllegalArgumentException
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.position(Buffer.java:218)
at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
ERROR 14:11:23,885 Fatal exception in thread Thread[HintedHandoff:1,1,main]
java.lang.RuntimeException: java.lang.IllegalArgumentException
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.position(Buffer.java:218)
at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
{noformat}
The 10.0.18.99 server log:
{noformat}
INFO 14:10:37,691 Binding thrift service to /0.0.0.0:9160
INFO 14:10:37,693 Using TFastFramedTransport with a max frame size of
15728640 bytes.
INFO 14:10:37,695 Listening for thrift clients...
INFO 14:10:38,337 GC for ParNew: 954 ms, 658827608 reclaimed leaving
966732432 used; max is 4265607168
INFO 14:11:27,142 Started hinted handoff for endpoint /10.0.18.129
ERROR 14:11:27,370 Error in ThreadPoolExecutor
java.lang.RuntimeException: java.lang.IllegalArgumentException
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.position(Buffer.java:218)
at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
ERROR 14:11:27,371 Fatal exception in thread Thread[HintedHandoff:1,1,main]
java.lang.RuntimeException: java.lang.IllegalArgumentException
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.position(Buffer.java:218)
at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
{noformat}
It happen durring batch_mutate test or after restart, when there are commitlogs to replay. Using current 0.7.1 from cassandra-0.7 branch.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2134) Error in ThreadPoolExecutor
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992001#comment-12992001 ]
Hudson commented on CASSANDRA-2134:
-----------------------------------
Integrated in Cassandra-0.7 #258 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/258/])
fix ByteBufferUtil.string position
patch by slebresne; reviewed by jbellis for CASSANDRA-2134
> Error in ThreadPoolExecutor
> ---------------------------
>
> Key: CASSANDRA-2134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2134
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.1
> Environment: Linux 2.6.32-bpo.4edois1-openvz-amd64 #1 SMP x86_64 GNU/Linux
> Reporter: Patrik Modesto
> Assignee: Sylvain Lebresne
> Fix For: 0.7.1
>
> Attachments: 0001-Fix-BBUtil.string-offset-related-to-position-instead.patch
>
>
> On my two-node test setup I get repeatedly following error:
> The 10.0.18.129 server log:
> {noformat}
> INFO 14:10:37,707 Node /10.0.18.99 has restarted, now UP again
> INFO 14:10:37,708 Checking remote schema before delivering hints
> INFO 14:10:37,708 Sleeping 45506ms to stagger hint delivery
> INFO 14:10:37,709 Node /10.0.18.99 state jump to normal
> INFO 14:11:23,215 Started hinted handoff for endpoint /10.0.18.99
> ERROR 14:11:23,884 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:23,885 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> The 10.0.18.99 server log:
> {noformat}
> INFO 14:10:37,691 Binding thrift service to /0.0.0.0:9160
> INFO 14:10:37,693 Using TFastFramedTransport with a max frame size of
> 15728640 bytes.
> INFO 14:10:37,695 Listening for thrift clients...
> INFO 14:10:38,337 GC for ParNew: 954 ms, 658827608 reclaimed leaving
> 966732432 used; max is 4265607168
> INFO 14:11:27,142 Started hinted handoff for endpoint /10.0.18.129
> ERROR 14:11:27,370 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:27,371 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> It happen durring batch_mutate test or after restart, when there are commitlogs to replay. Using current 0.7.1 from cassandra-0.7 branch.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2134) Error in ThreadPoolExecutor
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2134:
--------------------------------------
Remaining Estimate: 1m
Original Estimate: 1m
> Error in ThreadPoolExecutor
> ---------------------------
>
> Key: CASSANDRA-2134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2134
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.1
> Environment: Linux 2.6.32-bpo.4edois1-openvz-amd64 #1 SMP x86_64 GNU/Linux
> Reporter: Patrik Modesto
> Assignee: Sylvain Lebresne
> Fix For: 0.7.1
>
> Attachments: 0001-Fix-BBUtil.string-offset-related-to-position-instead.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> On my two-node test setup I get repeatedly following error:
> The 10.0.18.129 server log:
> {noformat}
> INFO 14:10:37,707 Node /10.0.18.99 has restarted, now UP again
> INFO 14:10:37,708 Checking remote schema before delivering hints
> INFO 14:10:37,708 Sleeping 45506ms to stagger hint delivery
> INFO 14:10:37,709 Node /10.0.18.99 state jump to normal
> INFO 14:11:23,215 Started hinted handoff for endpoint /10.0.18.99
> ERROR 14:11:23,884 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:23,885 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> The 10.0.18.99 server log:
> {noformat}
> INFO 14:10:37,691 Binding thrift service to /0.0.0.0:9160
> INFO 14:10:37,693 Using TFastFramedTransport with a max frame size of
> 15728640 bytes.
> INFO 14:10:37,695 Listening for thrift clients...
> INFO 14:10:38,337 GC for ParNew: 954 ms, 658827608 reclaimed leaving
> 966732432 used; max is 4265607168
> INFO 14:11:27,142 Started hinted handoff for endpoint /10.0.18.129
> ERROR 14:11:27,370 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:27,371 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> It happen durring batch_mutate test or after restart, when there are commitlogs to replay. Using current 0.7.1 from cassandra-0.7 branch.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2134) Error in ThreadPoolExecutor
Posted by "Patrik Modesto (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992018#comment-12992018 ]
Patrik Modesto commented on CASSANDRA-2134:
-------------------------------------------
I can still get exception in ThreadPoolExecutor.
1st server:
{noformat}
ERROR 16:50:34,349 Error in ThreadPoolExecutor
java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at org.apache.cassandra.db.Table.createReplicationStrategy(Table.java:318)
at org.apache.cassandra.db.Table.<init>(Table.java:258)
at org.apache.cassandra.db.Table.open(Table.java:107)
at org.apache.cassandra.db.HintedHandOffManager.sendMessage(HintedHandOffManager.java:131)
at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:313)
at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:391)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
ERROR 16:50:34,351 Fatal exception in thread Thread[HintedHandoff:1,1,main]
java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at org.apache.cassandra.db.Table.createReplicationStrategy(Table.java:318)
at org.apache.cassandra.db.Table.<init>(Table.java:258)
at org.apache.cassandra.db.Table.open(Table.java:107)
at org.apache.cassandra.db.HintedHandOffManager.sendMessage(HintedHandOffManager.java:131)
at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:313)
at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:391)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
{noformat}
2nd server:
{noformat}
ERROR 16:50:37,532 Error in ThreadPoolExecutor
java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at org.apache.cassandra.db.Table.createReplicationStrategy(Table.java:318)
at org.apache.cassandra.db.Table.<init>(Table.java:258)
at org.apache.cassandra.db.Table.open(Table.java:107)
at org.apache.cassandra.db.HintedHandOffManager.sendMessage(HintedHandOffManager.java:131)
at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:313)
at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:391)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
ERROR 16:50:37,547 Fatal exception in thread Thread[HintedHandoff:1,1,main]
java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
at org.apache.cassandra.db.Table.createReplicationStrategy(Table.java:318)
at org.apache.cassandra.db.Table.<init>(Table.java:258)
at org.apache.cassandra.db.Table.open(Table.java:107)
at org.apache.cassandra.db.HintedHandOffManager.sendMessage(HintedHandOffManager.java:131)
at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:313)
at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:391)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
{noformat}
> Error in ThreadPoolExecutor
> ---------------------------
>
> Key: CASSANDRA-2134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2134
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.1
> Environment: Linux 2.6.32-bpo.4edois1-openvz-amd64 #1 SMP x86_64 GNU/Linux
> Reporter: Patrik Modesto
> Assignee: Sylvain Lebresne
> Fix For: 0.7.1
>
> Attachments: 0001-Fix-BBUtil.string-offset-related-to-position-instead.patch
>
>
> On my two-node test setup I get repeatedly following error:
> The 10.0.18.129 server log:
> {noformat}
> INFO 14:10:37,707 Node /10.0.18.99 has restarted, now UP again
> INFO 14:10:37,708 Checking remote schema before delivering hints
> INFO 14:10:37,708 Sleeping 45506ms to stagger hint delivery
> INFO 14:10:37,709 Node /10.0.18.99 state jump to normal
> INFO 14:11:23,215 Started hinted handoff for endpoint /10.0.18.99
> ERROR 14:11:23,884 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:23,885 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> The 10.0.18.99 server log:
> {noformat}
> INFO 14:10:37,691 Binding thrift service to /0.0.0.0:9160
> INFO 14:10:37,693 Using TFastFramedTransport with a max frame size of
> 15728640 bytes.
> INFO 14:10:37,695 Listening for thrift clients...
> INFO 14:10:38,337 GC for ParNew: 954 ms, 658827608 reclaimed leaving
> 966732432 used; max is 4265607168
> INFO 14:11:27,142 Started hinted handoff for endpoint /10.0.18.129
> ERROR 14:11:27,370 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:27,371 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> It happen durring batch_mutate test or after restart, when there are commitlogs to replay. Using current 0.7.1 from cassandra-0.7 branch.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2134) Error in ThreadPoolExecutor
Posted by "Patrik Modesto (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992454#comment-12992454 ]
Patrik Modesto commented on CASSANDRA-2134:
-------------------------------------------
Deleting the old HintColumnFamily files helped. Thanks.
> Error in ThreadPoolExecutor
> ---------------------------
>
> Key: CASSANDRA-2134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2134
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.1
> Environment: Linux 2.6.32-bpo.4edois1-openvz-amd64 #1 SMP x86_64 GNU/Linux
> Reporter: Patrik Modesto
> Assignee: Sylvain Lebresne
> Fix For: 0.7.1
>
> Attachments: 0001-Fix-BBUtil.string-offset-related-to-position-instead.patch
>
>
> On my two-node test setup I get repeatedly following error:
> The 10.0.18.129 server log:
> {noformat}
> INFO 14:10:37,707 Node /10.0.18.99 has restarted, now UP again
> INFO 14:10:37,708 Checking remote schema before delivering hints
> INFO 14:10:37,708 Sleeping 45506ms to stagger hint delivery
> INFO 14:10:37,709 Node /10.0.18.99 state jump to normal
> INFO 14:11:23,215 Started hinted handoff for endpoint /10.0.18.99
> ERROR 14:11:23,884 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:23,885 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> The 10.0.18.99 server log:
> {noformat}
> INFO 14:10:37,691 Binding thrift service to /0.0.0.0:9160
> INFO 14:10:37,693 Using TFastFramedTransport with a max frame size of
> 15728640 bytes.
> INFO 14:10:37,695 Listening for thrift clients...
> INFO 14:10:38,337 GC for ParNew: 954 ms, 658827608 reclaimed leaving
> 966732432 used; max is 4265607168
> INFO 14:11:27,142 Started hinted handoff for endpoint /10.0.18.129
> ERROR 14:11:27,370 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:27,371 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> It happen durring batch_mutate test or after restart, when there are commitlogs to replay. Using current 0.7.1 from cassandra-0.7 branch.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2134) Error in ThreadPoolExecutor
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992043#comment-12992043 ]
Jonathan Ellis commented on CASSANDRA-2134:
-------------------------------------------
If you haven't removed the corrupt HintsColumnFamily files, you need to do so.
> Error in ThreadPoolExecutor
> ---------------------------
>
> Key: CASSANDRA-2134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2134
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.1
> Environment: Linux 2.6.32-bpo.4edois1-openvz-amd64 #1 SMP x86_64 GNU/Linux
> Reporter: Patrik Modesto
> Assignee: Sylvain Lebresne
> Fix For: 0.7.1
>
> Attachments: 0001-Fix-BBUtil.string-offset-related-to-position-instead.patch
>
>
> On my two-node test setup I get repeatedly following error:
> The 10.0.18.129 server log:
> {noformat}
> INFO 14:10:37,707 Node /10.0.18.99 has restarted, now UP again
> INFO 14:10:37,708 Checking remote schema before delivering hints
> INFO 14:10:37,708 Sleeping 45506ms to stagger hint delivery
> INFO 14:10:37,709 Node /10.0.18.99 state jump to normal
> INFO 14:11:23,215 Started hinted handoff for endpoint /10.0.18.99
> ERROR 14:11:23,884 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:23,885 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> The 10.0.18.99 server log:
> {noformat}
> INFO 14:10:37,691 Binding thrift service to /0.0.0.0:9160
> INFO 14:10:37,693 Using TFastFramedTransport with a max frame size of
> 15728640 bytes.
> INFO 14:10:37,695 Listening for thrift clients...
> INFO 14:10:38,337 GC for ParNew: 954 ms, 658827608 reclaimed leaving
> 966732432 used; max is 4265607168
> INFO 14:11:27,142 Started hinted handoff for endpoint /10.0.18.129
> ERROR 14:11:27,370 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:27,371 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> It happen durring batch_mutate test or after restart, when there are commitlogs to replay. Using current 0.7.1 from cassandra-0.7 branch.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2134) Error in ThreadPoolExecutor
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-2134:
----------------------------------------
Attachment: 0001-Fix-BBUtil.string-offset-related-to-position-instead.patch
Attached patch should fix this.
> Error in ThreadPoolExecutor
> ---------------------------
>
> Key: CASSANDRA-2134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2134
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.1
> Environment: Linux 2.6.32-bpo.4edois1-openvz-amd64 #1 SMP x86_64 GNU/Linux
> Reporter: Patrik Modesto
> Fix For: 0.7.1
>
> Attachments: 0001-Fix-BBUtil.string-offset-related-to-position-instead.patch
>
>
> On my two-node test setup I get repeatedly following error:
> The 10.0.18.129 server log:
> {noformat}
> INFO 14:10:37,707 Node /10.0.18.99 has restarted, now UP again
> INFO 14:10:37,708 Checking remote schema before delivering hints
> INFO 14:10:37,708 Sleeping 45506ms to stagger hint delivery
> INFO 14:10:37,709 Node /10.0.18.99 state jump to normal
> INFO 14:11:23,215 Started hinted handoff for endpoint /10.0.18.99
> ERROR 14:11:23,884 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:23,885 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> The 10.0.18.99 server log:
> {noformat}
> INFO 14:10:37,691 Binding thrift service to /0.0.0.0:9160
> INFO 14:10:37,693 Using TFastFramedTransport with a max frame size of
> 15728640 bytes.
> INFO 14:10:37,695 Listening for thrift clients...
> INFO 14:10:38,337 GC for ParNew: 954 ms, 658827608 reclaimed leaving
> 966732432 used; max is 4265607168
> INFO 14:11:27,142 Started hinted handoff for endpoint /10.0.18.129
> ERROR 14:11:27,370 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> ERROR 14:11:27,371 Fatal exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.position(Buffer.java:218)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:117)
> at org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:111)
> at org.apache.cassandra.db.HintedHandOffManager.getTableAndCFNames(HintedHandOffManager.java:237)
> at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:306)
> at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
> at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:385)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> {noformat}
> It happen durring batch_mutate test or after restart, when there are commitlogs to replay. Using current 0.7.1 from cassandra-0.7 branch.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira