You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brandon Williams (JIRA)" <ji...@apache.org> on 2010/01/07 18:45:54 UTC
[jira] Created: (CASSANDRA-681) Error deleting files during
bootstrap
Error deleting files during bootstrap
-------------------------------------
Key: CASSANDRA-681
URL: https://issues.apache.org/jira/browse/CASSANDRA-681
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.9
Environment: debian lenny amd64 OpenJDK 64-Bit Server VM (build 1.6.0_0-b11, mixed mode)
Reporter: Brandon Williams
Fix For: 0.9
I started a 3 node cluster and proceeded to bootstrap a 4th node. On one of the existing nodes I began to see tracebacks like this:
ERROR - Error in executor futuretask
java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
at java.util.concurrent.FutureTask.get(FutureTask.java:111)
at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:53)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1118)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:13)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
... 2 more
Caused by: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
at org.apache.cassandra.io.DeletionService$2.runMayThrow(DeletionService.java:45)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:9)
... 6 more
For various data, index, and filter files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-681) Error deleting files during
bootstrap
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-681:
-------------------------------------
Component/s: Core
Fix Version/s: (was: 0.9)
0.5
Priority: Minor (was: Major)
Affects Version/s: (was: 0.9)
0.5
> Error deleting files during bootstrap
> -------------------------------------
>
> Key: CASSANDRA-681
> URL: https://issues.apache.org/jira/browse/CASSANDRA-681
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: debian lenny amd64 OpenJDK 64-Bit Server VM (build 1.6.0_0-b11, mixed mode)
> Reporter: Brandon Williams
> Priority: Minor
> Fix For: 0.5
>
>
> I started a 3 node cluster and proceeded to bootstrap a 4th node. On one of the existing nodes I began to see tracebacks like this:
> ERROR - Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:53)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1118)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> Caused by: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:13)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> ... 2 more
> Caused by: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at org.apache.cassandra.io.DeletionService$2.runMayThrow(DeletionService.java:45)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:9)
> ... 6 more
> For various data, index, and filter files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-681) Error deleting files during
bootstrap
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-681:
-------------------------------------
Attachment: 681.patch
it looks like the problem was that Streaming.transferSSTables and StreamManager.finish were both attempting to delete the streamed file. this patch removes the one from transferSSTables.
> Error deleting files during bootstrap
> -------------------------------------
>
> Key: CASSANDRA-681
> URL: https://issues.apache.org/jira/browse/CASSANDRA-681
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: debian lenny amd64 OpenJDK 64-Bit Server VM (build 1.6.0_0-b11, mixed mode)
> Reporter: Brandon Williams
> Priority: Minor
> Fix For: 0.5
>
> Attachments: 681.patch
>
>
> I started a 3 node cluster and proceeded to bootstrap a 4th node. On one of the existing nodes I began to see tracebacks like this:
> ERROR - Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:53)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1118)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> Caused by: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:13)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> ... 2 more
> Caused by: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at org.apache.cassandra.io.DeletionService$2.runMayThrow(DeletionService.java:45)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:9)
> ... 6 more
> For various data, index, and filter files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-681) Error deleting files during
bootstrap
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799455#action_12799455 ]
Brandon Williams commented on CASSANDRA-681:
--------------------------------------------
+1, error no longer appears
> Error deleting files during bootstrap
> -------------------------------------
>
> Key: CASSANDRA-681
> URL: https://issues.apache.org/jira/browse/CASSANDRA-681
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: debian lenny amd64 OpenJDK 64-Bit Server VM (build 1.6.0_0-b11, mixed mode)
> Reporter: Brandon Williams
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 0.5
>
> Attachments: 681.patch
>
>
> I started a 3 node cluster and proceeded to bootstrap a 4th node. On one of the existing nodes I began to see tracebacks like this:
> ERROR - Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:53)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1118)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> Caused by: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:13)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> ... 2 more
> Caused by: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at org.apache.cassandra.io.DeletionService$2.runMayThrow(DeletionService.java:45)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:9)
> ... 6 more
> For various data, index, and filter files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-681) Error deleting files during
bootstrap
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798074#action_12798074 ]
Brandon Williams commented on CASSANDRA-681:
--------------------------------------------
I also received this while testing CASSANDRA-680, so it is not limited to bootstrap.
> Error deleting files during bootstrap
> -------------------------------------
>
> Key: CASSANDRA-681
> URL: https://issues.apache.org/jira/browse/CASSANDRA-681
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: debian lenny amd64 OpenJDK 64-Bit Server VM (build 1.6.0_0-b11, mixed mode)
> Reporter: Brandon Williams
> Priority: Minor
> Fix For: 0.5
>
>
> I started a 3 node cluster and proceeded to bootstrap a 4th node. On one of the existing nodes I began to see tracebacks like this:
> ERROR - Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:53)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1118)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> Caused by: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:13)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> ... 2 more
> Caused by: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at org.apache.cassandra.io.DeletionService$2.runMayThrow(DeletionService.java:45)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:9)
> ... 6 more
> For various data, index, and filter files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-681) Error deleting files during
bootstrap
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798835#action_12798835 ]
Jonathan Ellis commented on CASSANDRA-681:
------------------------------------------
were there any other errors (like the ones in CASSANDRA-657) in the logs?
> Error deleting files during bootstrap
> -------------------------------------
>
> Key: CASSANDRA-681
> URL: https://issues.apache.org/jira/browse/CASSANDRA-681
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: debian lenny amd64 OpenJDK 64-Bit Server VM (build 1.6.0_0-b11, mixed mode)
> Reporter: Brandon Williams
> Priority: Minor
> Fix For: 0.5
>
>
> I started a 3 node cluster and proceeded to bootstrap a 4th node. On one of the existing nodes I began to see tracebacks like this:
> ERROR - Error in executor futuretask
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:53)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1118)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> Caused by: java.lang.RuntimeException: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:13)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> ... 2 more
> Caused by: java.io.IOException: Unable to delete /mnt/drive3/data/Keyspace1/stream/Standard1-158-Index.db after 10 tries
> at org.apache.cassandra.io.DeletionService$2.runMayThrow(DeletionService.java:45)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:9)
> ... 6 more
> For various data, index, and filter files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.