You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by ruslan usifov <ru...@gmail.com> on 2011/02/21 11:43:53 UTC
java.io.IOException in CompactionExecutor
I launch clean cassandra 7.2 instalation, and after few days i look at
system.log follow error (more then 10 time):
ERROR [CompactionExecutor:1] 2011-02-19 02:56:17,965
AbstractCassandraDaemon.java (line 114) Fatal exception in thread
Thread[CompactionExecutor:1,1,main]
java.lang.RuntimeException: java.io.IOException: Unable to rename cache to
F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Unable to rename cache to
F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache
at
org.apache.cassandra.io.sstable.CacheWriter.saveCache(CacheWriter.java:85)
at
org.apache.cassandra.db.CompactionManager$9.runMayThrow(CompactionManager.java:746)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 6 more
Re: java.io.IOException in CompactionExecutor
Posted by Aaron Morton <aa...@thelastpickle.com>.
The work around is to disable saving caches, as they may not be correctly saved. However you do not have access to change the settings for the system CFs. I imagine it may only be an issue if a node is rebooted as it may load stale caches.
I think it's windows only as the function used to rename the saved cache says it has different operation on different OS's http://download.oracle.com/javase/6/docs/api/java/io/File.html#renameTo(java.io.File) and there seems to be a number of people complaining about the behavior on windows.
On my ubuntu desktop I ran a test where a user CF safes it's caches every 5 seconds and all was well.
32bit Win XP (sp3) may not fall into the list of supported OS's for cassandra, but we may confirm how it works on a more modern Windows version.
Let me when you have created the bug.
Thanks
Aaron
On 22 Feb, 2011,at 01:14 AM, ruslan usifov <ru...@gmail.com> wrote:
I have a question. How do you think this happens only on windows(on this platform i don't worry, because this is only test platform) or everywhere (linux)? And how dangerous this error, or at first time i may simply ignore it?
2011/2/21 Aaron Morton <aa...@thelastpickle.com>
Yes https://issues.apache.org/jira/browse/CASSANDRA
Thanks
Aaron
On 22 Feb, 2011,at 12:55 AM, ruslan usifov <ru...@gmail.com> wrote:
2011/2/21 Aaron Morton <aa...@thelastpickle.com>
The code creates a new .tmp file in the saved_caches directory and then renames it to a non .tmp file name, so there is nothing else with a handle open. The rename is to an existing file though.
Ruslan can you please raise a bug against 0.7.2 for this and include the platform.
You mean in JIRA?
This happens on WindowsXP (SP3) 32 bit
Re: java.io.IOException in CompactionExecutor
Posted by ruslan usifov <ru...@gmail.com>.
I have a question. How do you think this happens only on windows(on this
platform i don't worry, because this is only test platform) or everywhere
(linux)? And how dangerous this error, or at first time i may simply ignore
it?
2011/2/21 Aaron Morton <aa...@thelastpickle.com>
> Yes https://issues.apache.org/jira/browse/CASSANDRA
>
> Thanks
> <https://issues.apache.org/jira/browse/CASSANDRA>Aaron
>
>
> On 22 Feb, 2011,at 12:55 AM, ruslan usifov <ru...@gmail.com>
> wrote:
>
> 2011/2/21 Aaron Morton <aa...@thelastpickle.com>
>
>> The code creates a new .tmp file in the saved_caches directory and then
>> renames it to a non .tmp file name, so there is nothing else with a handle
>> open. The rename is to an existing file though.
>>
>> Ruslan can you please raise a bug against 0.7.2 for this and include the
>> platform.
>>
>>
> You mean in JIRA?
> This happens on WindowsXP (SP3) 32 bit
>
>
Re: java.io.IOException in CompactionExecutor
Posted by Aaron Morton <aa...@thelastpickle.com>.
Yes https://issues.apache.org/jira/browse/CASSANDRA
Thanks
Aaron
On 22 Feb, 2011,at 12:55 AM, ruslan usifov <ru...@gmail.com> wrote:
2011/2/21 Aaron Morton <aa...@thelastpickle.com>
The code creates a new .tmp file in the saved_caches directory and then renames it to a non .tmp file name, so there is nothing else with a handle open. The rename is to an existing file though.
Ruslan can you please raise a bug against 0.7.2 for this and include the platform.
You mean in JIRA?
This happens on WindowsXP (SP3) 32 bit
Re: java.io.IOException in CompactionExecutor
Posted by ruslan usifov <ru...@gmail.com>.
2011/2/21 Aaron Morton <aa...@thelastpickle.com>
> The code creates a new .tmp file in the saved_caches directory and then
> renames it to a non .tmp file name, so there is nothing else with a handle
> open. The rename is to an existing file though.
>
> Ruslan can you please raise a bug against 0.7.2 for this and include the
> platform.
>
>
You mean in JIRA?
This happens on WindowsXP (SP3) 32 bit
Re: java.io.IOException in CompactionExecutor
Posted by ruslan usifov <ru...@gmail.com>.
2011/2/21 Daniel Josefsson <da...@shazamteam.com>
> There is no antivirus program or similar running on that machine I guess?
>
> That could definitely lock the file if Cassandra is creating the .tmp file
> and then fairly shortly after tries to rename it.
>
No i haven't any antivirus installed
Re: java.io.IOException in CompactionExecutor
Posted by Daniel Josefsson <da...@shazamteam.com>.
There is no antivirus program or similar running on that machine I
guess?
That could definitely lock the file if Cassandra is creating the .tmp
file and then fairly shortly after tries to rename it.
/Daniel
On Mon, 2011-02-21 at 11:34 +0000, Aaron Morton wrote:
> The code creates a new .tmp file in the saved_caches directory and
> then renames it to a non .tmp file name, so there is nothing else with
> a handle open. The rename is to an existing file though.
>
>
> Ruslan can you please raise a bug against 0.7.2 for this and include
> the platform.
>
>
> Thanks
> Aaron
>
>
> On 22 Feb, 2011,at 12:22 AM, Norman Maurer <no...@apache.org> wrote:
>
>
> > The "problem" on windows is that it is a bit more worried about
> > rename
> > a file if the handle is still open.
> >
> > So maybe some stream not closed on the file.
> >
> > Bye,
> > Norman
> >
> >
> > 2011/2/21 Aaron Morton <aa...@thelastpickle.com>:
> > > From th F:/ I assume you are on Windows ? What version?
> > > Just did a quick test on Ubuntu 10.0.4 and it works, but the
> > File.renameTo()
> > > function used has different behavior depending on the host OS.
> > There may be
> > > some issues on
> > >
> > Window http://stackoverflow.com/questions/1000183/reliable-file-renameto-alternative-on-windows
> > > Aaron
> > >
> > >
> > > On 21 Feb, 2011,at 11:43 PM, ruslan usifov
> > <ru...@gmail.com> wrote:
> > >
> > > I launch clean cassandra 7.2 instalation, and after few days i
> > look at
> > > system.log follow error (more then 10 time):
> > >
> > >
> > > ERROR [CompactionExecutor:1] 2011-02-19 02:56:17,965
> > > AbstractCassandraDaemon.java (line 114) Fatal exception in thread
> > > Thread[CompactionExecutor:1,1,main]
> > > java.lang.RuntimeException: java.io.IOException: Unable to rename
> > cache to
> > > F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache
> > > at
> > >
> > org.apache.cassandra.utils.WrappedRunnablerun(WrappedRunnable.java:34)
> > > at
> > > java.util.concurrent.Executors
> > $RunnableAdapter.call(Executors.java:441)
> > > at java.util.concurrent.FutureTask
> > $Sync.innerRun(FutureTask.java:303)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor
> > $Worker.runTask(ThreadPoolExecutor.java:886)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor
> > $Worker.run(ThreadPoolExecutor.java:908)
> > > at java.lang.Thread.run(Thread.java:662)
> > > Caused by: java.io.IOException: Unable to rename cache to
> > > F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache
> > > at
> > >
> > org.apache.cassandra.io.sstable.CacheWriter.saveCache(CacheWriter.java:85)
> > > at
> > > org.apache.cassandra.db.CompactionManager
> > $9.runMayThrow(CompactionManager.java:746)
> > > at
> > >
> > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> > > ... 6 more
> > >
> > >
> >
--
Daniel Josefsson
Software Engineer
Shazam Entertainment Ltd
26-28 Hammersmith Grove, London W6 7HA
w: www.shazam.com
Please consider the environment before printing this document
This e-mail and its contents are strictly private and confidential. It
must not be disclosed, distributed or copied without our prior consent.
If you have received this transmission in error, please notify Shazam
Entertainment immediately on: +44 (0) 020 8742 6820 and then delete it
from your system. Please note that the information contained herein
shall additionally constitute Confidential Information for the purposes
of any NDA between the recipient/s and Shazam Entertainment. Shazam
Entertainment Limited is incorporated in England and Wales under company
number 3998831 and its registered office is at 26-28 Hammersmith Grove,
London W6 7HA.
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
Re: java.io.IOException in CompactionExecutor
Posted by Aaron Morton <aa...@thelastpickle.com>.
The code creates a new .tmp file in the saved_caches directory and then renames it to a non .tmp file name, so there is nothing else with a handle open. The rename is to an existing file though.
Ruslan can you please raise a bug against 0.7.2 for this and include the platform.
Thanks
Aaron
On 22 Feb, 2011,at 12:22 AM, Norman Maurer <no...@apache.org> wrote:
The "problem" on windows is that it is a bit more worried about rename
a file if the handle is still open..
So maybe some stream not closed on the file.
Bye,
Norman
2011/2/21 Aaron Morton <aa...@thelastpickle.com>:
> From th F:/ I assume you are on Windows ? What version?
> Just did a quick test on Ubuntu 10.0.4 and it works, but the File.renameTo()
> function used has different behavior depending on the host OS. There may be
> some issues on
> Window http://stackoverflow.com/questions/1000183/reliable-file-renameto-alternative-on-windows
> Aaron
>
>
> On 21 Feb, 2011,at 11:43 PM, ruslan usifov <ru...@gmail.com> wrote:
>
> I launch clean cassandra 7.2 instalation, and after few days i look at
> system.log follow error (more then 10 time):
>
>
> ERROR [CompactionExecutor:1] 2011-02-19 02:56:17,965
> AbstractCassandraDaemon.java (line 114) Fatal exception in thread
> Thread[CompactionExecutor:1,1,main]
> java.lang.RuntimeException: java.io.IOException: Unable to rename cache to
> F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache
> at
> org.apache.cassandra.utils.WrappedRunnablerun(WrappedRunnable.java:34)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to rename cache to
> F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache
> at
> org.apache.cassandra.io.sstableCacheWriter.saveCache(CacheWriter.java:85)
> at
> org.apache.cassandra.db.CompactionManager$9.runMayThrow(CompactionManager.java:746)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 6 more
>
>
Re: java.io.IOException in CompactionExecutor
Posted by Norman Maurer <no...@apache.org>.
The "problem" on windows is that it is a bit more worried about rename
a file if the handle is still open..
So maybe some stream not closed on the file.
Bye,
Norman
2011/2/21 Aaron Morton <aa...@thelastpickle.com>:
> From th F:/ I assume you are on Windows ? What version?
> Just did a quick test on Ubuntu 10.0.4 and it works, but the File.renameTo()
> function used has different behavior depending on the host OS. There may be
> some issues on
> Window http://stackoverflow.com/questions/1000183/reliable-file-renameto-alternative-on-windows
> Aaron
>
>
> On 21 Feb, 2011,at 11:43 PM, ruslan usifov <ru...@gmail.com> wrote:
>
> I launch clean cassandra 7.2 instalation, and after few days i look at
> system.log follow error (more then 10 time):
>
>
> ERROR [CompactionExecutor:1] 2011-02-19 02:56:17,965
> AbstractCassandraDaemon.java (line 114) Fatal exception in thread
> Thread[CompactionExecutor:1,1,main]
> java.lang.RuntimeException: java.io.IOException: Unable to rename cache to
> F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache
> at
> org.apache.cassandra.utils.WrappedRunnablerun(WrappedRunnable.java:34)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to rename cache to
> F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache
> at
> org.apache.cassandra.io.sstable.CacheWriter.saveCache(CacheWriter.java:85)
> at
> org.apache.cassandra.db.CompactionManager$9.runMayThrow(CompactionManager.java:746)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 6 more
>
>
Re: java.io.IOException in CompactionExecutor
Posted by Aaron Morton <aa...@thelastpickle.com>.
>From th F:/ I assume you are on Windows ? What version?
Just did a quick test on Ubuntu 10.0.4 and it works, but the File.renameTo() function used has different behavior depending on the host OS. There may be some issues on Window http://stackoverflow.com/questions/1000183/reliable-file-renameto-alternative-on-windows
Aaron
On 21 Feb, 2011,at 11:43 PM, ruslan usifov <ru...@gmail.com> wrote:
I launch clean cassandra 7.2 instalation, and after few days i look at system.log follow error (more then 10 time):
ERROR [CompactionExecutor:1] 2011-02-19 02:56:17,965 AbstractCassandraDaemon.java (line 114) Fatal exception in thread Thread[CompactionExecutor:1,1,main]
java.lang.RuntimeException: java.io.IOException: Unable to rename cache to F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Unable to rename cache to F:\Cassandra\7.2\saved_caches\system-LocationInfo-KeyCache
at org.apache.cassandra.io.sstable.CacheWriter.saveCache(CacheWriter.java:85)
at org.apache.cassandra.db.CompactionManager$9.runMayThrow(CompactionManager.java:746)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 6 more