You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by drvillo <f....@gmail.com> on 2007/02/15 18:02:51 UTC
Kaha store corrupted?
Hi
I'm using a kaha store to handle manual message spooling.
I've used with success for a week or so now but today I found
it unable to fetch any more message.
I can still read the size, but once I call ListContainerImpl.remove()
I get the following trace:
---
2007-02-15 17:56:24,076 [(12)-193.0.1.51] ERROR DataManagerImpl
- Looking for key 2 but not found in fileMap: {1=data-kaha-1 number = 1 ,
length = 33554371 refCount = 1, 3=data-kaha-3 number = 3 , length = 33554363
refCount = 318204, 4=data-kaha-4 number = 4 , length = 33554422 refCount =
319059, 5=data-kaha-5 number = 5 , length = 33554408 refCount = 319381,
6=data-kaha-6 number = 6 , length = 33554400 refCount = 318665,
7=data-kaha-7 number = 7 , length = 33554396 refCount = 317935,
8=data-kaha-8 number = 8 , length = 698115 refCount = 6602}
2007-02-15 17:56:24,079 [(12)-193.0.1.51] ERROR ListContainerImpl
- Failed to get value for offset=32526678, key=(-1, -1, 0), value=(2,
33554256, 99), previousItem=153, nextItem=32526729
java.io.IOException: Could not locate data file data-kaha-2
at
org.apache.activemq.kaha.impl.data.DataManagerImpl.getDataFile(DataManagerImpl.java:121)
at
org.apache.activemq.kaha.impl.data.SyncDataFileReader.readItem(SyncDataFileReader.java:60)
at
org.apache.activemq.kaha.impl.data.DataManagerImpl.readItem(DataManagerImpl.java:130)
at
org.apache.activemq.kaha.impl.container.ListContainerImpl.getValue(ListContainerImpl.java:838)
at
org.apache.activemq.kaha.impl.container.ListContainerImpl.remove(ListContainerImpl.java:527)
at
net.ripe.hstcnt.spool.KahaMessageSpooler._fetch(KahaMessageSpooler.java:77)
at
net.ripe.hstcnt.spool.KahaMessageSpooler.fetch(KahaMessageSpooler.java:70)
at
net.ripe.hstcnt.spool.SpoolerJMSBridge.bridgeRaw(SpoolerJMSBridge.java:107)
at net.ripe.hstcnt.monitor.SpoolerView.bridge(SpoolerView.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
at
com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
at
com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1424)
at
javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1262)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1357)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:786)
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
---
Needless to say there is the data-kaha-2 file:
total 341828
-rw-r--r-- 1 hostcount role 33554371 Feb 15 17:11 data-kaha-1
-rw-r--r-- 1 hostcount role 33554360 Feb 15 17:11 data-kaha-2
-rw-r--r-- 1 hostcount role 33554363 Feb 15 17:11 data-kaha-3
-rw-r--r-- 1 hostcount role 33554422 Feb 15 17:11 data-kaha-4
-rw-r--r-- 1 hostcount role 33554408 Feb 15 17:11 data-kaha-5
-rw-r--r-- 1 hostcount role 33554400 Feb 15 17:11 data-kaha-6
-rw-r--r-- 1 hostcount role 33554396 Feb 15 17:11 data-kaha-7
-rw-r--r-- 1 hostcount role 698115 Feb 15 17:11 data-kaha-8
-rw-r--r-- 1 hostcount role 114068130 Feb 15 17:11 index-kaha
but for one reason or another it's not mapped anymore in te ListContainer.
Is it possible that restarting sometimes cause some inconsistency.
I have to say that sometimes I've killed the process with -9...
Also: I tried to copy the files over to my local machine so to debug it,
but once done that it seems that I read an empty store...
Thanks a lot
Francesco
--
View this message in context: http://www.nabble.com/Kaha-store-corrupted--tf3234904s2354.html#a8989367
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Kaha store corrupted?
Posted by Albert Strasheim <fu...@gmail.com>.
Hello all
----- Original Message -----
From: "Rob Davies" <ra...@gmail.com>
To: <us...@activemq.apache.org>
Sent: Tuesday, February 20, 2007 6:44 PM
Subject: Re: Kaha store corrupted?
> could you change the prefetch for the subscriber ?
Will do. So far we haven't tweaked this value at all, so it could definately
be a problem.
Thanks for the help.
Cheers,
Albert
> On 20 Feb 2007, at 15:55, Albert Strasheim wrote:
>> Correction: we experimented with changing the consumers' acknowledge
>> mode from DUPS_OK_ACKNOWLEDGDE to AUTO_ACKNOWLEDGE, which seems to
>> have caused this problem. I will investigate some more.
Re: Kaha store corrupted?
Posted by Rob Davies <ra...@gmail.com>.
could you change the prefetch for the subscriber ?
cheers,
Rob
On 20 Feb 2007, at 15:55, Albert Strasheim wrote:
> Hello all
>
> On Tue, 20 Feb 2007, Albert Strasheim wrote:
>
>> Hey Rob
>>
>> On Tue, 20 Feb 2007, Rob Davies wrote:
>>
>>> Hey - I'm just finishing up some testing around some fixes -
>>> hopefully be finished today
>
> <snip>
>
>> This afternoon I tried to reproduce this with revision 509600, but
>> now
>> our app runs only for a short time before starting to throw
>> OutOfMemoryErrors (without any code having been changed on our side).
>
> Correction: we experimented with changing the consumers' acknowledge
> mode from DUPS_OK_ACKNOWLEDGDE to AUTO_ACKNOWLEDGE, which seems to
> have caused this problem. I will investigate some more.
>
> Cheers,
>
> Albert
Re: Kaha store corrupted?
Posted by Rob Davies <ra...@gmail.com>.
Albert,
there's a test case I added to SVN (supplied by Tom - thx Tom!) -
https://svn.apache.org/repos/asf/activemq/trunk/activemq-core/src/
test/java/org/apache/activemq/bugs/JmsDurableTopicSlowReceiveTest.java
setting the prefetch is important - the defaults are currently only
limited by number, not by payload size - so if not set to a low
number for large messages can result in either broker or client
running out of memory
cheers,
Rob
On 20 Feb 2007, at 15:55, Albert Strasheim wrote:
> Hello all
>
> On Tue, 20 Feb 2007, Albert Strasheim wrote:
>
>> Hey Rob
>>
>> On Tue, 20 Feb 2007, Rob Davies wrote:
>>
>>> Hey - I'm just finishing up some testing around some fixes -
>>> hopefully be finished today
>
> <snip>
>
>> This afternoon I tried to reproduce this with revision 509600, but
>> now
>> our app runs only for a short time before starting to throw
>> OutOfMemoryErrors (without any code having been changed on our side).
>
> Correction: we experimented with changing the consumers' acknowledge
> mode from DUPS_OK_ACKNOWLEDGDE to AUTO_ACKNOWLEDGE, which seems to
> have caused this problem. I will investigate some more.
>
> Cheers,
>
> Albert
Re: Kaha store corrupted?
Posted by Albert Strasheim <13...@sun.ac.za>.
Hello all
On Tue, 20 Feb 2007, Albert Strasheim wrote:
> Hey Rob
>
> On Tue, 20 Feb 2007, Rob Davies wrote:
>
> > Hey - I'm just finishing up some testing around some fixes -
> > hopefully be finished today
<snip>
> This afternoon I tried to reproduce this with revision 509600, but now
> our app runs only for a short time before starting to throw
> OutOfMemoryErrors (without any code having been changed on our side).
Correction: we experimented with changing the consumers' acknowledge
mode from DUPS_OK_ACKNOWLEDGDE to AUTO_ACKNOWLEDGE, which seems to
have caused this problem. I will investigate some more.
Cheers,
Albert
Re: Kaha store corrupted?
Posted by Albert Strasheim <13...@sun.ac.za>.
Hey Rob
On Tue, 20 Feb 2007, Rob Davies wrote:
> Hey - I'm just finishing up some testing around some fixes -
> hopefully be finished today
First off, thanks for all your hard work. I can see our app steadily
becoming more stable with each checkin from you. :-)
While the VMPendingSubscriberMessageStoragePolicy problems are being
sorted out, we are running our unit tests using defaults (i.e. Kaha
persistence kicks in when our producer goes faster than our consumer,
which is non-durable on a topic).
This morning with revision 509457 our application would run, and
messages would keep coming in, but after running for about 10 minutes we
would see messages arriving out of order (presumably they were being
read from the Kaha store in the wrong order, if that's possible?).
This afternoon I tried to reproduce this with revision 509600, but now
our app runs only for a short time before starting to throw
OutOfMemoryErrors (without any code having been changed on our side).
I'm trying to reproduce these problems in separate test cases, but it's
providing rather difficult.
Cheers,
Albert
Re: Kaha store corrupted?
Posted by Rob Davies <ra...@gmail.com>.
Hey - I'm just finishing up some testing around some fixes -
hopefully be finished today
cheers.
Rob
On 20 Feb 2007, at 08:45, Albert Strasheim wrote:
> Hey Rob
>
> On Sat, 17 Feb 2007, Rob Davies wrote:
>
>> Hi Albert,
>>
>> if you get a chance - could you try with the latest from SVN - to see
>> if you still have the issue with Kaha ?
>>
>> cheers,
>>
>> Rob
>
> Still running into UsageManager issues in my attempts to test the Kaha
> stuff. There are at least two test cases for the UsageManager hangs
> that
> could help. They are attached to these tickets:
>
> https://issues.apache.org/activemq/browse/AMQ-1136
> https://issues.apache.org/activemq/browse/AMQ-1148
>
> In AMQ-1148's test, it seems it is the transport thread that is
> getting
> stuck.
>
> Thanks!
>
> Cheers,
>
> Albert
Re: Kaha store corrupted?
Posted by Albert Strasheim <13...@sun.ac.za>.
Hey Rob
On Sat, 17 Feb 2007, Rob Davies wrote:
> Hi Albert,
>
> if you get a chance - could you try with the latest from SVN - to see
> if you still have the issue with Kaha ?
>
> cheers,
>
> Rob
Still running into UsageManager issues in my attempts to test the Kaha
stuff. There are at least two test cases for the UsageManager hangs that
could help. They are attached to these tickets:
https://issues.apache.org/activemq/browse/AMQ-1136
https://issues.apache.org/activemq/browse/AMQ-1148
In AMQ-1148's test, it seems it is the transport thread that is getting
stuck.
Thanks!
Cheers,
Albert
Re: Kaha store corrupted?
Posted by Rob Davies <ra...@gmail.com>.
Hi Albert,
if you get a chance - could you try with the latest from SVN - to see
if you still have the issue with Kaha ?
cheers,
Rob
On 17 Feb 2007, at 01:36, Albert Strasheim wrote:
> Hey Rob
>
> Most attempts to reproduce this issue has run me into hangs on
> UsageManager.waitForSpace. With the code as it is currently it is
> very hard to reproduce the Kaha problems without running into
> UsageManager problems.
>
> Maybe if we can do something about AMQ-1148 (and the issues that
> might come out of fixing that), I'll have a better chance at making
> a reproducable test case for the Kaha issue. My last message in the
> thread discussing AMQ-1148 was here:
>
> http://www.nabble.com/Fast-producer%2C-slow-consumer-with-spooling-
> to-disk--tf3123868.html#a8777610
>
> Cheers,
>
> Albert
>
> ----- Original Message ----- From: "Rob Davies" <ra...@gmail.com>
> To: <us...@activemq.apache.org>
> Sent: Saturday, February 17, 2007 1:55 AM
> Subject: Re: Kaha store corrupted?
>
>
>> I wonder if there's a way you could produce a junit test case -
>> to help narrow down the problem ?
>>
>> cheers,
>>
>> Rob
>> On 15 Feb 2007, at 20:01, Albert Strasheim wrote:
>>
>>> I reported the same (or a very similar) issue here:
>>>
>>> https://issues.apache.org/activemq/browse/AMQ-1082
>>>
>>> Maybe you could add some pertinent comments about your setup there.
>>> Might be that I got the wrong issue though.
>>>
>>> Cheers,
>>>
>>> Albert
>
Re: Kaha store corrupted?
Posted by Rob Davies <ra...@gmail.com>.
ah - ok - there's some tweaks going in to the usage manager at the
moment, which should result in behavior hopefully closer to what you
want
I'll give you a heads up when they're done
cheers,
Rob
On 17 Feb 2007, at 01:36, Albert Strasheim wrote:
> Hey Rob
>
> Most attempts to reproduce this issue has run me into hangs on
> UsageManager.waitForSpace. With the code as it is currently it is
> very hard to reproduce the Kaha problems without running into
> UsageManager problems.
>
> Maybe if we can do something about AMQ-1148 (and the issues that
> might come out of fixing that), I'll have a better chance at making
> a reproducable test case for the Kaha issue. My last message in the
> thread discussing AMQ-1148 was here:
>
> http://www.nabble.com/Fast-producer%2C-slow-consumer-with-spooling-
> to-disk--tf3123868.html#a8777610
>
> Cheers,
>
> Albert
>
> ----- Original Message ----- From: "Rob Davies" <ra...@gmail.com>
> To: <us...@activemq.apache.org>
> Sent: Saturday, February 17, 2007 1:55 AM
> Subject: Re: Kaha store corrupted?
>
>
>> I wonder if there's a way you could produce a junit test case -
>> to help narrow down the problem ?
>>
>> cheers,
>>
>> Rob
>> On 15 Feb 2007, at 20:01, Albert Strasheim wrote:
>>
>>> I reported the same (or a very similar) issue here:
>>>
>>> https://issues.apache.org/activemq/browse/AMQ-1082
>>>
>>> Maybe you could add some pertinent comments about your setup there.
>>> Might be that I got the wrong issue though.
>>>
>>> Cheers,
>>>
>>> Albert
>
Re: Kaha store corrupted?
Posted by Albert Strasheim <fu...@gmail.com>.
Hey Rob
Most attempts to reproduce this issue has run me into hangs on
UsageManager.waitForSpace. With the code as it is currently it is very hard
to reproduce the Kaha problems without running into UsageManager problems.
Maybe if we can do something about AMQ-1148 (and the issues that might come
out of fixing that), I'll have a better chance at making a reproducable test
case for the Kaha issue. My last message in the thread discussing AMQ-1148
was here:
http://www.nabble.com/Fast-producer%2C-slow-consumer-with-spooling-to-disk--tf3123868.html#a8777610
Cheers,
Albert
----- Original Message -----
From: "Rob Davies" <ra...@gmail.com>
To: <us...@activemq.apache.org>
Sent: Saturday, February 17, 2007 1:55 AM
Subject: Re: Kaha store corrupted?
>I wonder if there's a way you could produce a junit test case - to help
>narrow down the problem ?
>
> cheers,
>
> Rob
> On 15 Feb 2007, at 20:01, Albert Strasheim wrote:
>
>> I reported the same (or a very similar) issue here:
>>
>> https://issues.apache.org/activemq/browse/AMQ-1082
>>
>> Maybe you could add some pertinent comments about your setup there.
>> Might be that I got the wrong issue though.
>>
>> Cheers,
>>
>> Albert
Re: Kaha store corrupted?
Posted by Rob Davies <ra...@gmail.com>.
I wonder if there's a way you could produce a junit test case - to
help narrow down the problem ?
cheers,
Rob
On 15 Feb 2007, at 20:01, Albert Strasheim wrote:
> I reported the same (or a very similar) issue here:
>
> https://issues.apache.org/activemq/browse/AMQ-1082
>
> Maybe you could add some pertinent comments about your setup there.
> Might be that I got the wrong issue though.
>
> Cheers,
>
> Albert
>
> On Thu, 15 Feb 2007, drvillo wrote:
>
> <snip>
>
>> I'm using a kaha store to handle manual message spooling.
>> I've used with success for a week or so now but today I found
>> it unable to fetch any more message.
>> I can still read the size, but once I call ListContainerImpl.remove()
>> I get the following trace:
>
> <snip>
>
>> Needless to say there is the data-kaha-2 file:
>>
>> total 341828
>> -rw-r--r-- 1 hostcount role 33554371 Feb 15 17:11 data-kaha-1
>> -rw-r--r-- 1 hostcount role 33554360 Feb 15 17:11 data-kaha-2
>> -rw-r--r-- 1 hostcount role 33554363 Feb 15 17:11 data-kaha-3
>> -rw-r--r-- 1 hostcount role 33554422 Feb 15 17:11 data-kaha-4
>> -rw-r--r-- 1 hostcount role 33554408 Feb 15 17:11 data-kaha-5
>> -rw-r--r-- 1 hostcount role 33554400 Feb 15 17:11 data-kaha-6
>> -rw-r--r-- 1 hostcount role 33554396 Feb 15 17:11 data-kaha-7
>> -rw-r--r-- 1 hostcount role 698115 Feb 15 17:11 data-kaha-8
>> -rw-r--r-- 1 hostcount role 114068130 Feb 15 17:11 index-kaha
>>
>> but for one reason or another it's not mapped anymore in te
>> ListContainer.
>> Is it possible that restarting sometimes cause some inconsistency.
>> I have to say that sometimes I've killed the process with -9...
>>
>> Also: I tried to copy the files over to my local machine so to
>> debug it,
>> but once done that it seems that I read an empty store...
>>
>> Thanks a lot
>> Francesco
Re: Kaha store corrupted?
Posted by drvillo <f....@gmail.com>.
Hi
it seems somehow related but you say your data file was deleted,
whereas mine is definitely there.
Somehow the DataManagerImpl.fileMap doesn't contain data-kaha-2,
even if looking at the code
<code>
public DataManagerImpl(File dir, final String name){
this.dir=dir;
this.name=name;
dataFilePrefix = NAME_PREFIX+name+"-";
// build up list of current dataFiles
File[] files=dir.listFiles(new FilenameFilter(){
public boolean accept(File dir,String n){
return dir.equals(dir)&&n.startsWith(dataFilePrefix);
}
});
if(files!=null){
for(int i=0;i<files.length;i++){
File file=files[i];
String n=file.getName();
String
numStr=n.substring(dataFilePrefix.length(),n.length());
int num=Integer.parseInt(numStr);
DataFile dataFile=new DataFile(file,num);
fileMap.put(dataFile.getNumber(),dataFile);
if(currentWriteFile==null||currentWriteFile.getNumber().intValue()<num){
currentWriteFile=dataFile;
}
}
}
}
</code>
it seems it just should load it as the other ones...
Anyone knows when AMQ 4.2 will be released? I understand Kaha is supposed
to be production ready with that...
Cheers
Francesco
Albert Strasheim wrote:
>
> I reported the same (or a very similar) issue here:
>
> https://issues.apache.org/activemq/browse/AMQ-1082
>
> Maybe you could add some pertinent comments about your setup there.
> Might be that I got the wrong issue though.
>
> Cheers,
>
> Albert
>
> On Thu, 15 Feb 2007, drvillo wrote:
>
> <snip>
>
>> I'm using a kaha store to handle manual message spooling.
>> I've used with success for a week or so now but today I found
>> it unable to fetch any more message.
>> I can still read the size, but once I call ListContainerImpl.remove()
>> I get the following trace:
>
> <snip>
>
>> Needless to say there is the data-kaha-2 file:
>>
>> total 341828
>> -rw-r--r-- 1 hostcount role 33554371 Feb 15 17:11 data-kaha-1
>> -rw-r--r-- 1 hostcount role 33554360 Feb 15 17:11 data-kaha-2
>> -rw-r--r-- 1 hostcount role 33554363 Feb 15 17:11 data-kaha-3
>> -rw-r--r-- 1 hostcount role 33554422 Feb 15 17:11 data-kaha-4
>> -rw-r--r-- 1 hostcount role 33554408 Feb 15 17:11 data-kaha-5
>> -rw-r--r-- 1 hostcount role 33554400 Feb 15 17:11 data-kaha-6
>> -rw-r--r-- 1 hostcount role 33554396 Feb 15 17:11 data-kaha-7
>> -rw-r--r-- 1 hostcount role 698115 Feb 15 17:11 data-kaha-8
>> -rw-r--r-- 1 hostcount role 114068130 Feb 15 17:11 index-kaha
>>
>> but for one reason or another it's not mapped anymore in te
>> ListContainer.
>> Is it possible that restarting sometimes cause some inconsistency.
>> I have to say that sometimes I've killed the process with -9...
>>
>> Also: I tried to copy the files over to my local machine so to debug it,
>> but once done that it seems that I read an empty store...
>>
>> Thanks a lot
>> Francesco
>
>
--
View this message in context: http://www.nabble.com/Kaha-store-corrupted--tf3234904s2354.html#a9000955
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Kaha store corrupted?
Posted by Albert Strasheim <13...@sun.ac.za>.
I reported the same (or a very similar) issue here:
https://issues.apache.org/activemq/browse/AMQ-1082
Maybe you could add some pertinent comments about your setup there.
Might be that I got the wrong issue though.
Cheers,
Albert
On Thu, 15 Feb 2007, drvillo wrote:
<snip>
> I'm using a kaha store to handle manual message spooling.
> I've used with success for a week or so now but today I found
> it unable to fetch any more message.
> I can still read the size, but once I call ListContainerImpl.remove()
> I get the following trace:
<snip>
> Needless to say there is the data-kaha-2 file:
>
> total 341828
> -rw-r--r-- 1 hostcount role 33554371 Feb 15 17:11 data-kaha-1
> -rw-r--r-- 1 hostcount role 33554360 Feb 15 17:11 data-kaha-2
> -rw-r--r-- 1 hostcount role 33554363 Feb 15 17:11 data-kaha-3
> -rw-r--r-- 1 hostcount role 33554422 Feb 15 17:11 data-kaha-4
> -rw-r--r-- 1 hostcount role 33554408 Feb 15 17:11 data-kaha-5
> -rw-r--r-- 1 hostcount role 33554400 Feb 15 17:11 data-kaha-6
> -rw-r--r-- 1 hostcount role 33554396 Feb 15 17:11 data-kaha-7
> -rw-r--r-- 1 hostcount role 698115 Feb 15 17:11 data-kaha-8
> -rw-r--r-- 1 hostcount role 114068130 Feb 15 17:11 index-kaha
>
> but for one reason or another it's not mapped anymore in te ListContainer.
> Is it possible that restarting sometimes cause some inconsistency.
> I have to say that sometimes I've killed the process with -9...
>
> Also: I tried to copy the files over to my local machine so to debug it,
> but once done that it seems that I read an empty store...
>
> Thanks a lot
> Francesco