You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by jcamus <jc...@parkeon.com> on 2013/03/08 17:46:26 UTC
Data loss for ActiveMQ 5.6.0
Hi!
Am using ActiveMq to store files to be processed. I had the following logs,
when I restart my applications, I have still the problem (no files in the
queue are consumed), the only way is to purge the activemq data directory.
But all the files are definitely lost!
2013-03-06 22:09:58,423 ERROR [ActiveMQ Journal Checkpoint Worker]
[MessageDatabase.java:331] Checkpoint failed
java.io.IOException: network path not found
at java.io.RandomAccessFile.writeBytes(Native Method) ~[na:1.6.0_26]
at java.io.RandomAccessFile.write(RandomAccessFile.java:482) ~[na:1.6.0_26]
at org.apache.kahadb.page.PageFile.writeBatch(PageFile.java:1053)
~[kahadb-5.6.0.jar:5.6.0]
at org.apache.kahadb.page.PageFile.flush(PageFile.java:506)
~[kahadb-5.6.0.jar:5.6.0]
at
org.apache.activemq.store.kahadb.MessageDatabase.checkpointUpdate(MessageDatabase.java:1358)
~[activemq-core-5.6.0.jar:5.6.0]
at
org.apache.activemq.store.kahadb.MessageDatabase$10.execute(MessageDatabase.java:814)
~[activemq-core-5.6.0.jar:5.6.0]
at org.apache.kahadb.page.Transaction.execute(Transaction.java:769)
~[kahadb-5.6.0.jar:5.6.0]
at
org.apache.activemq.store.kahadb.MessageDatabase.checkpointCleanup(MessageDatabase.java:812)
~[activemq-core-5.6.0.jar:5.6.0]
at
org.apache.activemq.store.kahadb.MessageDatabase$3.run(MessageDatabase.java:324)
~[activemq-core-5.6.0.jar:5.6.0]
2013-03-06 22:09:58,424 INFO [ActiveMQ Journal Checkpoint Worker]
[DefaultIOExceptionHandler.java:122] Stopping the broker due to IO
exception, java.io.IOException: network path not found
java.io.IOException: network path not found
at java.io.RandomAccessFile.writeBytes(Native Method) ~[na:1.6.0_26]
at java.io.RandomAccessFile.write(RandomAccessFile.java:482) ~[na:1.6.0_26]
at org.apache.kahadb.page.PageFile.writeBatch(PageFile.java:1053)
~[kahadb-5.6.0.jar:5.6.0]
at org.apache.kahadb.page.PageFile.flush(PageFile.java:506)
~[kahadb-5.6.0.jar:5.6.0]
at
org.apache.activemq.store.kahadb.MessageDatabase.checkpointUpdate(MessageDatabase.java:1358)
~[activemq-core-5.6.0.jar:5.6.0]
at
org.apache.activemq.store.kahadb.MessageDatabase$10.execute(MessageDatabase.java:814)
~[activemq-core-5.6.0.jar:5.6.0]
at org.apache.kahadb.page.Transaction.execute(Transaction.java:769)
~[kahadb-5.6.0.jar:5.6.0]
at
org.apache.activemq.store.kahadb.MessageDatabase.checkpointCleanup(MessageDatabase.java:812)
~[activemq-core-5.6.0.jar:5.6.0]
at
org.apache.activemq.store.kahadb.MessageDatabase$3.run(MessageDatabase.java:324)
~[activemq-core-5.6.0.jar:5.6.0]
2013-03-06 22:09:58,429 INFO [Stopping the broker due to IO exception]
[BrokerService.java:605] ActiveMQ Message Broker (localhost,
ID:PKN-TRANS3-57347-1362414850324-0:1) is shutting down
2013-03-06 22:09:59,685 INFO [Stopping the broker due to IO exception]
[TransportConnector.java:307] Connector openwire Stopped
2013-03-06 22:10:00,061 INFO [Stopping the broker due to IO exception]
[TransportConnector.java:307] Connector vm://localhost Stopped
2013-03-06 22:10:00,072 INFO [Stopping the broker due to IO exception]
[PListStore.java:356] PListStore:[X:\Esb\Esb3\activemq\localhost\tmp_storage
] stopped
2013-03-06 22:10:00,072 INFO [Stopping the broker due to IO exception]
[KahaDBStore.java:206] Stopping async queue tasks
2013-03-06 22:10:00,073 INFO [Stopping the broker due to IO exception]
[KahaDBStore.java:220] Stopping async topic tasks
2013-03-06 22:10:00,073 INFO [Stopping the broker due to IO exception]
[KahaDBStore.java:246] Stopped KahaDB
2013-03-06 22:10:00,078 INFO [Stopping the broker due to IO exception]
[MessageDatabase.java:304] KahaDB: Recovering checkpoint thread after death
2013-03-06 22:10:00,084 ERROR [Stopping the broker due to IO exception]
[ServiceStopper.java:96] Could not stop service:
KahaDB:[X:\Esb\Esb3\activemq\data\kahadb]. Reason: java.io.IOException:
network path not found
2013-03-06 22:10:43,459 INFO [http-thread-pool-8181(1)]
[PListStore.java:331]
PListStore:[D:\glassfish\glassfish\domains\domain4\config\activemq-data\localhost\tmp_storage
] started
2013-03-06 22:10:43,460 INFO [http-thread-pool-8181(1)]
[BrokerService.java:521] Using Persistence Adapter:
KahaDBPersistenceAdapter[D:\glassfish\glassfish\domains\domain4\config\activemq-data\localhost\KahaDB]
2013-03-06 22:10:43,462 INFO [JMX connector] [ManagementContext.java:101]
JMX consoles can connect to
service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
2013-03-06 22:10:43,570 INFO [http-thread-pool-8181(1)]
[BrokerService.java:799] ActiveMQ 5.6.0 JMS Message Broker (localhost) is
starting
2013-03-06 22:10:43,571 INFO [http-thread-pool-8181(1)]
[BrokerService.java:801] For help or more information please see:
http://activemq.apache.org/
2013-03-06 22:10:43,573 INFO [http-thread-pool-8181(1)]
[BrokerService.java:564] ActiveMQ JMS Message Broker (localhost,
ID:PKN-TRANS3-57347-1362414850324-0:2) started
2013-03-06 22:10:43,575 WARN [http-thread-pool-8181(1)]
[BrokerService.java:1747] Store limit is 100000 mb, whilst the data
directory:
D:\glassfish\glassfish\domains\domain4\config\activemq-data\localhost\KahaDB
only has 91556 mb of usable space
2013-03-06 22:10:43,577 INFO [http-thread-pool-8181(1)]
[TransportConnector.java:265] Connector vm://localhost Started
Please HELP ! This problem is very critical and urgent for us !
--
View this message in context: http://activemq.2283324.n4.nabble.com/Data-loss-for-ActiveMQ-5-6-0-tp4664521.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Data loss for ActiveMQ 5.6.0
Posted by jcamus <jc...@parkeon.com>.
Thanks for your answer. But how to explain that data is totally lost? The
only way to have activemq working again normally is to purge the data
directory. But unfortunately all data is lost. Activemq seems not detecting
that there is a problem and it is still accepting new message to enqueue,
but unable to process them properly. I guess we have a problem with our
network, or our windows sever, but I am totally astonished not to have
Activemq refusing each file to be enqueued.
Regards.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Data-loss-with-ActiveMQ-5-6-0-tp4664521p4664564.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Data loss for ActiveMQ 5.6.0
Posted by Christian Posta <ch...@gmail.com>.
The checkpoint process is trying to write the index but it's having some
issue writing to your networked drive:
java.io.IOException: network path not found
On Fri, Mar 8, 2013 at 9:46 AM, jcamus <jc...@parkeon.com> wrote:
> Hi!
> Am using ActiveMq to store files to be processed. I had the following logs,
> when I restart my applications, I have still the problem (no files in the
> queue are consumed), the only way is to purge the activemq data directory.
> But all the files are definitely lost!
> 2013-03-06 22:09:58,423 ERROR [ActiveMQ Journal Checkpoint Worker]
> [MessageDatabase.java:331] Checkpoint failed
> java.io.IOException: network path not found
> at java.io.RandomAccessFile.writeBytes(Native Method)
> ~[na:1.6.0_26]
> at java.io.RandomAccessFile.write(RandomAccessFile.java:482)
> ~[na:1.6.0_26]
> at org.apache.kahadb.page.PageFile.writeBatch(PageFile.java:1053)
> ~[kahadb-5.6.0.jar:5.6.0]
> at org.apache.kahadb.page.PageFile.flush(PageFile.java:506)
> ~[kahadb-5.6.0.jar:5.6.0]
> at
>
> org.apache.activemq.store.kahadb.MessageDatabase.checkpointUpdate(MessageDatabase.java:1358)
> ~[activemq-core-5.6.0.jar:5.6.0]
> at
>
> org.apache.activemq.store.kahadb.MessageDatabase$10.execute(MessageDatabase.java:814)
> ~[activemq-core-5.6.0.jar:5.6.0]
> at org.apache.kahadb.page.Transaction.execute(Transaction.java:769)
> ~[kahadb-5.6.0.jar:5.6.0]
> at
>
> org.apache.activemq.store.kahadb.MessageDatabase.checkpointCleanup(MessageDatabase.java:812)
> ~[activemq-core-5.6.0.jar:5.6.0]
> at
>
> org.apache.activemq.store.kahadb.MessageDatabase$3.run(MessageDatabase.java:324)
> ~[activemq-core-5.6.0.jar:5.6.0]
> 2013-03-06 22:09:58,424 INFO [ActiveMQ Journal Checkpoint Worker]
> [DefaultIOExceptionHandler.java:122] Stopping the broker due to IO
> exception, java.io.IOException: network path not found
> java.io.IOException: network path not found
> at java.io.RandomAccessFile.writeBytes(Native Method)
> ~[na:1.6.0_26]
> at java.io.RandomAccessFile.write(RandomAccessFile.java:482)
> ~[na:1.6.0_26]
> at org.apache.kahadb.page.PageFile.writeBatch(PageFile.java:1053)
> ~[kahadb-5.6.0.jar:5.6.0]
> at org.apache.kahadb.page.PageFile.flush(PageFile.java:506)
> ~[kahadb-5.6.0.jar:5.6.0]
> at
>
> org.apache.activemq.store.kahadb.MessageDatabase.checkpointUpdate(MessageDatabase.java:1358)
> ~[activemq-core-5.6.0.jar:5.6.0]
> at
>
> org.apache.activemq.store.kahadb.MessageDatabase$10.execute(MessageDatabase.java:814)
> ~[activemq-core-5.6.0.jar:5.6.0]
> at org.apache.kahadb.page.Transaction.execute(Transaction.java:769)
> ~[kahadb-5.6.0.jar:5.6.0]
> at
>
> org.apache.activemq.store.kahadb.MessageDatabase.checkpointCleanup(MessageDatabase.java:812)
> ~[activemq-core-5.6.0.jar:5.6.0]
> at
>
> org.apache.activemq.store.kahadb.MessageDatabase$3.run(MessageDatabase.java:324)
> ~[activemq-core-5.6.0.jar:5.6.0]
> 2013-03-06 22:09:58,429 INFO [Stopping the broker due to IO exception]
> [BrokerService.java:605] ActiveMQ Message Broker (localhost,
> ID:PKN-TRANS3-57347-1362414850324-0:1) is shutting down
> 2013-03-06 22:09:59,685 INFO [Stopping the broker due to IO exception]
> [TransportConnector.java:307] Connector openwire Stopped
> 2013-03-06 22:10:00,061 INFO [Stopping the broker due to IO exception]
> [TransportConnector.java:307] Connector vm://localhost Stopped
> 2013-03-06 22:10:00,072 INFO [Stopping the broker due to IO exception]
> [PListStore.java:356]
> PListStore:[X:\Esb\Esb3\activemq\localhost\tmp_storage
> ] stopped
> 2013-03-06 22:10:00,072 INFO [Stopping the broker due to IO exception]
> [KahaDBStore.java:206] Stopping async queue tasks
> 2013-03-06 22:10:00,073 INFO [Stopping the broker due to IO exception]
> [KahaDBStore.java:220] Stopping async topic tasks
> 2013-03-06 22:10:00,073 INFO [Stopping the broker due to IO exception]
> [KahaDBStore.java:246] Stopped KahaDB
> 2013-03-06 22:10:00,078 INFO [Stopping the broker due to IO exception]
> [MessageDatabase.java:304] KahaDB: Recovering checkpoint thread after death
> 2013-03-06 22:10:00,084 ERROR [Stopping the broker due to IO exception]
> [ServiceStopper.java:96] Could not stop service:
> KahaDB:[X:\Esb\Esb3\activemq\data\kahadb]. Reason: java.io.IOException:
> network path not found
> 2013-03-06 22:10:43,459 INFO [http-thread-pool-8181(1)]
> [PListStore.java:331]
>
> PListStore:[D:\glassfish\glassfish\domains\domain4\config\activemq-data\localhost\tmp_storage
> ] started
> 2013-03-06 22:10:43,460 INFO [http-thread-pool-8181(1)]
> [BrokerService.java:521] Using Persistence Adapter:
>
> KahaDBPersistenceAdapter[D:\glassfish\glassfish\domains\domain4\config\activemq-data\localhost\KahaDB]
> 2013-03-06 22:10:43,462 INFO [JMX connector] [ManagementContext.java:101]
> JMX consoles can connect to
> service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
> 2013-03-06 22:10:43,570 INFO [http-thread-pool-8181(1)]
> [BrokerService.java:799] ActiveMQ 5.6.0 JMS Message Broker (localhost) is
> starting
> 2013-03-06 22:10:43,571 INFO [http-thread-pool-8181(1)]
> [BrokerService.java:801] For help or more information please see:
> http://activemq.apache.org/
> 2013-03-06 22:10:43,573 INFO [http-thread-pool-8181(1)]
> [BrokerService.java:564] ActiveMQ JMS Message Broker (localhost,
> ID:PKN-TRANS3-57347-1362414850324-0:2) started
> 2013-03-06 22:10:43,575 WARN [http-thread-pool-8181(1)]
> [BrokerService.java:1747] Store limit is 100000 mb, whilst the data
> directory:
>
> D:\glassfish\glassfish\domains\domain4\config\activemq-data\localhost\KahaDB
> only has 91556 mb of usable space
> 2013-03-06 22:10:43,577 INFO [http-thread-pool-8181(1)]
> [TransportConnector.java:265] Connector vm://localhost Started
>
> Please HELP ! This problem is very critical and urgent for us !
>
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Data-loss-for-ActiveMQ-5-6-0-tp4664521.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta