You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Keith W <ke...@gmail.com> on 2016/07/26 12:34:09 UTC
Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
Hi Adel,
Thanks for reporting, it is certainly not normal.
Could you raise a Jira and attach a minimal qpid.log, scrubbed of any
information you deem confidential (IP addresses etc)?
I am interested to see all the log statements produced by the Broker
from when it started to when it failed in the above way. My best
guess is something else has gone wrong prior and then there is this
second failure producing the unexpected shutdown.
I am not aware of anything pertinent changing between 6.0.1 and 6.0.4
which might have a bearing here, but you should consider upgrading to
6.0.4 anyway as there have been a number of important defect fixes
including security fixes.
Kind regards Keith.
On 26 July 2016 at 13:01, Adel Boutros <ad...@live.com> wrote:
> Hello,
>
> Randomly, I have the below exception on the console when launching the broker. Is this expected?
> I am using Berkley DB jar "je-5.0.104.jar" as described here https://qpid.apache.org/releases/qpid-java-6.0.1/java-broker/book/Java-Broker-Miscellaneous-Installing-Oracle-BDB-JE.html.
>
> ########################################################################
> #
> # Unhandled Exception java.lang.IllegalStateException: Message store is not open in Thread virtualhost-default-pool-2
> #
> # Exiting
> #
> ########################################################################
> java.lang.IllegalStateException: Message store is not open
> at org.apache.qpid.server.store.berkeleydb.BDBMessageStore.checkMessageStoreOpen(BDBMessageStore.java:154)
> at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.remove(AbstractBDBMessageStore.java:1202)
> at org.apache.qpid.server.message.AbstractServerMessageImpl.decrementReference(AbstractServerMessageImpl.java:110)
> at org.apache.qpid.server.message.AbstractServerMessageImpl.access$500(AbstractServerMessageImpl.java:38)
> at org.apache.qpid.server.message.AbstractServerMessageImpl$Reference.release(AbstractServerMessageImpl.java:308)
> at org.apache.qpid.server.queue.QueueEntryImpl.dispose(QueueEntryImpl.java:510)
> at org.apache.qpid.server.queue.QueueEntryImpl.delete(QueueEntryImpl.java:524)
> at org.apache.qpid.server.queue.AbstractQueue$13.postCommit(AbstractQueue.java:1903)
> at org.apache.qpid.server.txn.AutoCommitTransaction.dequeue(AutoCommitTransaction.java:90)
> at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1897)
> at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1892)
> at org.apache.qpid.server.queue.AbstractQueue.checkMessageStatus(AbstractQueue.java:2490)
> at org.apache.qpid.server.virtualhost.AbstractVirtualHost$VirtualHostHouseKeepingTask.execute(AbstractVirtualHost.java:1225)
> at org.apache.qpid.server.virtualhost.HouseKeepingTask$1.run(HouseKeepingTask.java:65)
> at java.security.AccessController.doPrivileged(Native Method)
> at org.apache.qpid.server.virtualhost.HouseKeepingTask.run(HouseKeepingTask.java:60)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
> Regards,
> Adel
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: [Java Broker - 6.0.1] Unhandled exception with Berkley DB
message store
Posted by Adel Boutros <ad...@live.com>.
Indeed, what you are saying is correct.
Actually in my test, I have a Dead Letter queue sort of whose messages are never consumed. This queue has a TTL set for its messages.
> From: keith.wall@gmail.com
> Date: Wed, 27 Jul 2016 08:52:50 +0100
> Subject: Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
> To: users@qpid.apache.org
>
> On 27 July 2016 at 08:42, Adel Boutros <ad...@live.com> wrote:
> > If the broker is supposed to shutdown on this exception then I confirm it is happening on the exit of the broker because my tests are completing successfully.
> > Adel
>
> Let me check I understand. You are running your tests, the tests
> completed, and you then shutdown the Broker. Sometimes the Broker
> fails during shutdown with the exception you report. Is that a
> correct summary?
>
> I think your test includes messages with TTL set and that your test
> suite may leaves messages on the Broker.
>
>
> >
> >> From: keith.wall@gmail.com
> >> Date: Wed, 27 Jul 2016 08:30:31 +0100
> >> Subject: Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
> >> To: users@qpid.apache.org
> >>
> >> On 26 July 2016 at 14:32, Adel Boutros <ad...@live.com> wrote:
> >> >
> >> > Hello Keith,Unfortunately, as the issue is random and I am restart the broker a lot of times in my tests, I don't have a qpid.log to provide. I will try to reproduce and send it.
> >>
> >> Great.
> >>
> >> > As for the update to 6.0.4, one my colleagues tried to upgrade from 6.0.0 to 6.0.4 and encountered failures at the level of our binding filters tests. So we upgraded to 6.0.1 instead. We didn't have time yet to inspect the failures yet and report them if needed.
> >>
> >> Sounds curious. If you have a repeatable regression, we'd very much
> >> appreciate a separate report.
> >>
> >> > In the meantime, I have created a Jira issue for the unhandled exception. Maybe the exception is expected but I was hoping to see it in the logs not as part of an unhandled exception.https://issues.apache.org/jira/browse/QPID-7363
> >>
> >> This type of exception is a 'should never happen' and at the moment
> >> the strategy is to shut down the whole Broker. We have plans to
> >> change this over the next major releases to be more discriminating
> >> about the units within the Broker that should go down in response to
> >> such a condition.
> >>
> >> Regards,Adel
> >> >> From: keith.wall@gmail.com
> >> >> Date: Tue, 26 Jul 2016 13:34:09 +0100
> >> >> Subject: Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
> >> >> To: users@qpid.apache.org
> >> >>
> >> >> Hi Adel,
> >> >>
> >> >> Thanks for reporting, it is certainly not normal.
> >> >>
> >> >> Could you raise a Jira and attach a minimal qpid.log, scrubbed of any
> >> >> information you deem confidential (IP addresses etc)?
> >> >> I am interested to see all the log statements produced by the Broker
> >> >> from when it started to when it failed in the above way. My best
> >> >> guess is something else has gone wrong prior and then there is this
> >> >> second failure producing the unexpected shutdown.
> >> >>
> >> >> I am not aware of anything pertinent changing between 6.0.1 and 6.0.4
> >> >> which might have a bearing here, but you should consider upgrading to
> >> >> 6.0.4 anyway as there have been a number of important defect fixes
> >> >> including security fixes.
> >> >>
> >> >> Kind regards Keith.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On 26 July 2016 at 13:01, Adel Boutros <ad...@live.com> wrote:
> >> >> > Hello,
> >> >> >
> >> >> > Randomly, I have the below exception on the console when launching the broker. Is this expected?
> >> >> > I am using Berkley DB jar "je-5.0.104.jar" as described here https://qpid.apache.org/releases/qpid-java-6.0.1/java-broker/book/Java-Broker-Miscellaneous-Installing-Oracle-BDB-JE.html.
> >> >> >
> >> >> > ########################################################################
> >> >> > #
> >> >> > # Unhandled Exception java.lang.IllegalStateException: Message store is not open in Thread virtualhost-default-pool-2
> >> >> > #
> >> >> > # Exiting
> >> >> > #
> >> >> > ########################################################################
> >> >> > java.lang.IllegalStateException: Message store is not open
> >> >> > at org.apache.qpid.server.store.berkeleydb.BDBMessageStore.checkMessageStoreOpen(BDBMessageStore.java:154)
> >> >> > at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.remove(AbstractBDBMessageStore.java:1202)
> >> >> > at org.apache.qpid.server.message.AbstractServerMessageImpl.decrementReference(AbstractServerMessageImpl.java:110)
> >> >> > at org.apache.qpid.server.message.AbstractServerMessageImpl.access$500(AbstractServerMessageImpl.java:38)
> >> >> > at org.apache.qpid.server.message.AbstractServerMessageImpl$Reference.release(AbstractServerMessageImpl.java:308)
> >> >> > at org.apache.qpid.server.queue.QueueEntryImpl.dispose(QueueEntryImpl.java:510)
> >> >> > at org.apache.qpid.server.queue.QueueEntryImpl.delete(QueueEntryImpl.java:524)
> >> >> > at org.apache.qpid.server.queue.AbstractQueue$13.postCommit(AbstractQueue.java:1903)
> >> >> > at org.apache.qpid.server.txn.AutoCommitTransaction.dequeue(AutoCommitTransaction.java:90)
> >> >> > at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1897)
> >> >> > at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1892)
> >> >> > at org.apache.qpid.server.queue.AbstractQueue.checkMessageStatus(AbstractQueue.java:2490)
> >> >> > at org.apache.qpid.server.virtualhost.AbstractVirtualHost$VirtualHostHouseKeepingTask.execute(AbstractVirtualHost.java:1225)
> >> >> > at org.apache.qpid.server.virtualhost.HouseKeepingTask$1.run(HouseKeepingTask.java:65)
> >> >> > at java.security.AccessController.doPrivileged(Native Method)
> >> >> > at org.apache.qpid.server.virtualhost.HouseKeepingTask.run(HouseKeepingTask.java:60)
> >> >> > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> >> > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> >> >> > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> >> >> > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> >> >> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> >> > at java.lang.Thread.run(Thread.java:745)
> >> >> >
> >> >> > Regards,
> >> >> > Adel
> >> >> >
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> >> >> For additional commands, e-mail: users-help@qpid.apache.org
> >> >>
> >> >
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> >> For additional commands, e-mail: users-help@qpid.apache.org
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
Posted by Keith W <ke...@gmail.com>.
On 27 July 2016 at 08:42, Adel Boutros <ad...@live.com> wrote:
> If the broker is supposed to shutdown on this exception then I confirm it is happening on the exit of the broker because my tests are completing successfully.
> Adel
Let me check I understand. You are running your tests, the tests
completed, and you then shutdown the Broker. Sometimes the Broker
fails during shutdown with the exception you report. Is that a
correct summary?
I think your test includes messages with TTL set and that your test
suite may leaves messages on the Broker.
>
>> From: keith.wall@gmail.com
>> Date: Wed, 27 Jul 2016 08:30:31 +0100
>> Subject: Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
>> To: users@qpid.apache.org
>>
>> On 26 July 2016 at 14:32, Adel Boutros <ad...@live.com> wrote:
>> >
>> > Hello Keith,Unfortunately, as the issue is random and I am restart the broker a lot of times in my tests, I don't have a qpid.log to provide. I will try to reproduce and send it.
>>
>> Great.
>>
>> > As for the update to 6.0.4, one my colleagues tried to upgrade from 6.0.0 to 6.0.4 and encountered failures at the level of our binding filters tests. So we upgraded to 6.0.1 instead. We didn't have time yet to inspect the failures yet and report them if needed.
>>
>> Sounds curious. If you have a repeatable regression, we'd very much
>> appreciate a separate report.
>>
>> > In the meantime, I have created a Jira issue for the unhandled exception. Maybe the exception is expected but I was hoping to see it in the logs not as part of an unhandled exception.https://issues.apache.org/jira/browse/QPID-7363
>>
>> This type of exception is a 'should never happen' and at the moment
>> the strategy is to shut down the whole Broker. We have plans to
>> change this over the next major releases to be more discriminating
>> about the units within the Broker that should go down in response to
>> such a condition.
>>
>> Regards,Adel
>> >> From: keith.wall@gmail.com
>> >> Date: Tue, 26 Jul 2016 13:34:09 +0100
>> >> Subject: Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
>> >> To: users@qpid.apache.org
>> >>
>> >> Hi Adel,
>> >>
>> >> Thanks for reporting, it is certainly not normal.
>> >>
>> >> Could you raise a Jira and attach a minimal qpid.log, scrubbed of any
>> >> information you deem confidential (IP addresses etc)?
>> >> I am interested to see all the log statements produced by the Broker
>> >> from when it started to when it failed in the above way. My best
>> >> guess is something else has gone wrong prior and then there is this
>> >> second failure producing the unexpected shutdown.
>> >>
>> >> I am not aware of anything pertinent changing between 6.0.1 and 6.0.4
>> >> which might have a bearing here, but you should consider upgrading to
>> >> 6.0.4 anyway as there have been a number of important defect fixes
>> >> including security fixes.
>> >>
>> >> Kind regards Keith.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On 26 July 2016 at 13:01, Adel Boutros <ad...@live.com> wrote:
>> >> > Hello,
>> >> >
>> >> > Randomly, I have the below exception on the console when launching the broker. Is this expected?
>> >> > I am using Berkley DB jar "je-5.0.104.jar" as described here https://qpid.apache.org/releases/qpid-java-6.0.1/java-broker/book/Java-Broker-Miscellaneous-Installing-Oracle-BDB-JE.html.
>> >> >
>> >> > ########################################################################
>> >> > #
>> >> > # Unhandled Exception java.lang.IllegalStateException: Message store is not open in Thread virtualhost-default-pool-2
>> >> > #
>> >> > # Exiting
>> >> > #
>> >> > ########################################################################
>> >> > java.lang.IllegalStateException: Message store is not open
>> >> > at org.apache.qpid.server.store.berkeleydb.BDBMessageStore.checkMessageStoreOpen(BDBMessageStore.java:154)
>> >> > at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.remove(AbstractBDBMessageStore.java:1202)
>> >> > at org.apache.qpid.server.message.AbstractServerMessageImpl.decrementReference(AbstractServerMessageImpl.java:110)
>> >> > at org.apache.qpid.server.message.AbstractServerMessageImpl.access$500(AbstractServerMessageImpl.java:38)
>> >> > at org.apache.qpid.server.message.AbstractServerMessageImpl$Reference.release(AbstractServerMessageImpl.java:308)
>> >> > at org.apache.qpid.server.queue.QueueEntryImpl.dispose(QueueEntryImpl.java:510)
>> >> > at org.apache.qpid.server.queue.QueueEntryImpl.delete(QueueEntryImpl.java:524)
>> >> > at org.apache.qpid.server.queue.AbstractQueue$13.postCommit(AbstractQueue.java:1903)
>> >> > at org.apache.qpid.server.txn.AutoCommitTransaction.dequeue(AutoCommitTransaction.java:90)
>> >> > at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1897)
>> >> > at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1892)
>> >> > at org.apache.qpid.server.queue.AbstractQueue.checkMessageStatus(AbstractQueue.java:2490)
>> >> > at org.apache.qpid.server.virtualhost.AbstractVirtualHost$VirtualHostHouseKeepingTask.execute(AbstractVirtualHost.java:1225)
>> >> > at org.apache.qpid.server.virtualhost.HouseKeepingTask$1.run(HouseKeepingTask.java:65)
>> >> > at java.security.AccessController.doPrivileged(Native Method)
>> >> > at org.apache.qpid.server.virtualhost.HouseKeepingTask.run(HouseKeepingTask.java:60)
>> >> > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>> >> > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>> >> > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>> >> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >> > at java.lang.Thread.run(Thread.java:745)
>> >> >
>> >> > Regards,
>> >> > Adel
>> >> >
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> >> For additional commands, e-mail: users-help@qpid.apache.org
>> >>
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: [Java Broker - 6.0.1] Unhandled exception with Berkley DB
message store
Posted by Adel Boutros <ad...@live.com>.
If the broker is supposed to shutdown on this exception then I confirm it is happening on the exit of the broker because my tests are completing successfully.
Adel
> From: keith.wall@gmail.com
> Date: Wed, 27 Jul 2016 08:30:31 +0100
> Subject: Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
> To: users@qpid.apache.org
>
> On 26 July 2016 at 14:32, Adel Boutros <ad...@live.com> wrote:
> >
> > Hello Keith,Unfortunately, as the issue is random and I am restart the broker a lot of times in my tests, I don't have a qpid.log to provide. I will try to reproduce and send it.
>
> Great.
>
> > As for the update to 6.0.4, one my colleagues tried to upgrade from 6.0.0 to 6.0.4 and encountered failures at the level of our binding filters tests. So we upgraded to 6.0.1 instead. We didn't have time yet to inspect the failures yet and report them if needed.
>
> Sounds curious. If you have a repeatable regression, we'd very much
> appreciate a separate report.
>
> > In the meantime, I have created a Jira issue for the unhandled exception. Maybe the exception is expected but I was hoping to see it in the logs not as part of an unhandled exception.https://issues.apache.org/jira/browse/QPID-7363
>
> This type of exception is a 'should never happen' and at the moment
> the strategy is to shut down the whole Broker. We have plans to
> change this over the next major releases to be more discriminating
> about the units within the Broker that should go down in response to
> such a condition.
>
> Regards,Adel
> >> From: keith.wall@gmail.com
> >> Date: Tue, 26 Jul 2016 13:34:09 +0100
> >> Subject: Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
> >> To: users@qpid.apache.org
> >>
> >> Hi Adel,
> >>
> >> Thanks for reporting, it is certainly not normal.
> >>
> >> Could you raise a Jira and attach a minimal qpid.log, scrubbed of any
> >> information you deem confidential (IP addresses etc)?
> >> I am interested to see all the log statements produced by the Broker
> >> from when it started to when it failed in the above way. My best
> >> guess is something else has gone wrong prior and then there is this
> >> second failure producing the unexpected shutdown.
> >>
> >> I am not aware of anything pertinent changing between 6.0.1 and 6.0.4
> >> which might have a bearing here, but you should consider upgrading to
> >> 6.0.4 anyway as there have been a number of important defect fixes
> >> including security fixes.
> >>
> >> Kind regards Keith.
> >>
> >>
> >>
> >>
> >>
> >> On 26 July 2016 at 13:01, Adel Boutros <ad...@live.com> wrote:
> >> > Hello,
> >> >
> >> > Randomly, I have the below exception on the console when launching the broker. Is this expected?
> >> > I am using Berkley DB jar "je-5.0.104.jar" as described here https://qpid.apache.org/releases/qpid-java-6.0.1/java-broker/book/Java-Broker-Miscellaneous-Installing-Oracle-BDB-JE.html.
> >> >
> >> > ########################################################################
> >> > #
> >> > # Unhandled Exception java.lang.IllegalStateException: Message store is not open in Thread virtualhost-default-pool-2
> >> > #
> >> > # Exiting
> >> > #
> >> > ########################################################################
> >> > java.lang.IllegalStateException: Message store is not open
> >> > at org.apache.qpid.server.store.berkeleydb.BDBMessageStore.checkMessageStoreOpen(BDBMessageStore.java:154)
> >> > at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.remove(AbstractBDBMessageStore.java:1202)
> >> > at org.apache.qpid.server.message.AbstractServerMessageImpl.decrementReference(AbstractServerMessageImpl.java:110)
> >> > at org.apache.qpid.server.message.AbstractServerMessageImpl.access$500(AbstractServerMessageImpl.java:38)
> >> > at org.apache.qpid.server.message.AbstractServerMessageImpl$Reference.release(AbstractServerMessageImpl.java:308)
> >> > at org.apache.qpid.server.queue.QueueEntryImpl.dispose(QueueEntryImpl.java:510)
> >> > at org.apache.qpid.server.queue.QueueEntryImpl.delete(QueueEntryImpl.java:524)
> >> > at org.apache.qpid.server.queue.AbstractQueue$13.postCommit(AbstractQueue.java:1903)
> >> > at org.apache.qpid.server.txn.AutoCommitTransaction.dequeue(AutoCommitTransaction.java:90)
> >> > at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1897)
> >> > at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1892)
> >> > at org.apache.qpid.server.queue.AbstractQueue.checkMessageStatus(AbstractQueue.java:2490)
> >> > at org.apache.qpid.server.virtualhost.AbstractVirtualHost$VirtualHostHouseKeepingTask.execute(AbstractVirtualHost.java:1225)
> >> > at org.apache.qpid.server.virtualhost.HouseKeepingTask$1.run(HouseKeepingTask.java:65)
> >> > at java.security.AccessController.doPrivileged(Native Method)
> >> > at org.apache.qpid.server.virtualhost.HouseKeepingTask.run(HouseKeepingTask.java:60)
> >> > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> >> > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> >> > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> >> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> >
> >> > Regards,
> >> > Adel
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> >> For additional commands, e-mail: users-help@qpid.apache.org
> >>
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
Posted by Keith W <ke...@gmail.com>.
On 26 July 2016 at 14:32, Adel Boutros <ad...@live.com> wrote:
>
> Hello Keith,Unfortunately, as the issue is random and I am restart the broker a lot of times in my tests, I don't have a qpid.log to provide. I will try to reproduce and send it.
Great.
> As for the update to 6.0.4, one my colleagues tried to upgrade from 6.0.0 to 6.0.4 and encountered failures at the level of our binding filters tests. So we upgraded to 6.0.1 instead. We didn't have time yet to inspect the failures yet and report them if needed.
Sounds curious. If you have a repeatable regression, we'd very much
appreciate a separate report.
> In the meantime, I have created a Jira issue for the unhandled exception. Maybe the exception is expected but I was hoping to see it in the logs not as part of an unhandled exception.https://issues.apache.org/jira/browse/QPID-7363
This type of exception is a 'should never happen' and at the moment
the strategy is to shut down the whole Broker. We have plans to
change this over the next major releases to be more discriminating
about the units within the Broker that should go down in response to
such a condition.
Regards,Adel
>> From: keith.wall@gmail.com
>> Date: Tue, 26 Jul 2016 13:34:09 +0100
>> Subject: Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
>> To: users@qpid.apache.org
>>
>> Hi Adel,
>>
>> Thanks for reporting, it is certainly not normal.
>>
>> Could you raise a Jira and attach a minimal qpid.log, scrubbed of any
>> information you deem confidential (IP addresses etc)?
>> I am interested to see all the log statements produced by the Broker
>> from when it started to when it failed in the above way. My best
>> guess is something else has gone wrong prior and then there is this
>> second failure producing the unexpected shutdown.
>>
>> I am not aware of anything pertinent changing between 6.0.1 and 6.0.4
>> which might have a bearing here, but you should consider upgrading to
>> 6.0.4 anyway as there have been a number of important defect fixes
>> including security fixes.
>>
>> Kind regards Keith.
>>
>>
>>
>>
>>
>> On 26 July 2016 at 13:01, Adel Boutros <ad...@live.com> wrote:
>> > Hello,
>> >
>> > Randomly, I have the below exception on the console when launching the broker. Is this expected?
>> > I am using Berkley DB jar "je-5.0.104.jar" as described here https://qpid.apache.org/releases/qpid-java-6.0.1/java-broker/book/Java-Broker-Miscellaneous-Installing-Oracle-BDB-JE.html.
>> >
>> > ########################################################################
>> > #
>> > # Unhandled Exception java.lang.IllegalStateException: Message store is not open in Thread virtualhost-default-pool-2
>> > #
>> > # Exiting
>> > #
>> > ########################################################################
>> > java.lang.IllegalStateException: Message store is not open
>> > at org.apache.qpid.server.store.berkeleydb.BDBMessageStore.checkMessageStoreOpen(BDBMessageStore.java:154)
>> > at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.remove(AbstractBDBMessageStore.java:1202)
>> > at org.apache.qpid.server.message.AbstractServerMessageImpl.decrementReference(AbstractServerMessageImpl.java:110)
>> > at org.apache.qpid.server.message.AbstractServerMessageImpl.access$500(AbstractServerMessageImpl.java:38)
>> > at org.apache.qpid.server.message.AbstractServerMessageImpl$Reference.release(AbstractServerMessageImpl.java:308)
>> > at org.apache.qpid.server.queue.QueueEntryImpl.dispose(QueueEntryImpl.java:510)
>> > at org.apache.qpid.server.queue.QueueEntryImpl.delete(QueueEntryImpl.java:524)
>> > at org.apache.qpid.server.queue.AbstractQueue$13.postCommit(AbstractQueue.java:1903)
>> > at org.apache.qpid.server.txn.AutoCommitTransaction.dequeue(AutoCommitTransaction.java:90)
>> > at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1897)
>> > at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1892)
>> > at org.apache.qpid.server.queue.AbstractQueue.checkMessageStatus(AbstractQueue.java:2490)
>> > at org.apache.qpid.server.virtualhost.AbstractVirtualHost$VirtualHostHouseKeepingTask.execute(AbstractVirtualHost.java:1225)
>> > at org.apache.qpid.server.virtualhost.HouseKeepingTask$1.run(HouseKeepingTask.java:65)
>> > at java.security.AccessController.doPrivileged(Native Method)
>> > at org.apache.qpid.server.virtualhost.HouseKeepingTask.run(HouseKeepingTask.java:60)
>> > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>> > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>> > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> > at java.lang.Thread.run(Thread.java:745)
>> >
>> > Regards,
>> > Adel
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: [Java Broker - 6.0.1] Unhandled exception with Berkley DB
message store
Posted by Adel Boutros <ad...@live.com>.
Hello Keith,Unfortunately, as the issue is random and I am restart the broker a lot of times in my tests, I don't have a qpid.log to provide. I will try to reproduce and send it.As for the update to 6.0.4, one my colleagues tried to upgrade from 6.0.0 to 6.0.4 and encountered failures at the level of our binding filters tests. So we upgraded to 6.0.1 instead. We didn't have time yet to inspect the failures yet and report them if needed.
In the meantime, I have created a Jira issue for the unhandled exception. Maybe the exception is expected but I was hoping to see it in the logs not as part of an unhandled exception.https://issues.apache.org/jira/browse/QPID-7363Regards,Adel
> From: keith.wall@gmail.com
> Date: Tue, 26 Jul 2016 13:34:09 +0100
> Subject: Re: [Java Broker - 6.0.1] Unhandled exception with Berkley DB message store
> To: users@qpid.apache.org
>
> Hi Adel,
>
> Thanks for reporting, it is certainly not normal.
>
> Could you raise a Jira and attach a minimal qpid.log, scrubbed of any
> information you deem confidential (IP addresses etc)?
> I am interested to see all the log statements produced by the Broker
> from when it started to when it failed in the above way. My best
> guess is something else has gone wrong prior and then there is this
> second failure producing the unexpected shutdown.
>
> I am not aware of anything pertinent changing between 6.0.1 and 6.0.4
> which might have a bearing here, but you should consider upgrading to
> 6.0.4 anyway as there have been a number of important defect fixes
> including security fixes.
>
> Kind regards Keith.
>
>
>
>
>
> On 26 July 2016 at 13:01, Adel Boutros <ad...@live.com> wrote:
> > Hello,
> >
> > Randomly, I have the below exception on the console when launching the broker. Is this expected?
> > I am using Berkley DB jar "je-5.0.104.jar" as described here https://qpid.apache.org/releases/qpid-java-6.0.1/java-broker/book/Java-Broker-Miscellaneous-Installing-Oracle-BDB-JE.html.
> >
> > ########################################################################
> > #
> > # Unhandled Exception java.lang.IllegalStateException: Message store is not open in Thread virtualhost-default-pool-2
> > #
> > # Exiting
> > #
> > ########################################################################
> > java.lang.IllegalStateException: Message store is not open
> > at org.apache.qpid.server.store.berkeleydb.BDBMessageStore.checkMessageStoreOpen(BDBMessageStore.java:154)
> > at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.remove(AbstractBDBMessageStore.java:1202)
> > at org.apache.qpid.server.message.AbstractServerMessageImpl.decrementReference(AbstractServerMessageImpl.java:110)
> > at org.apache.qpid.server.message.AbstractServerMessageImpl.access$500(AbstractServerMessageImpl.java:38)
> > at org.apache.qpid.server.message.AbstractServerMessageImpl$Reference.release(AbstractServerMessageImpl.java:308)
> > at org.apache.qpid.server.queue.QueueEntryImpl.dispose(QueueEntryImpl.java:510)
> > at org.apache.qpid.server.queue.QueueEntryImpl.delete(QueueEntryImpl.java:524)
> > at org.apache.qpid.server.queue.AbstractQueue$13.postCommit(AbstractQueue.java:1903)
> > at org.apache.qpid.server.txn.AutoCommitTransaction.dequeue(AutoCommitTransaction.java:90)
> > at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1897)
> > at org.apache.qpid.server.queue.AbstractQueue.dequeueEntry(AbstractQueue.java:1892)
> > at org.apache.qpid.server.queue.AbstractQueue.checkMessageStatus(AbstractQueue.java:2490)
> > at org.apache.qpid.server.virtualhost.AbstractVirtualHost$VirtualHostHouseKeepingTask.execute(AbstractVirtualHost.java:1225)
> > at org.apache.qpid.server.virtualhost.HouseKeepingTask$1.run(HouseKeepingTask.java:65)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at org.apache.qpid.server.virtualhost.HouseKeepingTask.run(HouseKeepingTask.java:60)
> > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:745)
> >
> > Regards,
> > Adel
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>