You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by ChristianWander <ch...@piffedeckel.de> on 2015/09/04 09:52:03 UTC
Offline durable subscriptions lost when rebuilding kahadb index.
Hi,
I'm using ActiveMQ 5.10.2 and the included example code
(jms-example-durable-sub) to play around with durable subscriptions.
The examples work fine and the broker remembers the durable subscriptions
when the consumer goes offline and the broker is restarted.
However, if I stop the broker and delete the files db.data and db.redo to
enforce a rebuild of the kahadb index according to here
<https://access.redhat.com/solutions/276323> , the broker forgets about the
offline durable subscriptions.
Shouldn't this be fixed with [AMQ4212]
<https://issues.apache.org/jira/browse/AMQ-4212> or am I doing something
wrong here?
Chris
--
View this message in context: http://activemq.2283324.n4.nabble.com/Offline-durable-subscriptions-lost-when-rebuilding-kahadb-index-tp4701701.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Offline durable subscriptions lost when rebuilding kahadb index.
Posted by Tim Bain <tb...@alumni.duke.edu>.
Thanks to Tim Bish for implementing the fix, and to Chris for letting us
know about it.
On Jan 20, 2016 6:26 AM, "Christopher Shannon" <
christopher.l.shannon@gmail.com> wrote:
> A bug has been fixed with this:
> https://issues.apache.org/jira/browse/AMQ-6131
>
> Durables should come back now even after an index rebuild starting with
> 5.13.1 and 5.14.0
>
> On Tue, Dec 29, 2015 at 9:09 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
>
> > Also, AMQ-4212 clearly requested that this exact problem be resolved, it
> is
> > marked as Fixed in 5.9, and neither Tim nor Gary made any comments that
> > indicate that a full solution wasn't implemented. What you're saying
> > sounds like we didn't actually resolve the problem, which doesn't match
> > what the JIRA record indicates.
> >
> > If the goal was a complete fix, then we have a regression (or a bug in
> the
> > fix) and a new JIRA bug should be submitted. If the decision got made
> not
> > to attempt a full fix, then the JIRA record should have been updated to
> > reflect what we actually did (and a new record that's a clone of the
> > original should have been submitted to track the fact that we haven't
> > actually addressed the issue requested by the original, even though we
> did
> > work under the original and are closing it to reflect that we're done
> with
> > the smaller scope we decided to pursue).
> >
> > Tim, can you provide any insight into whether we've got a regression/bug
> or
> > just need to do some cleanup in JIRA to avoid misleading people?
> >
> > Tim
> > On Dec 29, 2015 6:55 AM, "Tim Bain" <tb...@alumni.duke.edu> wrote:
> >
> > > A separate KahaDB instance solely for durable subscription messages
> would
> > > be a solution that doesn't require compaction. We already keep
> scheduled
> > > messages in their own KahaDB instance (right?) for exactly the same
> > > reason. I'm not sure the lifecycle of durable subscription messages
> and
> > > the lifecycle of scheduled messages are guaranteed to be similar enough
> > to
> > > put them in the same KahaDB instance, but we can certainly use the same
> > > paradigm a second time.
> > >
> > > Tim
> > >
> > > Tim
> > > On Dec 28, 2015 10:30 AM, "Christopher Shannon" <
> > > christopher.l.shannon@gmail.com> wrote:
> > >
> > >> This is the way KahaDB currently works. There is no way to keep
> around
> > >> the
> > >> durable subscriptions (other than the index) because then KahaDB would
> > >> never be able to GC any files. The main way I can think of to fix
> this
> > >> would be to do some sort of compaction, as Tim pointed out in the
> > comments
> > >> of AMQ-4212. However, compaction would not be a trivial thing to
> > >> implement.
> > >>
> > >> On Fri, Dec 25, 2015 at 10:00 AM, Tim Bain <tb...@alumni.duke.edu>
> > wrote:
> > >>
> > >> > Chris,
> > >> >
> > >> > Sorry no one responded to you before now, but this certainly does
> > sound
> > >> > like AMQ-4212 didn't fix all causes of the problem. Can you submit
> a
> > >> bug
> > >> > in JIRA and attach a set of data files and the corresponding index
> > that
> > >> > will demonstrate the problem?
> > >> >
> > >> > Thanks,
> > >> > Tim
> > >> > On Sep 4, 2015 7:03 AM, "ChristianWander" <ch...@piffedeckel.de>
> > wrote:
> > >> >
> > >> > > Hi,
> > >> > >
> > >> > > I'm using ActiveMQ 5.10.2 and the included example code
> > >> > > (jms-example-durable-sub) to play around with durable
> subscriptions.
> > >> > >
> > >> > > The examples work fine and the broker remembers the durable
> > >> subscriptions
> > >> > > when the consumer goes offline and the broker is restarted.
> > >> > >
> > >> > > However, if I stop the broker and delete the files db.data and
> > >> db.redo to
> > >> > > enforce a rebuild of the kahadb index according to here
> > >> > > <https://access.redhat.com/solutions/276323> , the broker
> forgets
> > >> about
> > >> > > the
> > >> > > offline durable subscriptions.
> > >> > >
> > >> > > Shouldn't this be fixed with [AMQ4212]
> > >> > > <https://issues.apache.org/jira/browse/AMQ-4212> or am I doing
> > >> > something
> > >> > > wrong here?
> > >> > >
> > >> > > Chris
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > View this message in context:
> > >> > >
> > >> >
> > >>
> >
> http://activemq.2283324.n4.nabble.com/Offline-durable-subscriptions-lost-when-rebuilding-kahadb-index-tp4701701.html
> > >> > > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> > >> >
> > >>
> > >
> >
>
Re: Offline durable subscriptions lost when rebuilding kahadb index.
Posted by Christopher Shannon <ch...@gmail.com>.
A bug has been fixed with this:
https://issues.apache.org/jira/browse/AMQ-6131
Durables should come back now even after an index rebuild starting with
5.13.1 and 5.14.0
On Tue, Dec 29, 2015 at 9:09 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
> Also, AMQ-4212 clearly requested that this exact problem be resolved, it is
> marked as Fixed in 5.9, and neither Tim nor Gary made any comments that
> indicate that a full solution wasn't implemented. What you're saying
> sounds like we didn't actually resolve the problem, which doesn't match
> what the JIRA record indicates.
>
> If the goal was a complete fix, then we have a regression (or a bug in the
> fix) and a new JIRA bug should be submitted. If the decision got made not
> to attempt a full fix, then the JIRA record should have been updated to
> reflect what we actually did (and a new record that's a clone of the
> original should have been submitted to track the fact that we haven't
> actually addressed the issue requested by the original, even though we did
> work under the original and are closing it to reflect that we're done with
> the smaller scope we decided to pursue).
>
> Tim, can you provide any insight into whether we've got a regression/bug or
> just need to do some cleanup in JIRA to avoid misleading people?
>
> Tim
> On Dec 29, 2015 6:55 AM, "Tim Bain" <tb...@alumni.duke.edu> wrote:
>
> > A separate KahaDB instance solely for durable subscription messages would
> > be a solution that doesn't require compaction. We already keep scheduled
> > messages in their own KahaDB instance (right?) for exactly the same
> > reason. I'm not sure the lifecycle of durable subscription messages and
> > the lifecycle of scheduled messages are guaranteed to be similar enough
> to
> > put them in the same KahaDB instance, but we can certainly use the same
> > paradigm a second time.
> >
> > Tim
> >
> > Tim
> > On Dec 28, 2015 10:30 AM, "Christopher Shannon" <
> > christopher.l.shannon@gmail.com> wrote:
> >
> >> This is the way KahaDB currently works. There is no way to keep around
> >> the
> >> durable subscriptions (other than the index) because then KahaDB would
> >> never be able to GC any files. The main way I can think of to fix this
> >> would be to do some sort of compaction, as Tim pointed out in the
> comments
> >> of AMQ-4212. However, compaction would not be a trivial thing to
> >> implement.
> >>
> >> On Fri, Dec 25, 2015 at 10:00 AM, Tim Bain <tb...@alumni.duke.edu>
> wrote:
> >>
> >> > Chris,
> >> >
> >> > Sorry no one responded to you before now, but this certainly does
> sound
> >> > like AMQ-4212 didn't fix all causes of the problem. Can you submit a
> >> bug
> >> > in JIRA and attach a set of data files and the corresponding index
> that
> >> > will demonstrate the problem?
> >> >
> >> > Thanks,
> >> > Tim
> >> > On Sep 4, 2015 7:03 AM, "ChristianWander" <ch...@piffedeckel.de>
> wrote:
> >> >
> >> > > Hi,
> >> > >
> >> > > I'm using ActiveMQ 5.10.2 and the included example code
> >> > > (jms-example-durable-sub) to play around with durable subscriptions.
> >> > >
> >> > > The examples work fine and the broker remembers the durable
> >> subscriptions
> >> > > when the consumer goes offline and the broker is restarted.
> >> > >
> >> > > However, if I stop the broker and delete the files db.data and
> >> db.redo to
> >> > > enforce a rebuild of the kahadb index according to here
> >> > > <https://access.redhat.com/solutions/276323> , the broker forgets
> >> about
> >> > > the
> >> > > offline durable subscriptions.
> >> > >
> >> > > Shouldn't this be fixed with [AMQ4212]
> >> > > <https://issues.apache.org/jira/browse/AMQ-4212> or am I doing
> >> > something
> >> > > wrong here?
> >> > >
> >> > > Chris
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > View this message in context:
> >> > >
> >> >
> >>
> http://activemq.2283324.n4.nabble.com/Offline-durable-subscriptions-lost-when-rebuilding-kahadb-index-tp4701701.html
> >> > > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >> >
> >>
> >
>
Re: Offline durable subscriptions lost when rebuilding kahadb index.
Posted by Tim Bain <tb...@alumni.duke.edu>.
Also, AMQ-4212 clearly requested that this exact problem be resolved, it is
marked as Fixed in 5.9, and neither Tim nor Gary made any comments that
indicate that a full solution wasn't implemented. What you're saying
sounds like we didn't actually resolve the problem, which doesn't match
what the JIRA record indicates.
If the goal was a complete fix, then we have a regression (or a bug in the
fix) and a new JIRA bug should be submitted. If the decision got made not
to attempt a full fix, then the JIRA record should have been updated to
reflect what we actually did (and a new record that's a clone of the
original should have been submitted to track the fact that we haven't
actually addressed the issue requested by the original, even though we did
work under the original and are closing it to reflect that we're done with
the smaller scope we decided to pursue).
Tim, can you provide any insight into whether we've got a regression/bug or
just need to do some cleanup in JIRA to avoid misleading people?
Tim
On Dec 29, 2015 6:55 AM, "Tim Bain" <tb...@alumni.duke.edu> wrote:
> A separate KahaDB instance solely for durable subscription messages would
> be a solution that doesn't require compaction. We already keep scheduled
> messages in their own KahaDB instance (right?) for exactly the same
> reason. I'm not sure the lifecycle of durable subscription messages and
> the lifecycle of scheduled messages are guaranteed to be similar enough to
> put them in the same KahaDB instance, but we can certainly use the same
> paradigm a second time.
>
> Tim
>
> Tim
> On Dec 28, 2015 10:30 AM, "Christopher Shannon" <
> christopher.l.shannon@gmail.com> wrote:
>
>> This is the way KahaDB currently works. There is no way to keep around
>> the
>> durable subscriptions (other than the index) because then KahaDB would
>> never be able to GC any files. The main way I can think of to fix this
>> would be to do some sort of compaction, as Tim pointed out in the comments
>> of AMQ-4212. However, compaction would not be a trivial thing to
>> implement.
>>
>> On Fri, Dec 25, 2015 at 10:00 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
>>
>> > Chris,
>> >
>> > Sorry no one responded to you before now, but this certainly does sound
>> > like AMQ-4212 didn't fix all causes of the problem. Can you submit a
>> bug
>> > in JIRA and attach a set of data files and the corresponding index that
>> > will demonstrate the problem?
>> >
>> > Thanks,
>> > Tim
>> > On Sep 4, 2015 7:03 AM, "ChristianWander" <ch...@piffedeckel.de> wrote:
>> >
>> > > Hi,
>> > >
>> > > I'm using ActiveMQ 5.10.2 and the included example code
>> > > (jms-example-durable-sub) to play around with durable subscriptions.
>> > >
>> > > The examples work fine and the broker remembers the durable
>> subscriptions
>> > > when the consumer goes offline and the broker is restarted.
>> > >
>> > > However, if I stop the broker and delete the files db.data and
>> db.redo to
>> > > enforce a rebuild of the kahadb index according to here
>> > > <https://access.redhat.com/solutions/276323> , the broker forgets
>> about
>> > > the
>> > > offline durable subscriptions.
>> > >
>> > > Shouldn't this be fixed with [AMQ4212]
>> > > <https://issues.apache.org/jira/browse/AMQ-4212> or am I doing
>> > something
>> > > wrong here?
>> > >
>> > > Chris
>> > >
>> > >
>> > >
>> > > --
>> > > View this message in context:
>> > >
>> >
>> http://activemq.2283324.n4.nabble.com/Offline-durable-subscriptions-lost-when-rebuilding-kahadb-index-tp4701701.html
>> > > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>> >
>>
>
Re: Offline durable subscriptions lost when rebuilding kahadb index.
Posted by Tim Bain <tb...@alumni.duke.edu>.
A separate KahaDB instance solely for durable subscription messages would
be a solution that doesn't require compaction. We already keep scheduled
messages in their own KahaDB instance (right?) for exactly the same
reason. I'm not sure the lifecycle of durable subscription messages and
the lifecycle of scheduled messages are guaranteed to be similar enough to
put them in the same KahaDB instance, but we can certainly use the same
paradigm a second time.
Tim
Tim
On Dec 28, 2015 10:30 AM, "Christopher Shannon" <
christopher.l.shannon@gmail.com> wrote:
> This is the way KahaDB currently works. There is no way to keep around the
> durable subscriptions (other than the index) because then KahaDB would
> never be able to GC any files. The main way I can think of to fix this
> would be to do some sort of compaction, as Tim pointed out in the comments
> of AMQ-4212. However, compaction would not be a trivial thing to
> implement.
>
> On Fri, Dec 25, 2015 at 10:00 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
>
> > Chris,
> >
> > Sorry no one responded to you before now, but this certainly does sound
> > like AMQ-4212 didn't fix all causes of the problem. Can you submit a bug
> > in JIRA and attach a set of data files and the corresponding index that
> > will demonstrate the problem?
> >
> > Thanks,
> > Tim
> > On Sep 4, 2015 7:03 AM, "ChristianWander" <ch...@piffedeckel.de> wrote:
> >
> > > Hi,
> > >
> > > I'm using ActiveMQ 5.10.2 and the included example code
> > > (jms-example-durable-sub) to play around with durable subscriptions.
> > >
> > > The examples work fine and the broker remembers the durable
> subscriptions
> > > when the consumer goes offline and the broker is restarted.
> > >
> > > However, if I stop the broker and delete the files db.data and db.redo
> to
> > > enforce a rebuild of the kahadb index according to here
> > > <https://access.redhat.com/solutions/276323> , the broker forgets
> about
> > > the
> > > offline durable subscriptions.
> > >
> > > Shouldn't this be fixed with [AMQ4212]
> > > <https://issues.apache.org/jira/browse/AMQ-4212> or am I doing
> > something
> > > wrong here?
> > >
> > > Chris
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > >
> >
> http://activemq.2283324.n4.nabble.com/Offline-durable-subscriptions-lost-when-rebuilding-kahadb-index-tp4701701.html
> > > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >
>
Re: Offline durable subscriptions lost when rebuilding kahadb index.
Posted by Christopher Shannon <ch...@gmail.com>.
This is the way KahaDB currently works. There is no way to keep around the
durable subscriptions (other than the index) because then KahaDB would
never be able to GC any files. The main way I can think of to fix this
would be to do some sort of compaction, as Tim pointed out in the comments
of AMQ-4212. However, compaction would not be a trivial thing to implement.
On Fri, Dec 25, 2015 at 10:00 AM, Tim Bain <tb...@alumni.duke.edu> wrote:
> Chris,
>
> Sorry no one responded to you before now, but this certainly does sound
> like AMQ-4212 didn't fix all causes of the problem. Can you submit a bug
> in JIRA and attach a set of data files and the corresponding index that
> will demonstrate the problem?
>
> Thanks,
> Tim
> On Sep 4, 2015 7:03 AM, "ChristianWander" <ch...@piffedeckel.de> wrote:
>
> > Hi,
> >
> > I'm using ActiveMQ 5.10.2 and the included example code
> > (jms-example-durable-sub) to play around with durable subscriptions.
> >
> > The examples work fine and the broker remembers the durable subscriptions
> > when the consumer goes offline and the broker is restarted.
> >
> > However, if I stop the broker and delete the files db.data and db.redo to
> > enforce a rebuild of the kahadb index according to here
> > <https://access.redhat.com/solutions/276323> , the broker forgets about
> > the
> > offline durable subscriptions.
> >
> > Shouldn't this be fixed with [AMQ4212]
> > <https://issues.apache.org/jira/browse/AMQ-4212> or am I doing
> something
> > wrong here?
> >
> > Chris
> >
> >
> >
> > --
> > View this message in context:
> >
> http://activemq.2283324.n4.nabble.com/Offline-durable-subscriptions-lost-when-rebuilding-kahadb-index-tp4701701.html
> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Offline durable subscriptions lost when rebuilding kahadb index.
Posted by Tim Bain <tb...@alumni.duke.edu>.
Chris,
Sorry no one responded to you before now, but this certainly does sound
like AMQ-4212 didn't fix all causes of the problem. Can you submit a bug
in JIRA and attach a set of data files and the corresponding index that
will demonstrate the problem?
Thanks,
Tim
On Sep 4, 2015 7:03 AM, "ChristianWander" <ch...@piffedeckel.de> wrote:
> Hi,
>
> I'm using ActiveMQ 5.10.2 and the included example code
> (jms-example-durable-sub) to play around with durable subscriptions.
>
> The examples work fine and the broker remembers the durable subscriptions
> when the consumer goes offline and the broker is restarted.
>
> However, if I stop the broker and delete the files db.data and db.redo to
> enforce a rebuild of the kahadb index according to here
> <https://access.redhat.com/solutions/276323> , the broker forgets about
> the
> offline durable subscriptions.
>
> Shouldn't this be fixed with [AMQ4212]
> <https://issues.apache.org/jira/browse/AMQ-4212> or am I doing something
> wrong here?
>
> Chris
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Offline-durable-subscriptions-lost-when-rebuilding-kahadb-index-tp4701701.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.