0.32 linear store

[Matt Broadstone <mb...@gmail.com>]

Hey again,
Another issue I've run into switching to 0.32 is the fact that durable
topics I create are no longer persisted between broker restarts. The 0.28
release that was previously provided only had the legacy store, but 0.32
has the linear store which I'm trying out.

I provide a path to my store-dir and that's it, is there some other
configuration option I'm missing here for this? The qpidd logs don't
indicate any failure, and indicate that the linear-store is in fact being
used. Running qpidd with debug+ shows a whole bunch of information, and I
see the exchanges being added but no corresponding messages from the store
(this may be expected).

Regards,
Matt

Re: 0.32 linear store

[Matt Broadstone <mb...@gmail.com>]

On Wed, May 13, 2015 at 4:12 PM, Matt Broadstone <mb...@gmail.com> wrote:

>
>
> On Wed, May 13, 2015 at 4:10 PM, Matt Broadstone <mb...@gmail.com>
> wrote:
>
>> On Wed, May 13, 2015 at 4:04 PM, Kim van der Riet <ki...@redhat.com>
>> wrote:
>>
>>> The legacystore and linearstore have slightly different directory
>>> structures which were designed to be mutually exclusive.  There was some
>>> debate as to whether to perform an automatic "upgrade" between these, but
>>> it was decided that this would not be a prudent option, as it would make a
>>> subsequent downgrade difficult and there would be multiple copies of
>>> messages, etc. Also, legacystore is still available, although deprecated.
>>> Most users will want to wait until a point where they have no messages to
>>> perform an upgrade.
>>>
>>> For those users which absolutely need to upgrade, there are a few basic
>>> manipulations of the store directory that can be performed by hand or
>>> script (as they will repeat for each queue present).  While this is simple,
>>> it is error-prone, and I don't recommend it unless absolutely necessary.
>>> If you need this, I can provide a description of the steps needed.
>>>
>>> The discussion on this point can be seen here; it also contains an
>>> informational description of the directory change:
>>>
>>>
>>> http://mail-archives.apache.org/mod_mbox/qpid-users/201411.mbox/%3C1415731947.26261.81.camel%40localhost%3E
>>>
>>> Hope that helps,
>>> Kim
>>>
>>>
>> Hey Kim,
>> I'm actually running this in complete isolation, so it's a "fresh
>> install" so to speak. Having said that, qpidd doesn't seem to actually be
>> storing anything. I've pointed my store-dir to /var/run/qpidd, and I can
>> see there's a "qls" folder there, however when I add a durable topic the
>> size of that directory does not increase, and the topic doesn't show up
>> upon broker restart. Is there some configuration I'm missing? +1 to this
>> ticket: https://issues.apache.org/jira/browse/QPID-5493
>>
>> Matt
>>
>>
> I should mention that my test case is just: qpid-config -a
> user/pass@localhost add exchange topic test.topic --durable
>
> Matt
>
>

For future reference, the store was being stored on /var/run, which is a
ram disk in this case. Not an issue when run from a regular disk, apologies
for the noise.

Matt



>
>>
>>>
>>> On 05/13/2015 03:36 PM, Matt Broadstone wrote:
>>>
>>>> Hey again,
>>>> Another issue I've run into switching to 0.32 is the fact that durable
>>>> topics I create are no longer persisted between broker restarts. The
>>>> 0.28
>>>> release that was previously provided only had the legacy store, but 0.32
>>>> has the linear store which I'm trying out.
>>>>
>>>> I provide a path to my store-dir and that's it, is there some other
>>>> configuration option I'm missing here for this? The qpidd logs don't
>>>> indicate any failure, and indicate that the linear-store is in fact
>>>> being
>>>> used. Running qpidd with debug+ shows a whole bunch of information, and
>>>> I
>>>> see the exchanges being added but no corresponding messages from the
>>>> store
>>>> (this may be expected).
>>>>
>>>> Regards,
>>>> Matt
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>
>>>
>>
>

Re: 0.32 linear store

[Matt Broadstone <mb...@gmail.com>]

On Wed, May 13, 2015 at 4:10 PM, Matt Broadstone <mb...@gmail.com> wrote:

> On Wed, May 13, 2015 at 4:04 PM, Kim van der Riet <ki...@redhat.com>
> wrote:
>
>> The legacystore and linearstore have slightly different directory
>> structures which were designed to be mutually exclusive.  There was some
>> debate as to whether to perform an automatic "upgrade" between these, but
>> it was decided that this would not be a prudent option, as it would make a
>> subsequent downgrade difficult and there would be multiple copies of
>> messages, etc. Also, legacystore is still available, although deprecated.
>> Most users will want to wait until a point where they have no messages to
>> perform an upgrade.
>>
>> For those users which absolutely need to upgrade, there are a few basic
>> manipulations of the store directory that can be performed by hand or
>> script (as they will repeat for each queue present).  While this is simple,
>> it is error-prone, and I don't recommend it unless absolutely necessary.
>> If you need this, I can provide a description of the steps needed.
>>
>> The discussion on this point can be seen here; it also contains an
>> informational description of the directory change:
>>
>>
>> http://mail-archives.apache.org/mod_mbox/qpid-users/201411.mbox/%3C1415731947.26261.81.camel%40localhost%3E
>>
>> Hope that helps,
>> Kim
>>
>>
> Hey Kim,
> I'm actually running this in complete isolation, so it's a "fresh install"
> so to speak. Having said that, qpidd doesn't seem to actually be storing
> anything. I've pointed my store-dir to /var/run/qpidd, and I can see
> there's a "qls" folder there, however when I add a durable topic the size
> of that directory does not increase, and the topic doesn't show up upon
> broker restart. Is there some configuration I'm missing? +1 to this ticket:
> https://issues.apache.org/jira/browse/QPID-5493
>
> Matt
>
>
I should mention that my test case is just: qpid-config -a
user/pass@localhost add exchange topic test.topic --durable

Matt


>
>
>>
>> On 05/13/2015 03:36 PM, Matt Broadstone wrote:
>>
>>> Hey again,
>>> Another issue I've run into switching to 0.32 is the fact that durable
>>> topics I create are no longer persisted between broker restarts. The 0.28
>>> release that was previously provided only had the legacy store, but 0.32
>>> has the linear store which I'm trying out.
>>>
>>> I provide a path to my store-dir and that's it, is there some other
>>> configuration option I'm missing here for this? The qpidd logs don't
>>> indicate any failure, and indicate that the linear-store is in fact being
>>> used. Running qpidd with debug+ shows a whole bunch of information, and I
>>> see the exchanges being added but no corresponding messages from the
>>> store
>>> (this may be expected).
>>>
>>> Regards,
>>> Matt
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
>

Re: 0.32 linear store

[Matt Broadstone <mb...@gmail.com>]

On Wed, May 13, 2015 at 4:04 PM, Kim van der Riet <ki...@redhat.com>
wrote:

> The legacystore and linearstore have slightly different directory
> structures which were designed to be mutually exclusive.  There was some
> debate as to whether to perform an automatic "upgrade" between these, but
> it was decided that this would not be a prudent option, as it would make a
> subsequent downgrade difficult and there would be multiple copies of
> messages, etc. Also, legacystore is still available, although deprecated.
> Most users will want to wait until a point where they have no messages to
> perform an upgrade.
>
> For those users which absolutely need to upgrade, there are a few basic
> manipulations of the store directory that can be performed by hand or
> script (as they will repeat for each queue present).  While this is simple,
> it is error-prone, and I don't recommend it unless absolutely necessary.
> If you need this, I can provide a description of the steps needed.
>
> The discussion on this point can be seen here; it also contains an
> informational description of the directory change:
>
>
> http://mail-archives.apache.org/mod_mbox/qpid-users/201411.mbox/%3C1415731947.26261.81.camel%40localhost%3E
>
> Hope that helps,
> Kim
>
>
Hey Kim,
I'm actually running this in complete isolation, so it's a "fresh install"
so to speak. Having said that, qpidd doesn't seem to actually be storing
anything. I've pointed my store-dir to /var/run/qpidd, and I can see
there's a "qls" folder there, however when I add a durable topic the size
of that directory does not increase, and the topic doesn't show up upon
broker restart. Is there some configuration I'm missing? +1 to this ticket:
https://issues.apache.org/jira/browse/QPID-5493

Matt



>
> On 05/13/2015 03:36 PM, Matt Broadstone wrote:
>
>> Hey again,
>> Another issue I've run into switching to 0.32 is the fact that durable
>> topics I create are no longer persisted between broker restarts. The 0.28
>> release that was previously provided only had the legacy store, but 0.32
>> has the linear store which I'm trying out.
>>
>> I provide a path to my store-dir and that's it, is there some other
>> configuration option I'm missing here for this? The qpidd logs don't
>> indicate any failure, and indicate that the linear-store is in fact being
>> used. Running qpidd with debug+ shows a whole bunch of information, and I
>> see the exchanges being added but no corresponding messages from the store
>> (this may be expected).
>>
>> Regards,
>> Matt
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>

Re: 0.32 linear store

[Kim van der Riet <ki...@redhat.com>]

The legacystore and linearstore have slightly different directory 
structures which were designed to be mutually exclusive.  There was some 
debate as to whether to perform an automatic "upgrade" between these, 
but it was decided that this would not be a prudent option, as it would 
make a subsequent downgrade difficult and there would be multiple copies 
of messages, etc. Also, legacystore is still available, although 
deprecated.  Most users will want to wait until a point where they have 
no messages to perform an upgrade.

For those users which absolutely need to upgrade, there are a few basic 
manipulations of the store directory that can be performed by hand or 
script (as they will repeat for each queue present).  While this is 
simple, it is error-prone, and I don't recommend it unless absolutely 
necessary.  If you need this, I can provide a description of the steps 
needed.

The discussion on this point can be seen here; it also contains an 
informational description of the directory change:

http://mail-archives.apache.org/mod_mbox/qpid-users/201411.mbox/%3C1415731947.26261.81.camel%40localhost%3E

Hope that helps,
Kim

On 05/13/2015 03:36 PM, Matt Broadstone wrote:
> Hey again,
> Another issue I've run into switching to 0.32 is the fact that durable
> topics I create are no longer persisted between broker restarts. The 0.28
> release that was previously provided only had the legacy store, but 0.32
> has the linear store which I'm trying out.
>
> I provide a path to my store-dir and that's it, is there some other
> configuration option I'm missing here for this? The qpidd logs don't
> indicate any failure, and indicate that the linear-store is in fact being
> used. Running qpidd with debug+ shows a whole bunch of information, and I
> see the exchanges being added but no corresponding messages from the store
> (this may be expected).
>
> Regards,
> Matt
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org