You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@servicemix.apache.org by samg <sa...@wellogic.com> on 2007/06/21 23:56:55 UTC
JMS and recoverability (moved from User list)
(original thread here:
http://www.nabble.com/replies-to-JMS-and-recoverability-tf3431435s12049.html
)
Hello again,
I made the simple change in servicemix-jms necessary to use a permanent
queue rather than a temporary one for the replies of in-out message
exchanges. While this fixes the problem of not being able to find a deleted
temporary queue when restarting after a crash, there is (and I should have
seen this coming) another problem. The collection of pending exchanges in
the multiplexing provider processor is not persistent, and so of course even
though the response can be passed back through the queue, upon arriving back
at the provider processor its exchange is missing.
This is beginning to make me think I'm barking up the wrong tree; it seems
like the only way to make this work would be to not only persist whatever is
necessary to reconstruct the exchange, but also to guarantee that every
component involved in the workflow does the same (not an issue in my case,
but would it generalize?). There would also need to be a facility for
instantiating an exchange and telling it that it is actually not a new
exchange, but instead the continuation of (response to) an exchange that was
lost during an outage.
I'm not sure that this is a good approach.
Do other people who want this kind of reliability do it all with in-only
messages?
Regards,
Sam
--
View this message in context: http://www.nabble.com/JMS-and-recoverability-%28moved-from-User-list%29-tf3961576s12049.html#a11242457
Sent from the ServiceMix - Dev mailing list archive at Nabble.com.
Re: JMS and recoverability (moved from User list)
Posted by Guillaume Nodet <gn...@gmail.com>.
On 6/28/07, samg <sa...@wellogic.com> wrote:
>
> Hello again,
> I've made the changes to use Store instead of a Map in
> MultiplexingProviderProcessor and supporting classes. It defaults to
> MemoryStore if nothing is specified, and this seems to function the same way
> as with direct use of ConcurrentHashMap (which is what MemoryStore is
> inside).
> I have a couple of questions:
> 1) Do I have to run my own database if I want to use the JdbcStore? I
> couldn't find an example that uses it. If there's an embedded db, could
> someone point me at configuration instructions?
The JdbcStoreFactory must be given a DataSource. You should be able
to configure it using spring (with direct configuration or by looking for a jndi
object that you can configure in conf/jndi.xml).
> 2) I was going to change MultiplexingConsumerProcessor in the same way, but
> I see that it is storing not just the exchange but a Context object. A
> Context points at other things, so I'm guessing that it won't happily
> serialize into the db. The question then becomes: will it be possible to
> instead serialize the Exchange (and potentially some other data), and on
> retrieval use some other Context, and still achieve the same results?
> -Sam
>
Yeah, I guess it may work, though the context will try to access the input
message, but it may be avoided by storing the needed properties in a map
of something like that. I think it would be easier to make the context
persistent with only the needed informations.
Btw, it may be more interesting to work on the newer endpoints instead (3.2)
as they are much more powerful that the ones in 3.1. I'm currently working
on the InOut support.
>
> samg wrote:
> >
> > I added a property to the JmsEndpoint and modified the Multiplexing and
> > StandardProviderProcessor to let me specify a named queue for the reply to
> > the in-out message. It doesn't do any good without a fix to the other
> > problem though, so I thought I'd hold off on submitting a patch.
> > I took a look at the Store you mentioned; I think that could be a
> > solution, but would we have to persist the whole Exchange that way? That
> > could be expensive. Is one required to send the "in" message back when
> > replying, or could we drop that before persisting?
> > Is there anywhere else in ServiceMix's exchange handling that would have
> > to persist something in order to handle the reply message on its way back?
> > As you mentioned, putting the exchange on the JMS message could also work,
> > but probably not if the recipient is not another ServiceMix.. which I
> > think would surprise some users ;)
> > -Sam
> >
> >
> >
> > gnodet wrote:
> >>
> >> On 6/21/07, samg <sa...@wellogic.com> wrote:
> >>>
> >>>
> >>> (original thread here:
> >>>
> >>> http://www.nabble.com/replies-to-JMS-and-recoverability-tf3431435s12049.html
> >>> )
> >>>
> >>> Hello again,
> >>> I made the simple change in servicemix-jms necessary to use a permanent
> >>> queue rather than a temporary one for the replies of in-out message
> >>> exchanges.
> >>
> >>
> >> Which change ? It should just be a change in configuration.
> >> Else could you please create a JIRA and attach your patch ?
> >>
> >> While this fixes the problem of not being able to find a deleted
> >>> temporary queue when restarting after a crash, there is (and I should
> >>> have
> >>> seen this coming) another problem. The collection of pending exchanges
> >>> in
> >>> the multiplexing provider processor is not persistent, and so of course
> >>> even
> >>> though the response can be passed back through the queue, upon arriving
> >>> back
> >>> at the provider processor its exchange is missing.
> >>> This is beginning to make me think I'm barking up the wrong tree; it
> >>> seems
> >>> like the only way to make this work would be to not only persist
> >>> whatever
> >>> is
> >>> necessary to reconstruct the exchange, but also to guarantee that every
> >>> component involved in the workflow does the same (not an issue in my
> >>> case,
> >>> but would it generalize?). There would also need to be a facility for
> >>> instantiating an exchange and telling it that it is actually not a new
> >>> exchange, but instead the continuation of (response to) an exchange that
> >>> was
> >>> lost during an outage.
> >>> I'm not sure that this is a good approach.
> >>> Do other people who want this kind of reliability do it all with in-only
> >>> messages?
> >>
> >>
> >> Yeah, there are several components that have a state kept in memory which
> >> it not the best way to do things. What if the exchange itself is put
> >> inside a JMS
> >> property on the message ? Maybe as an optional storage. We can also use
> >> the Store from servicemix-services instead of a Map (that would be a
> >> first
> >> step).
> >> I think the JMS property could be used but this means that the consumer
> >> has
> >> to send
> >> back the exchange with the reply (which is not a problem if the consumer
> >> is
> >> another
> >> ServiceMix). Else using a store is the best appropriate thing to do I
> >> guess.
> >> Wdyt ?
> >>
> >>
> >> Regards,
> >>> Sam
> >>> --
> >>> View this message in context:
> >>> http://www.nabble.com/JMS-and-recoverability-%28moved-from-User-list%29-tf3961576s12049.html#a11242457
> >>> Sent from the ServiceMix - Dev mailing list archive at Nabble.com.
> >>>
> >>>
> >>
> >>
> >> --
> >> Cheers,
> >> Guillaume Nodet
> >> ------------------------
> >> Principal Engineer, IONA
> >> Blog: http://gnodet.blogspot.com/
> >>
> >>
> >
> >
>
> --
> View this message in context: http://www.nabble.com/JMS-and-recoverability-%28moved-from-User-list%29-tf3961576s12049.html#a11351720
> Sent from the ServiceMix - Dev mailing list archive at Nabble.com.
>
>
--
Cheers,
Guillaume Nodet
------------------------
Principal Engineer, IONA
Blog: http://gnodet.blogspot.com/
Re: JMS and recoverability (moved from User list)
Posted by samg <sa...@wellogic.com>.
Hello again,
I've made the changes to use Store instead of a Map in
MultiplexingProviderProcessor and supporting classes. It defaults to
MemoryStore if nothing is specified, and this seems to function the same way
as with direct use of ConcurrentHashMap (which is what MemoryStore is
inside).
I have a couple of questions:
1) Do I have to run my own database if I want to use the JdbcStore? I
couldn't find an example that uses it. If there's an embedded db, could
someone point me at configuration instructions?
2) I was going to change MultiplexingConsumerProcessor in the same way, but
I see that it is storing not just the exchange but a Context object. A
Context points at other things, so I'm guessing that it won't happily
serialize into the db. The question then becomes: will it be possible to
instead serialize the Exchange (and potentially some other data), and on
retrieval use some other Context, and still achieve the same results?
-Sam
samg wrote:
>
> I added a property to the JmsEndpoint and modified the Multiplexing and
> StandardProviderProcessor to let me specify a named queue for the reply to
> the in-out message. It doesn't do any good without a fix to the other
> problem though, so I thought I'd hold off on submitting a patch.
> I took a look at the Store you mentioned; I think that could be a
> solution, but would we have to persist the whole Exchange that way? That
> could be expensive. Is one required to send the "in" message back when
> replying, or could we drop that before persisting?
> Is there anywhere else in ServiceMix's exchange handling that would have
> to persist something in order to handle the reply message on its way back?
> As you mentioned, putting the exchange on the JMS message could also work,
> but probably not if the recipient is not another ServiceMix.. which I
> think would surprise some users ;)
> -Sam
>
>
>
> gnodet wrote:
>>
>> On 6/21/07, samg <sa...@wellogic.com> wrote:
>>>
>>>
>>> (original thread here:
>>>
>>> http://www.nabble.com/replies-to-JMS-and-recoverability-tf3431435s12049.html
>>> )
>>>
>>> Hello again,
>>> I made the simple change in servicemix-jms necessary to use a permanent
>>> queue rather than a temporary one for the replies of in-out message
>>> exchanges.
>>
>>
>> Which change ? It should just be a change in configuration.
>> Else could you please create a JIRA and attach your patch ?
>>
>> While this fixes the problem of not being able to find a deleted
>>> temporary queue when restarting after a crash, there is (and I should
>>> have
>>> seen this coming) another problem. The collection of pending exchanges
>>> in
>>> the multiplexing provider processor is not persistent, and so of course
>>> even
>>> though the response can be passed back through the queue, upon arriving
>>> back
>>> at the provider processor its exchange is missing.
>>> This is beginning to make me think I'm barking up the wrong tree; it
>>> seems
>>> like the only way to make this work would be to not only persist
>>> whatever
>>> is
>>> necessary to reconstruct the exchange, but also to guarantee that every
>>> component involved in the workflow does the same (not an issue in my
>>> case,
>>> but would it generalize?). There would also need to be a facility for
>>> instantiating an exchange and telling it that it is actually not a new
>>> exchange, but instead the continuation of (response to) an exchange that
>>> was
>>> lost during an outage.
>>> I'm not sure that this is a good approach.
>>> Do other people who want this kind of reliability do it all with in-only
>>> messages?
>>
>>
>> Yeah, there are several components that have a state kept in memory which
>> it not the best way to do things. What if the exchange itself is put
>> inside a JMS
>> property on the message ? Maybe as an optional storage. We can also use
>> the Store from servicemix-services instead of a Map (that would be a
>> first
>> step).
>> I think the JMS property could be used but this means that the consumer
>> has
>> to send
>> back the exchange with the reply (which is not a problem if the consumer
>> is
>> another
>> ServiceMix). Else using a store is the best appropriate thing to do I
>> guess.
>> Wdyt ?
>>
>>
>> Regards,
>>> Sam
>>> --
>>> View this message in context:
>>> http://www.nabble.com/JMS-and-recoverability-%28moved-from-User-list%29-tf3961576s12049.html#a11242457
>>> Sent from the ServiceMix - Dev mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>> --
>> Cheers,
>> Guillaume Nodet
>> ------------------------
>> Principal Engineer, IONA
>> Blog: http://gnodet.blogspot.com/
>>
>>
>
>
--
View this message in context: http://www.nabble.com/JMS-and-recoverability-%28moved-from-User-list%29-tf3961576s12049.html#a11351720
Sent from the ServiceMix - Dev mailing list archive at Nabble.com.
Re: JMS and recoverability (moved from User list)
Posted by samg <sa...@wellogic.com>.
I added a property to the JmsEndpoint and modified the Multiplexing and
StandardProviderProcessor to let me specify a named queue for the reply to
the in-out message. It doesn't do any good without a fix to the other
problem though, so I thought I'd hold off on submitting a patch.
I took a look at the Store you mentioned; I think that could be a solution,
but would we have to persist the whole Exchange that way? That could be
expensive. Is one required to send the "in" message back when replying, or
could we drop that before persisting?
Is there anywhere else in ServiceMix's exchange handling that would have to
persist something in order to handle the reply message on its way back?
As you mentioned, putting the exchange on the JMS message could also work,
but probably not if the recipient is not another ServiceMix.. which I think
would surprise some users ;)
-Sam
gnodet wrote:
>
> On 6/21/07, samg <sa...@wellogic.com> wrote:
>>
>>
>> (original thread here:
>>
>> http://www.nabble.com/replies-to-JMS-and-recoverability-tf3431435s12049.html
>> )
>>
>> Hello again,
>> I made the simple change in servicemix-jms necessary to use a permanent
>> queue rather than a temporary one for the replies of in-out message
>> exchanges.
>
>
> Which change ? It should just be a change in configuration.
> Else could you please create a JIRA and attach your patch ?
>
> While this fixes the problem of not being able to find a deleted
>> temporary queue when restarting after a crash, there is (and I should
>> have
>> seen this coming) another problem. The collection of pending exchanges
>> in
>> the multiplexing provider processor is not persistent, and so of course
>> even
>> though the response can be passed back through the queue, upon arriving
>> back
>> at the provider processor its exchange is missing.
>> This is beginning to make me think I'm barking up the wrong tree; it
>> seems
>> like the only way to make this work would be to not only persist whatever
>> is
>> necessary to reconstruct the exchange, but also to guarantee that every
>> component involved in the workflow does the same (not an issue in my
>> case,
>> but would it generalize?). There would also need to be a facility for
>> instantiating an exchange and telling it that it is actually not a new
>> exchange, but instead the continuation of (response to) an exchange that
>> was
>> lost during an outage.
>> I'm not sure that this is a good approach.
>> Do other people who want this kind of reliability do it all with in-only
>> messages?
>
>
> Yeah, there are several components that have a state kept in memory which
> it not the best way to do things. What if the exchange itself is put
> inside a JMS
> property on the message ? Maybe as an optional storage. We can also use
> the Store from servicemix-services instead of a Map (that would be a first
> step).
> I think the JMS property could be used but this means that the consumer
> has
> to send
> back the exchange with the reply (which is not a problem if the consumer
> is
> another
> ServiceMix). Else using a store is the best appropriate thing to do I
> guess.
> Wdyt ?
>
>
> Regards,
>> Sam
>> --
>> View this message in context:
>> http://www.nabble.com/JMS-and-recoverability-%28moved-from-User-list%29-tf3961576s12049.html#a11242457
>> Sent from the ServiceMix - Dev mailing list archive at Nabble.com.
>>
>>
>
>
> --
> Cheers,
> Guillaume Nodet
> ------------------------
> Principal Engineer, IONA
> Blog: http://gnodet.blogspot.com/
>
>
--
View this message in context: http://www.nabble.com/JMS-and-recoverability-%28moved-from-User-list%29-tf3961576s12049.html#a11307004
Sent from the ServiceMix - Dev mailing list archive at Nabble.com.
Re: JMS and recoverability (moved from User list)
Posted by Guillaume Nodet <gn...@gmail.com>.
On 6/21/07, samg <sa...@wellogic.com> wrote:
>
>
> (original thread here:
>
> http://www.nabble.com/replies-to-JMS-and-recoverability-tf3431435s12049.html
> )
>
> Hello again,
> I made the simple change in servicemix-jms necessary to use a permanent
> queue rather than a temporary one for the replies of in-out message
> exchanges.
Which change ? It should just be a change in configuration.
Else could you please create a JIRA and attach your patch ?
While this fixes the problem of not being able to find a deleted
> temporary queue when restarting after a crash, there is (and I should have
> seen this coming) another problem. The collection of pending exchanges in
> the multiplexing provider processor is not persistent, and so of course
> even
> though the response can be passed back through the queue, upon arriving
> back
> at the provider processor its exchange is missing.
> This is beginning to make me think I'm barking up the wrong tree; it seems
> like the only way to make this work would be to not only persist whatever
> is
> necessary to reconstruct the exchange, but also to guarantee that every
> component involved in the workflow does the same (not an issue in my case,
> but would it generalize?). There would also need to be a facility for
> instantiating an exchange and telling it that it is actually not a new
> exchange, but instead the continuation of (response to) an exchange that
> was
> lost during an outage.
> I'm not sure that this is a good approach.
> Do other people who want this kind of reliability do it all with in-only
> messages?
Yeah, there are several components that have a state kept in memory which
it not the best way to do things. What if the exchange itself is put
inside a JMS
property on the message ? Maybe as an optional storage. We can also use
the Store from servicemix-services instead of a Map (that would be a first
step).
I think the JMS property could be used but this means that the consumer has
to send
back the exchange with the reply (which is not a problem if the consumer is
another
ServiceMix). Else using a store is the best appropriate thing to do I
guess.
Wdyt ?
Regards,
> Sam
> --
> View this message in context:
> http://www.nabble.com/JMS-and-recoverability-%28moved-from-User-list%29-tf3961576s12049.html#a11242457
> Sent from the ServiceMix - Dev mailing list archive at Nabble.com.
>
>
--
Cheers,
Guillaume Nodet
------------------------
Principal Engineer, IONA
Blog: http://gnodet.blogspot.com/