You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by Dejan Bosanac <de...@nighttale.net> on 2007/12/14 12:46:57 UTC
Message transforming on broker
I was thinking recently if there should be a configurable support for
message transforming on the broker level. For example, one use case
would be to do XML/Object transformation of all messages
received/transmitted through the Stomp protocol. I guess now that
Camel is integrated in AMQ
--
Dejan Bosanac
www.scriptinginjava.net
Re: Message transforming on broker
Posted by Dejan Bosanac <de...@nighttale.net>.
Hi,
I've documented a stomp message transformation feature (the part that is
currently committed)
http://cwiki.apache.org/confluence/display/ACTIVEMQ/Stomp#Stomp-Messagetransformations
We'll implement the functionality in PHP client soon, so it would be good if
someone could look at the other patch (
https://issues.apache.org/activemq/browse/AMQ-1567) that includes map
messages, xstream customization, transformation errors , etc. It would be
great if have that committed, documented and implemented in the PHP client.
If there's anything that should be modified there, please let me know.
Thanks
--
Dejan Bosanac
www.scriptinginjava.net
On Wed, Jan 23, 2008 at 10:07 PM, Dejan Bosanac <de...@nighttale.net> wrote:
> > Patches applied Thanks! Keep em coming :)
>
> Here it comes :) ...
> https://issues.apache.org/activemq/browse/AMQ-1567 ... It's one big
> cumulative patch, if it easier for you I can break it into few
> separate patches.
>
>
> > That sounds ok to me. Perhaps we should add a header to the legacy
> > frame to that folks know that the transformation failed either on the
> > send or subscriber side.
>
> I've added "transformation-error" header that contains a message of
> the exception threw. Now consumer can check for that header before it
> assumes that transformation took place.
>
> > The one fishy thing is that it sounds like it's being used like a
> > content-type. Perhaps we should make the "transformation" purely a
> > transient header that only controls sender / subscriber behavior.
>
> Fixed.
>
> > I'd like to work on making the default jms-xml transformation support
> > all the JMS message types, especially MapMessage.
>
> Now the transformer supports Object, Map and Byte messages. I've
> changed header semantics a bit, so instead of jms-xml we now use
> jmx-object-xml. Current translator supports the following header
> values
>
> jms-byte
> jms-object-xml
> jms-object-json
> jms-map-xml
> jms-map-json
>
> I've also renamed the class from XStreamFrameTranslator to
> JmsFrameTranslator, since it kind of do translation of standard jsm
> messages, the thing it uses XStream for that is not that important. I
> didn't make OXM transformer, but if there is an interest it could be
> easily created, just like appropriate message transformer
> (http://issues.apache.org/activemq/browse/AMQ-1499)
>
> * Spring support
>
> Now XStream could be configured in the application context (just as it
> has been used in test cases). I made a simple XStreamFactoryBean for
> this purpose.
>
> I think this should be complete now. I have a few minor tweaking on
> todo list, but they are not critical. If this is OK, I would first
> concentrate on the documentation.
> A few questions about the documentation: should this go into the
> specification or is it a AMQ-specific feature? If it should be
> documented in the protocol, should we freeze 1.0 as it is and make a
> 1.1 with this as extra (so that current clients could remain 1.0
> compatible)?
>
>
> --
> Dejan Bosanac
> www.scriptinginjava.net
>
Re: Message transforming on broker
Posted by Dejan Bosanac <de...@nighttale.net>.
> Patches applied Thanks! Keep em coming :)
Here it comes :) ...
https://issues.apache.org/activemq/browse/AMQ-1567 ... It's one big
cumulative patch, if it easier for you I can break it into few
separate patches.
> That sounds ok to me. Perhaps we should add a header to the legacy
> frame to that folks know that the transformation failed either on the
> send or subscriber side.
I've added "transformation-error" header that contains a message of
the exception threw. Now consumer can check for that header before it
assumes that transformation took place.
> The one fishy thing is that it sounds like it's being used like a
> content-type. Perhaps we should make the "transformation" purely a
> transient header that only controls sender / subscriber behavior.
Fixed.
> I'd like to work on making the default jms-xml transformation support
> all the JMS message types, especially MapMessage.
Now the transformer supports Object, Map and Byte messages. I've
changed header semantics a bit, so instead of jms-xml we now use
jmx-object-xml. Current translator supports the following header
values
jms-byte
jms-object-xml
jms-object-json
jms-map-xml
jms-map-json
I've also renamed the class from XStreamFrameTranslator to
JmsFrameTranslator, since it kind of do translation of standard jsm
messages, the thing it uses XStream for that is not that important. I
didn't make OXM transformer, but if there is an interest it could be
easily created, just like appropriate message transformer
(http://issues.apache.org/activemq/browse/AMQ-1499)
* Spring support
Now XStream could be configured in the application context (just as it
has been used in test cases). I made a simple XStreamFactoryBean for
this purpose.
I think this should be complete now. I have a few minor tweaking on
todo list, but they are not critical. If this is OK, I would first
concentrate on the documentation.
A few questions about the documentation: should this go into the
specification or is it a AMQ-specific feature? If it should be
documented in the protocol, should we freeze 1.0 as it is and make a
1.1 with this as extra (so that current clients could remain 1.0
compatible)?
--
Dejan Bosanac
www.scriptinginjava.net
Re: Message transforming on broker
Posted by Hiram Chirino <hi...@hiramchirino.com>.
Patches applied Thanks! Keep em coming :)
On Jan 16, 2008 3:40 AM, Dejan Bosanac <de...@nighttale.net> wrote:
> Hi Hiram
>
> On Jan 15, 2008 8:42 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > > - For JSON conversion to work we need to use 1.2.2 version of XStream,
> > > which is currently hardcoded in the pom, but if we apply this change
> > > (https://issues.apache.org/activemq/browse/AMQ-1493) we can use it as
> > > normal.
> > >
> >
> > Sure.. We can get those applied. BTW when you submitted those patches
> > you did not click the check box that said your granting the ASF a lic.
> > to use your patch. If you get a chance please reattach the patches
> > with those check boxes clicked.
>
> I've just uploaded AMQ-1493 patch again with the grant, the AMQ-943
> had the license in the first place so it should be all good now. Sorry
> about that.
>
>
> >
> > > - the implementation ignores any errors that could happen during
> > > transformation (unknown translator, wrong xml, etc) and uses legacy
> > > translator in those cases. I think this is desired behavior, but if
> > > you think different it could be easily changed.
> >
> > That sounds ok to me. Perhaps we should add a header to the legacy
> > frame to that folks know that the transformation failed either on the
> > send or subscriber side.
>
> That's great idea.
>
> >
> > >
> > > - the transformation is done when user sends or subscribes with the
> > > "transformation" header. Also, if the JMS client sends a message with
> > > "transformation" header set it will be transformed before delivered to
> > > the STOMP client even if it has not subscribed with the
> > > "transformation" header. The subscription "transformation" has a
> > > priority against message "transformation" in case that both are set
> > >
> >
> > The one fishy thing is that it sounds like it's being used like a
> > content-type. Perhaps we should make the "transformation" purely a
> > transient header that only controls sender / subscriber behavior.
>
> Makes sense. Will change that.
>
> > > - the open question is how to add a support configuration of XStream
> > > (and other marshallers) in the Spring context. The current
> > > implementation is nice, but to be really useful people needs (or at
> > > least I do) to have configured XStream to handle desired XML(JSON)
> > > format. The transport layer is not easily exposed to the spring
> > > application context. I found one solution that could work, but is not
> > > so elegant.
> > >
> >
> > This sounds fine to me! Please make it a separate patch.
>
> Great.
>
> > Oh an BTW.. once we get this committed..
> >
> > I'd like to work on making the default jms-xml transformation support
> > all the JMS message types, especially MapMessage.
>
> Definitely, converting arrays, hashtables, etc to map messages would
> be very useful.
>
> I think it would be best to commit these patch and then I'll continue
> to work further on Spring support, other transporters and message
> types from there. I'll also document it in the AMQ and Stomp wikis and
> create a "reference implementation" in a PHP client.
>
>
> Thanks
> --
> Dejan Bosanac
> www.scriptinginjava.net
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
Open Source SOA
http://open.iona.com
Re: Message transforming on broker
Posted by Dejan Bosanac <de...@nighttale.net>.
Hi Hiram
On Jan 15, 2008 8:42 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > - For JSON conversion to work we need to use 1.2.2 version of XStream,
> > which is currently hardcoded in the pom, but if we apply this change
> > (https://issues.apache.org/activemq/browse/AMQ-1493) we can use it as
> > normal.
> >
>
> Sure.. We can get those applied. BTW when you submitted those patches
> you did not click the check box that said your granting the ASF a lic.
> to use your patch. If you get a chance please reattach the patches
> with those check boxes clicked.
I've just uploaded AMQ-1493 patch again with the grant, the AMQ-943
had the license in the first place so it should be all good now. Sorry
about that.
>
> > - the implementation ignores any errors that could happen during
> > transformation (unknown translator, wrong xml, etc) and uses legacy
> > translator in those cases. I think this is desired behavior, but if
> > you think different it could be easily changed.
>
> That sounds ok to me. Perhaps we should add a header to the legacy
> frame to that folks know that the transformation failed either on the
> send or subscriber side.
That's great idea.
>
> >
> > - the transformation is done when user sends or subscribes with the
> > "transformation" header. Also, if the JMS client sends a message with
> > "transformation" header set it will be transformed before delivered to
> > the STOMP client even if it has not subscribed with the
> > "transformation" header. The subscription "transformation" has a
> > priority against message "transformation" in case that both are set
> >
>
> The one fishy thing is that it sounds like it's being used like a
> content-type. Perhaps we should make the "transformation" purely a
> transient header that only controls sender / subscriber behavior.
Makes sense. Will change that.
> > - the open question is how to add a support configuration of XStream
> > (and other marshallers) in the Spring context. The current
> > implementation is nice, but to be really useful people needs (or at
> > least I do) to have configured XStream to handle desired XML(JSON)
> > format. The transport layer is not easily exposed to the spring
> > application context. I found one solution that could work, but is not
> > so elegant.
> >
>
> This sounds fine to me! Please make it a separate patch.
Great.
> Oh an BTW.. once we get this committed..
>
> I'd like to work on making the default jms-xml transformation support
> all the JMS message types, especially MapMessage.
Definitely, converting arrays, hashtables, etc to map messages would
be very useful.
I think it would be best to commit these patch and then I'll continue
to work further on Spring support, other transporters and message
types from there. I'll also document it in the AMQ and Stomp wikis and
create a "reference implementation" in a PHP client.
Thanks
--
Dejan Bosanac
www.scriptinginjava.net
Re: Message transforming on broker
Posted by Hiram Chirino <hi...@hiramchirino.com>.
Oh an BTW.. once we get this committed..
I'd like to work on making the default jms-xml transformation support
all the JMS message types, especially MapMessage.
On Jan 15, 2008 2:42 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
> Awesome!
>
> On Jan 15, 2008 6:47 AM, Dejan Bosanac <de...@nighttale.net> wrote:
> > I've finally got my hands on this and implemented it in a way we
> > discussed it here. Patch is uploaded to the old issue
> > (https://issues.apache.org/activemq/browse/AMQ-943).
> >
> > Here are a couple of notes and questions:
> >
> > - I've used the "transformation" header to lookup the appropriate
> > frame translator
> >
> > - In the current implementation there is a XStreamFrameTranslator
> > which handles basic XML and JSON transformations
>
> Perfect!
>
> >
> > - For JSON conversion to work we need to use 1.2.2 version of XStream,
> > which is currently hardcoded in the pom, but if we apply this change
> > (https://issues.apache.org/activemq/browse/AMQ-1493) we can use it as
> > normal.
> >
>
> Sure.. We can get those applied. BTW when you submitted those patches
> you did not click the check box that said your granting the ASF a lic.
> to use your patch. If you get a chance please reattach the patches
> with those check boxes clicked.
>
> > - the implementation ignores any errors that could happen during
> > transformation (unknown translator, wrong xml, etc) and uses legacy
> > translator in those cases. I think this is desired behavior, but if
> > you think different it could be easily changed.
>
> That sounds ok to me. Perhaps we should add a header to the legacy
> frame to that folks know that the transformation failed either on the
> send or subscriber side.
>
> >
> > - the transformation is done when user sends or subscribes with the
> > "transformation" header. Also, if the JMS client sends a message with
> > "transformation" header set it will be transformed before delivered to
> > the STOMP client even if it has not subscribed with the
> > "transformation" header. The subscription "transformation" has a
> > priority against message "transformation" in case that both are set
> >
>
> The one fishy thing is that it sounds like it's being used like a
> content-type. Perhaps we should make the "transformation" purely a
> transient header that only controls sender / subscriber behavior.
>
> > - the open question is how to add a support configuration of XStream
> > (and other marshallers) in the Spring context. The current
> > implementation is nice, but to be really useful people needs (or at
> > least I do) to have configured XStream to handle desired XML(JSON)
> > format. The transport layer is not easily exposed to the spring
> > application context. I found one solution that could work, but is not
> > so elegant.
> >
>
> This sounds fine to me! Please make it a separate patch.
>
>
> > 1. make XBeanBrokerService ApplicationContextAware
> > 2. make StompTransportFactory BrokerServiceAware
> > 3. when creating transports if the transport factory implements
> > BrokerServiceAware it is passed to it
> > 4. then if the broker service is application context aware we can
> > extract it in the factory.
> >
> > any better ideas? When we sort this out, we can make OXM translator
> > that could use well configured XStream, Castor, XMLBeans, etc.
> >
> > All comments are welcomed.
> >
> > Thanks
> > --
> > Dejan Bosanac
> > www.scriptinginjava.net
> >
> >
> >
> > On Dec 15, 2007 6:59 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > >
> > > On Dec 15, 2007 11:11 AM, Dejan Bosanac <de...@nighttale.net> wrote:
> > > > Hi Hiram
> > > >
> > > > On Dec 14, 2007 6:40 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > > > > I personally think that a STOMP client should request a transformation
> > > > > strategy. For example on a subscribe, he could do
> > > > >
> > > > > SUBSCRIBE
> > > > > destination: /queue/foo
> > > > > transformation: jms-json
> > > > > ack: client
> > > > >
> > > > > And on the reverse side, The send could look like:
> > > > >
> > > > > SEND
> > > > > destination:/queue/a
> > > > > transformation: jms-json
> > > > >
> > > > > ["Hello World",1]
> > > > > ^@
> > > > >
> > > > > And have the Stomp transport handle those "transformation" headers by
> > > > > looking up the "jms-json" transformation in the spring context. This
> > > > > would allow the broker to have extensible transformations. And the
> > > > > looked up object would just need to implement the following interface:
> > > > >
> > > > > http://activemq.apache.org/maven/activemq-core/apidocs/org/apache/activemq/transport/stomp/FrameTranslator.html
> > > >
> > > > That's great. This idea has been discussed on the list some time ago
> > > > and I think it's a good approach. I though that maybe it could be
> > > > useful for other transports as well, but you're probably right that we
> > > > should keep this Stomp-related for now.
> > > >
> > > > A few questions about your idea:
> > > >
> > > > * Maybe it would be better to have "transfomers" registered at boot
> > > > time, rather then doing lookups when message arrives? If so, there has
> > > > to be a mechanism to configure Stomp transport with transformers that
> > > > will be used. What method do you propose for this, if any? Maybe
> > > > StompTransportFactory could do lookup on init?
> > >
> > > Well the one up side to doing the lookup per send is that the sender
> > > can choose different transformation strategies depending on the
> > > message he is sending. Perhaps he got a JSON message in, so he sends
> > > using the json transformation. Perhaps he got a XML message so he
> > > sends it using an XML transformation etc.
> > >
> > > The look up cost should be cheap but if you think perhaps it's an
> > > expensive operation, we could cache lookups. The easiest way to make
> > > this extensible and easy to get stuff registered for lookup is to use
> > > the FactoryFinder [1] to lookup implementations by searching the
> > > META-INF directories. For examples how we use this see the impl of
> > > DiscoveryAgentFactory [2]
> > >
> > > [1] http://activemq.apache.org/maven/activemq-core/apidocs/org/apache/activemq/util/FactoryFinder.html
> > > [2] https://svn.apache.org/repos/asf/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/transport/discovery/DiscoveryAgentFactory.java
> > > >
> > > > * We have to allow users to add their "domain" classes to the amq
> > > > classpath, as I guess the most of them will want to transform XML/JSON
> > > > to objects of those classes. That's not a problem for brokers embedded
> > > > in applications, but I guess they could do that in the off-the-shelf
> > > > broker by adding appropriate JAR to the lib/ folder?
> > > >
> > >
> > > Yep that's the standard way we expand functionality in the broker.
> > >
> > > >
> > > > >
> > > > > Just need to find someone who is willing to take a stab at enhancing
> > > > > the stomp transport so that it does those "transfromation" header
> > > > > handling logic. And add a few more interesting implementations of
> > > > > FrameTranslator.
> > > >
> > > > No problem about that. I'll invest some time in this, both in amq and
> > > > PHP Stomp client. But before that I thought I handle authentication
> > > > for Stomp as it is far more important IMHO. I'll start with this patch
> > > > (https://issues.apache.org/activemq/browse/AMQ-1272) add test cases
> > > > and implement proper behavior.
> > > >
> > >
> > > That's awesome!
> > >
> > > >
> > > > Cheers
> > > > --
> > > > Dejan Bosanac
> > > > www.scriptinginjava.net
> > > >
> > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Hiram
> > >
> > > Blog: http://hiramchirino.com
> > >
> > > Open Source SOA
> > > http://open.iona.com
> > >
> >
>
>
>
> --
>
> Regards,
> Hiram
>
> Blog: http://hiramchirino.com
>
> Open Source SOA
> http://open.iona.com
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
Open Source SOA
http://open.iona.com
Re: Message transforming on broker
Posted by Hiram Chirino <hi...@hiramchirino.com>.
Awesome!
On Jan 15, 2008 6:47 AM, Dejan Bosanac <de...@nighttale.net> wrote:
> I've finally got my hands on this and implemented it in a way we
> discussed it here. Patch is uploaded to the old issue
> (https://issues.apache.org/activemq/browse/AMQ-943).
>
> Here are a couple of notes and questions:
>
> - I've used the "transformation" header to lookup the appropriate
> frame translator
>
> - In the current implementation there is a XStreamFrameTranslator
> which handles basic XML and JSON transformations
Perfect!
>
> - For JSON conversion to work we need to use 1.2.2 version of XStream,
> which is currently hardcoded in the pom, but if we apply this change
> (https://issues.apache.org/activemq/browse/AMQ-1493) we can use it as
> normal.
>
Sure.. We can get those applied. BTW when you submitted those patches
you did not click the check box that said your granting the ASF a lic.
to use your patch. If you get a chance please reattach the patches
with those check boxes clicked.
> - the implementation ignores any errors that could happen during
> transformation (unknown translator, wrong xml, etc) and uses legacy
> translator in those cases. I think this is desired behavior, but if
> you think different it could be easily changed.
That sounds ok to me. Perhaps we should add a header to the legacy
frame to that folks know that the transformation failed either on the
send or subscriber side.
>
> - the transformation is done when user sends or subscribes with the
> "transformation" header. Also, if the JMS client sends a message with
> "transformation" header set it will be transformed before delivered to
> the STOMP client even if it has not subscribed with the
> "transformation" header. The subscription "transformation" has a
> priority against message "transformation" in case that both are set
>
The one fishy thing is that it sounds like it's being used like a
content-type. Perhaps we should make the "transformation" purely a
transient header that only controls sender / subscriber behavior.
> - the open question is how to add a support configuration of XStream
> (and other marshallers) in the Spring context. The current
> implementation is nice, but to be really useful people needs (or at
> least I do) to have configured XStream to handle desired XML(JSON)
> format. The transport layer is not easily exposed to the spring
> application context. I found one solution that could work, but is not
> so elegant.
>
This sounds fine to me! Please make it a separate patch.
> 1. make XBeanBrokerService ApplicationContextAware
> 2. make StompTransportFactory BrokerServiceAware
> 3. when creating transports if the transport factory implements
> BrokerServiceAware it is passed to it
> 4. then if the broker service is application context aware we can
> extract it in the factory.
>
> any better ideas? When we sort this out, we can make OXM translator
> that could use well configured XStream, Castor, XMLBeans, etc.
>
> All comments are welcomed.
>
> Thanks
> --
> Dejan Bosanac
> www.scriptinginjava.net
>
>
>
> On Dec 15, 2007 6:59 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
> >
> > On Dec 15, 2007 11:11 AM, Dejan Bosanac <de...@nighttale.net> wrote:
> > > Hi Hiram
> > >
> > > On Dec 14, 2007 6:40 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > > > I personally think that a STOMP client should request a transformation
> > > > strategy. For example on a subscribe, he could do
> > > >
> > > > SUBSCRIBE
> > > > destination: /queue/foo
> > > > transformation: jms-json
> > > > ack: client
> > > >
> > > > And on the reverse side, The send could look like:
> > > >
> > > > SEND
> > > > destination:/queue/a
> > > > transformation: jms-json
> > > >
> > > > ["Hello World",1]
> > > > ^@
> > > >
> > > > And have the Stomp transport handle those "transformation" headers by
> > > > looking up the "jms-json" transformation in the spring context. This
> > > > would allow the broker to have extensible transformations. And the
> > > > looked up object would just need to implement the following interface:
> > > >
> > > > http://activemq.apache.org/maven/activemq-core/apidocs/org/apache/activemq/transport/stomp/FrameTranslator.html
> > >
> > > That's great. This idea has been discussed on the list some time ago
> > > and I think it's a good approach. I though that maybe it could be
> > > useful for other transports as well, but you're probably right that we
> > > should keep this Stomp-related for now.
> > >
> > > A few questions about your idea:
> > >
> > > * Maybe it would be better to have "transfomers" registered at boot
> > > time, rather then doing lookups when message arrives? If so, there has
> > > to be a mechanism to configure Stomp transport with transformers that
> > > will be used. What method do you propose for this, if any? Maybe
> > > StompTransportFactory could do lookup on init?
> >
> > Well the one up side to doing the lookup per send is that the sender
> > can choose different transformation strategies depending on the
> > message he is sending. Perhaps he got a JSON message in, so he sends
> > using the json transformation. Perhaps he got a XML message so he
> > sends it using an XML transformation etc.
> >
> > The look up cost should be cheap but if you think perhaps it's an
> > expensive operation, we could cache lookups. The easiest way to make
> > this extensible and easy to get stuff registered for lookup is to use
> > the FactoryFinder [1] to lookup implementations by searching the
> > META-INF directories. For examples how we use this see the impl of
> > DiscoveryAgentFactory [2]
> >
> > [1] http://activemq.apache.org/maven/activemq-core/apidocs/org/apache/activemq/util/FactoryFinder.html
> > [2] https://svn.apache.org/repos/asf/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/transport/discovery/DiscoveryAgentFactory.java
> > >
> > > * We have to allow users to add their "domain" classes to the amq
> > > classpath, as I guess the most of them will want to transform XML/JSON
> > > to objects of those classes. That's not a problem for brokers embedded
> > > in applications, but I guess they could do that in the off-the-shelf
> > > broker by adding appropriate JAR to the lib/ folder?
> > >
> >
> > Yep that's the standard way we expand functionality in the broker.
> >
> > >
> > > >
> > > > Just need to find someone who is willing to take a stab at enhancing
> > > > the stomp transport so that it does those "transfromation" header
> > > > handling logic. And add a few more interesting implementations of
> > > > FrameTranslator.
> > >
> > > No problem about that. I'll invest some time in this, both in amq and
> > > PHP Stomp client. But before that I thought I handle authentication
> > > for Stomp as it is far more important IMHO. I'll start with this patch
> > > (https://issues.apache.org/activemq/browse/AMQ-1272) add test cases
> > > and implement proper behavior.
> > >
> >
> > That's awesome!
> >
> > >
> > > Cheers
> > > --
> > > Dejan Bosanac
> > > www.scriptinginjava.net
> > >
> >
> >
> >
> >
> > --
> > Regards,
> > Hiram
> >
> > Blog: http://hiramchirino.com
> >
> > Open Source SOA
> > http://open.iona.com
> >
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
Open Source SOA
http://open.iona.com
Re: Message transforming on broker
Posted by Dejan Bosanac <de...@nighttale.net>.
I've finally got my hands on this and implemented it in a way we
discussed it here. Patch is uploaded to the old issue
(https://issues.apache.org/activemq/browse/AMQ-943).
Here are a couple of notes and questions:
- I've used the "transformation" header to lookup the appropriate
frame translator
- In the current implementation there is a XStreamFrameTranslator
which handles basic XML and JSON transformations
- For JSON conversion to work we need to use 1.2.2 version of XStream,
which is currently hardcoded in the pom, but if we apply this change
(https://issues.apache.org/activemq/browse/AMQ-1493) we can use it as
normal.
- the implementation ignores any errors that could happen during
transformation (unknown translator, wrong xml, etc) and uses legacy
translator in those cases. I think this is desired behavior, but if
you think different it could be easily changed.
- the transformation is done when user sends or subscribes with the
"transformation" header. Also, if the JMS client sends a message with
"transformation" header set it will be transformed before delivered to
the STOMP client even if it has not subscribed with the
"transformation" header. The subscription "transformation" has a
priority against message "transformation" in case that both are set
- the open question is how to add a support configuration of XStream
(and other marshallers) in the Spring context. The current
implementation is nice, but to be really useful people needs (or at
least I do) to have configured XStream to handle desired XML(JSON)
format. The transport layer is not easily exposed to the spring
application context. I found one solution that could work, but is not
so elegant.
1. make XBeanBrokerService ApplicationContextAware
2. make StompTransportFactory BrokerServiceAware
3. when creating transports if the transport factory implements
BrokerServiceAware it is passed to it
4. then if the broker service is application context aware we can
extract it in the factory.
any better ideas? When we sort this out, we can make OXM translator
that could use well configured XStream, Castor, XMLBeans, etc.
All comments are welcomed.
Thanks
--
Dejan Bosanac
www.scriptinginjava.net
On Dec 15, 2007 6:59 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
>
> On Dec 15, 2007 11:11 AM, Dejan Bosanac <de...@nighttale.net> wrote:
> > Hi Hiram
> >
> > On Dec 14, 2007 6:40 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > > I personally think that a STOMP client should request a transformation
> > > strategy. For example on a subscribe, he could do
> > >
> > > SUBSCRIBE
> > > destination: /queue/foo
> > > transformation: jms-json
> > > ack: client
> > >
> > > And on the reverse side, The send could look like:
> > >
> > > SEND
> > > destination:/queue/a
> > > transformation: jms-json
> > >
> > > ["Hello World",1]
> > > ^@
> > >
> > > And have the Stomp transport handle those "transformation" headers by
> > > looking up the "jms-json" transformation in the spring context. This
> > > would allow the broker to have extensible transformations. And the
> > > looked up object would just need to implement the following interface:
> > >
> > > http://activemq.apache.org/maven/activemq-core/apidocs/org/apache/activemq/transport/stomp/FrameTranslator.html
> >
> > That's great. This idea has been discussed on the list some time ago
> > and I think it's a good approach. I though that maybe it could be
> > useful for other transports as well, but you're probably right that we
> > should keep this Stomp-related for now.
> >
> > A few questions about your idea:
> >
> > * Maybe it would be better to have "transfomers" registered at boot
> > time, rather then doing lookups when message arrives? If so, there has
> > to be a mechanism to configure Stomp transport with transformers that
> > will be used. What method do you propose for this, if any? Maybe
> > StompTransportFactory could do lookup on init?
>
> Well the one up side to doing the lookup per send is that the sender
> can choose different transformation strategies depending on the
> message he is sending. Perhaps he got a JSON message in, so he sends
> using the json transformation. Perhaps he got a XML message so he
> sends it using an XML transformation etc.
>
> The look up cost should be cheap but if you think perhaps it's an
> expensive operation, we could cache lookups. The easiest way to make
> this extensible and easy to get stuff registered for lookup is to use
> the FactoryFinder [1] to lookup implementations by searching the
> META-INF directories. For examples how we use this see the impl of
> DiscoveryAgentFactory [2]
>
> [1] http://activemq.apache.org/maven/activemq-core/apidocs/org/apache/activemq/util/FactoryFinder.html
> [2] https://svn.apache.org/repos/asf/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/transport/discovery/DiscoveryAgentFactory.java
> >
> > * We have to allow users to add their "domain" classes to the amq
> > classpath, as I guess the most of them will want to transform XML/JSON
> > to objects of those classes. That's not a problem for brokers embedded
> > in applications, but I guess they could do that in the off-the-shelf
> > broker by adding appropriate JAR to the lib/ folder?
> >
>
> Yep that's the standard way we expand functionality in the broker.
>
> >
> > >
> > > Just need to find someone who is willing to take a stab at enhancing
> > > the stomp transport so that it does those "transfromation" header
> > > handling logic. And add a few more interesting implementations of
> > > FrameTranslator.
> >
> > No problem about that. I'll invest some time in this, both in amq and
> > PHP Stomp client. But before that I thought I handle authentication
> > for Stomp as it is far more important IMHO. I'll start with this patch
> > (https://issues.apache.org/activemq/browse/AMQ-1272) add test cases
> > and implement proper behavior.
> >
>
> That's awesome!
>
> >
> > Cheers
> > --
> > Dejan Bosanac
> > www.scriptinginjava.net
> >
>
>
>
>
> --
> Regards,
> Hiram
>
> Blog: http://hiramchirino.com
>
> Open Source SOA
> http://open.iona.com
>
Re: Message transforming on broker
Posted by Hiram Chirino <hi...@hiramchirino.com>.
On Dec 15, 2007 11:11 AM, Dejan Bosanac <de...@nighttale.net> wrote:
> Hi Hiram
>
> On Dec 14, 2007 6:40 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > I personally think that a STOMP client should request a transformation
> > strategy. For example on a subscribe, he could do
> >
> > SUBSCRIBE
> > destination: /queue/foo
> > transformation: jms-json
> > ack: client
> >
> > And on the reverse side, The send could look like:
> >
> > SEND
> > destination:/queue/a
> > transformation: jms-json
> >
> > ["Hello World",1]
> > ^@
> >
> > And have the Stomp transport handle those "transformation" headers by
> > looking up the "jms-json" transformation in the spring context. This
> > would allow the broker to have extensible transformations. And the
> > looked up object would just need to implement the following interface:
> >
> > http://activemq.apache.org/maven/activemq-core/apidocs/org/apache/activemq/transport/stomp/FrameTranslator.html
>
> That's great. This idea has been discussed on the list some time ago
> and I think it's a good approach. I though that maybe it could be
> useful for other transports as well, but you're probably right that we
> should keep this Stomp-related for now.
>
> A few questions about your idea:
>
> * Maybe it would be better to have "transfomers" registered at boot
> time, rather then doing lookups when message arrives? If so, there has
> to be a mechanism to configure Stomp transport with transformers that
> will be used. What method do you propose for this, if any? Maybe
> StompTransportFactory could do lookup on init?
Well the one up side to doing the lookup per send is that the sender
can choose different transformation strategies depending on the
message he is sending. Perhaps he got a JSON message in, so he sends
using the json transformation. Perhaps he got a XML message so he
sends it using an XML transformation etc.
The look up cost should be cheap but if you think perhaps it's an
expensive operation, we could cache lookups. The easiest way to make
this extensible and easy to get stuff registered for lookup is to use
the FactoryFinder [1] to lookup implementations by searching the
META-INF directories. For examples how we use this see the impl of
DiscoveryAgentFactory [2]
[1] http://activemq.apache.org/maven/activemq-core/apidocs/org/apache/activemq/util/FactoryFinder.html
[2] https://svn.apache.org/repos/asf/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/transport/discovery/DiscoveryAgentFactory.java
>
> * We have to allow users to add their "domain" classes to the amq
> classpath, as I guess the most of them will want to transform XML/JSON
> to objects of those classes. That's not a problem for brokers embedded
> in applications, but I guess they could do that in the off-the-shelf
> broker by adding appropriate JAR to the lib/ folder?
>
Yep that's the standard way we expand functionality in the broker.
>
> >
> > Just need to find someone who is willing to take a stab at enhancing
> > the stomp transport so that it does those "transfromation" header
> > handling logic. And add a few more interesting implementations of
> > FrameTranslator.
>
> No problem about that. I'll invest some time in this, both in amq and
> PHP Stomp client. But before that I thought I handle authentication
> for Stomp as it is far more important IMHO. I'll start with this patch
> (https://issues.apache.org/activemq/browse/AMQ-1272) add test cases
> and implement proper behavior.
>
That's awesome!
>
> Cheers
> --
> Dejan Bosanac
> www.scriptinginjava.net
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
Open Source SOA
http://open.iona.com
Re: Message transforming on broker
Posted by Dejan Bosanac <de...@nighttale.net>.
Hi Hiram
On Dec 14, 2007 6:40 PM, Hiram Chirino <hi...@hiramchirino.com> wrote:
> I personally think that a STOMP client should request a transformation
> strategy. For example on a subscribe, he could do
>
> SUBSCRIBE
> destination: /queue/foo
> transformation: jms-json
> ack: client
>
> And on the reverse side, The send could look like:
>
> SEND
> destination:/queue/a
> transformation: jms-json
>
> ["Hello World",1]
> ^@
>
> And have the Stomp transport handle those "transformation" headers by
> looking up the "jms-json" transformation in the spring context. This
> would allow the broker to have extensible transformations. And the
> looked up object would just need to implement the following interface:
>
> http://activemq.apache.org/maven/activemq-core/apidocs/org/apache/activemq/transport/stomp/FrameTranslator.html
That's great. This idea has been discussed on the list some time ago
and I think it's a good approach. I though that maybe it could be
useful for other transports as well, but you're probably right that we
should keep this Stomp-related for now.
A few questions about your idea:
* Maybe it would be better to have "transfomers" registered at boot
time, rather then doing lookups when message arrives? If so, there has
to be a mechanism to configure Stomp transport with transformers that
will be used. What method do you propose for this, if any? Maybe
StompTransportFactory could do lookup on init?
* We have to allow users to add their "domain" classes to the amq
classpath, as I guess the most of them will want to transform XML/JSON
to objects of those classes. That's not a problem for brokers embedded
in applications, but I guess they could do that in the off-the-shelf
broker by adding appropriate JAR to the lib/ folder?
>
> Just need to find someone who is willing to take a stab at enhancing
> the stomp transport so that it does those "transfromation" header
> handling logic. And add a few more interesting implementations of
> FrameTranslator.
No problem about that. I'll invest some time in this, both in amq and
PHP Stomp client. But before that I thought I handle authentication
for Stomp as it is far more important IMHO. I'll start with this patch
(https://issues.apache.org/activemq/browse/AMQ-1272) add test cases
and implement proper behavior.
Cheers
--
Dejan Bosanac
www.scriptinginjava.net
Re: Message transforming on broker
Posted by Hiram Chirino <hi...@hiramchirino.com>.
I personally think that a STOMP client should request a transformation
strategy. For example on a subscribe, he could do
SUBSCRIBE
destination: /queue/foo
transformation: jms-json
ack: client
And on the reverse side, The send could look like:
SEND
destination:/queue/a
transformation: jms-json
["Hello World",1]
^@
And have the Stomp transport handle those "transformation" headers by
looking up the "jms-json" transformation in the spring context. This
would allow the broker to have extensible transformations. And the
looked up object would just need to implement the following interface:
http://activemq.apache.org/maven/activemq-core/apidocs/org/apache/activemq/transport/stomp/FrameTranslator.html
Just need to find someone who is willing to take a stab at enhancing
the stomp transport so that it does those "transfromation" header
handling logic. And add a few more interesting implementations of
FrameTranslator.
Regards,
Hiram
On Dec 14, 2007 6:54 AM, Dejan Bosanac <de...@nighttale.net> wrote:
> And now to continue :) .... sorry about this ...
>
>
> > I was thinking recently if there should be a configurable support for
> > message transforming on the broker level. For example, one use case
> > would be to do XML/Object transformation of all messages
> > received/transmitted through the Stomp protocol.
>
> I guess now that Camel is integrated in AMQ it could be achieved
> through message translators
>
> http://activemq.apache.org/camel/message-translator.html
>
> but maybe a support for some kind of transport listeners, could be
> handy in these kind of situations.
>
> The client-side transformers could not be used since they are tied to
> consumers/produces but some sort of lightweight interface for this
> purpose could be provided.
>
> To me it seems that transport listeners are the place where this can
> be done easily, but someone with more experience with AMQ architecture
> could provide a better suggestion (of course if it is worth doing it
> at all).
>
>
> Cheers
> --
> Dejan Bosanac
> www.scriptinginjava.net
>
>
> On Dec 14, 2007 12:46 PM, Dejan Bosanac <de...@nighttale.net> wrote:
> > I was thinking recently if there should be a configurable support for
> > message transforming on the broker level. For example, one use case
> > would be to do XML/Object transformation of all messages
> > received/transmitted through the Stomp protocol. I guess now that
> > Camel is integrated in AMQ
> >
> > --
> > Dejan Bosanac
> > www.scriptinginjava.net
> >
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
Open Source SOA
http://open.iona.com
Re: Message transforming on broker
Posted by Dejan Bosanac <de...@nighttale.net>.
And now to continue :) .... sorry about this ...
> I was thinking recently if there should be a configurable support for
> message transforming on the broker level. For example, one use case
> would be to do XML/Object transformation of all messages
> received/transmitted through the Stomp protocol.
I guess now that Camel is integrated in AMQ it could be achieved
through message translators
http://activemq.apache.org/camel/message-translator.html
but maybe a support for some kind of transport listeners, could be
handy in these kind of situations.
The client-side transformers could not be used since they are tied to
consumers/produces but some sort of lightweight interface for this
purpose could be provided.
To me it seems that transport listeners are the place where this can
be done easily, but someone with more experience with AMQ architecture
could provide a better suggestion (of course if it is worth doing it
at all).
Cheers
--
Dejan Bosanac
www.scriptinginjava.net
On Dec 14, 2007 12:46 PM, Dejan Bosanac <de...@nighttale.net> wrote:
> I was thinking recently if there should be a configurable support for
> message transforming on the broker level. For example, one use case
> would be to do XML/Object transformation of all messages
> received/transmitted through the Stomp protocol. I guess now that
> Camel is integrated in AMQ
>
> --
> Dejan Bosanac
> www.scriptinginjava.net
>