You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@synapse.apache.org by Andreas Veithen <an...@skynet.be> on 2008/09/16 22:43:21 UTC
Re: Test failures [was: Proposal to implement ws-eventing and an event distribution model in Synapse]
Asankha,
I did some modifications to try to fix this. Can you test again?
Andreas
On 15 sept. 08, at 15:17, Asankha C. Perera wrote:
> Andreas
>
> Logs are attached
>
> asankha
>
> Andreas Veithen wrote:
>>
>> Asankha,
>>
>> The tests all work correctly on my machine. I will need your
>> Surefire logs to figure out why they are failing on your machine.
>>
>> Andreas
>>
>> Quoting "Asankha C. Perera" <as...@wso2.com>:
>>
>>> I reverted by commit.. but still see failures:
>>>
>>> Tests in error:
>>> 0001
>>> :test
>>> =
>>> AsyncXML
>>> ,data
>>> =
>>> ASCII
>>> ,messageType
>>> =
>>> SOAP11
>>> ,client
>>> =
>>> java
>>> .net
>>> ,endpoint
>>> =
>>> axis
>>> (org
>>> .apache
>>> .synapse.transport.testkit.tests.async.XMLAsyncMessageTestCase)
>>> 0034
>>> :test
>>> =
>>> AsyncTextPlain
>>> ,data
>>> =
>>> ASCII
>>> ,client
>>> =
>>> java
>>> .net
>>> ,endpoint
>>> =
>>> axis
>>> (org.apache.synapse.transport.testkit.tests.async.TextPlainTestCase)
>>> 0035
>>> :test
>>> =
>>> AsyncTextPlain
>>> ,data
>>> =
>>> UTF8
>>> ,client
>>> =
>>> java
>>> .net
>>> ,endpoint
>>> =
>>> axis
>>> (org.apache.synapse.transport.testkit.tests.async.TextPlainTestCase)
>>> 0036
>>> :test
>>> =
>>> AsyncTextPlain
>>> ,data
>>> =
>>> Latin1
>>> ,client
>>> =
>>> java
>>> .net
>>> ,endpoint
>>> =
>>> axis
>>> (org.apache.synapse.transport.testkit.tests.async.TextPlainTestCase)
>>> 0043
>>> :test
>>> =
>>> AsyncBinary
>>> ,client
>>> =
>>> java
>>> .net
>>> ,endpoint
>>> =
>>> axis
>>> (org.apache.synapse.transport.testkit.tests.async.BinaryTestCase)
>>> 0046
>>> :test
>>> =
>>> REST
>>> ,client
>>> =
>>> java
>>> .net
>>> ,endpoint
>>> =axis(org.apache.synapse.transport.testkit.tests.async.RESTTestCase)
>>> 0143
>>> :test
>>> =
>>> EchoXML
>>> ,data
>>> =
>>> ASCII
>>> ,messageType
>>> =
>>> POX
>>> ,broker
>>> =
>>> qpid
>>> ,replyDestType
>>> =
>>> queue
>>> ,destType
>>> =
>>> queue
>>> ,contentTypeMode
>>> =
>>> TRANSPORT
>>> ,client
>>> =
>>> axis
>>> ,endpoint
>>> =
>>> mock
>>> (org
>>> .apache
>>> .synapse
>>> .transport.testkit.tests.echo.XMLRequestResponseMessageTestCase)
>>>
>>> Andreas, can you fix these and checkin the changes
>>>
>>> asankha
>>>
>>>
>>> Asankha C. Perera wrote:
>>>> Ruwan
>>>>
>>>> I am reverting my changes as we speak, until such time I can fix
>>>> them and make sure the tests run.. sorry for this..
>>>>
>>>> asankha
>>>>
>>>> Andreas Veithen wrote:
>>>>> For the test failures: On Friday, Asankha made a change in the
>>>>> mail transport that causes 12 unit tests to fail. In addition
>>>>> to that, some tests in the transports module may fail on some
>>>>> platforms/systems because of concurrency issues. They run fine
>>>>> on my machines (Mac OS X and Windows XP), but Asankha told me
>>>>> that he sees this kind of failures (should be corrected in the
>>>>> meantime).
>>>>>
>>>>> Andreas
>>>>>
>>>>> Quoting Ruwan Linton <ru...@gmail.com>:
>>>>>
>>>>>> Sorry Paul, I was also unable to comment on your last
>>>>>> queries.... as you may
>>>>>> know I am traveling. I think the above 2 points that you made
>>>>>> have some
>>>>>> interesting value in it. Therefore I would like to keep the
>>>>>> ability to
>>>>>> configure the event source and the notification service
>>>>>> sections in the
>>>>>> configuration.
>>>>>>
>>>>>> As you told, I just started the implementation and you can
>>>>>> expect the
>>>>>> initial version soon. For that lets go with the proposed
>>>>>> configuration and I
>>>>>> will keep the two elements to configure the notification
>>>>>> section and
>>>>>> publishing section.
>>>>>>
>>>>>> BTW: I am seeing a test failure in the trunk? Do any one know
>>>>>> the reason for
>>>>>> this? Andreas?
>>>>>>
>>>>>> (PS: please make sure all tests are passing before commit)
>>>>>>
>>>>>> Thanks,
>>>>>> Ruwan
>>>>>>
>>>>>>
>>>>>> On Mon, Sep 15, 2008 at 4:02 PM, Paul Fremantle
>>>>>> <pz...@gmail.com> wrote:
>>>>>>
>>>>>>> Any other comments on this? I know Ruwan is working on it and
>>>>>>> I know
>>>>>>> we would appreciate feedback and thoughts.
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>> ---------- Forwarded message ----------
>>>>>>> From: Paul Fremantle <pz...@gmail.com>
>>>>>>> Date: Fri, Sep 12, 2008 at 11:56 AM
>>>>>>> Subject: Re: Proposal to implement ws-eventing and an event
>>>>>>> distribution model in Synapse
>>>>>>> To: dev@synapse.apache.org
>>>>>>>
>>>>>>>
>>>>>>> Ruwan
>>>>>>>
>>>>>>> The reasons I think we should keep the WSEventing section are:
>>>>>>>
>>>>>>> 1) We should support other models - WS-Notification, etc.
>>>>>>> Although we
>>>>>>> have started from Eventing, this is a fairly generic model of
>>>>>>> events
>>>>>>> and I think we should keep it layered.
>>>>>>>
>>>>>>> 2) You may need to configure security or other aspects onto the
>>>>>>> WSEventing endpoint, so you need to offer the same configuration
>>>>>>> elements that proxy does (except target).
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>> On Fri, Sep 12, 2008 at 10:01 AM, Ruwan Linton <ruwan.linton@gmail.com
>>>>>>> >
>>>>>>> wrote:
>>>>>>>> Paul,
>>>>>>>>
>>>>>>>> Very nice explanation of the concepts that we have been
>>>>>>>> trying to put
>>>>>>>> together into the code. Let me add some more to your
>>>>>>>> explanation and
>>>>>>> refine
>>>>>>>> the configuration a bit more.
>>>>>>>>
>>>>>>>> <eventSource name="blah">
>>>>>>>> <subscriptionManager
>>>>>>>> class
>>>>>>>> =
>>>>>>>> "org.apache.synapse.events.DefaultInMemorySubscriptionManager">
>>>>>>>> <property name="blah">some xml prop</property>
>>>>>>>> <property name="other" value="some text property"/>
>>>>>>>> </subscriptionManager>
>>>>>>>> <staticSubscriptions>
>>>>>>>> <subscription id="static1">
>>>>>>>> <filter..../>
>>>>>>>> <sequence.../>
>>>>>>>> <endpoint../>
>>>>>>>> </subscription>*
>>>>>>>> <staticSubscriptions>?
>>>>>>>> <eventSource/>
>>>>>>>>
>>>>>>>> Here I am getting rid of the wsEventing configuration element
>>>>>>>> where you
>>>>>>>> specify the subscription service and the event source
>>>>>>>> service. So
>>>>>>> my idea
>>>>>>> is
>>>>>>>> we can extend the proxy services model here and create a new
>>>>>>>> EventingMessageReceiver, which listens for all the requests
>>>>>>>> coming to
>>>>>>> this
>>>>>>>> event source. (I must also say at this point event source is
>>>>>>>> now a
>>>>>>> service
>>>>>>>> inside synapse and that fits with the model of extending the
>>>>>>>> proxy
>>>>>>> service
>>>>>>>> behavior)
>>>>>>>>
>>>>>>>> This EventingMessageReceiver knows how to filter out the the
>>>>>>> subscription
>>>>>>>> messages from the notification messages and it uses the
>>>>>>>> specified
>>>>>>>> subscription manager if it is a subscription request, and if
>>>>>>>> it is a
>>>>>>>> notification message this receiver will delegate the request to
>>>>>>> the event
>>>>>>>> publisher where you find the set of subscribers with matching
>>>>>>>> filter
>>>>>>>> conditions and execute the mediation sequence and then send
>>>>>>>> the event to
>>>>>>> the
>>>>>>>> specified endpoint.
>>>>>>>>
>>>>>>>> Paul, what do you think about this implementation. I am
>>>>>>>> halfway through
>>>>>>> the
>>>>>>>> implementation and can have a look at this in the weekend :-)
>>>>>>>> I have
>>>>>>>> attached an architecture diagram which explains this concept
>>>>>>>> a little
>>>>>>> more
>>>>>>>> and that explains that the event source itself is now exposed
>>>>>>>> as a
>>>>>>> service
>>>>>>>> to which you can send subscriptions and notifications.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ruwan
>>>>>>>>
>>>>>>>> On Fri, Sep 12, 2008 at 9:15 AM, Paul Fremantle <pzfreo@gmail.com
>>>>>>>> >
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Ruwan, AsankaA and I have been building a POC using WS-
>>>>>>>>> Eventing this
>>>>>>>>> week and we think we have come up with a reasonable model.
>>>>>>>>> We've
>>>>>>>>> already iterated several times, and in writing it out I have
>>>>>>>>> iterated
>>>>>>>>> beyond what the three of us discussed, so I am expecting more
>>>>>>>>> iterations now.
>>>>>>>>>
>>>>>>>>> What we implemented is a mediator that distributes events
>>>>>>>>> based on a
>>>>>>>>> filter. The initial code was almost dead simple:
>>>>>>>>>
>>>>>>>>> for (Subscription subs : manager.getSubscribers()) {
>>>>>>>>> boolean response =
>>>>>>>>> subs.getFilterMediator().test(mc);
>>>>>>>>> if (response) {
>>>>>>>>> Endpoint ep =
>>>>>>>>> subs.getEndpoint();
>>>>>>>>>
>>>>>>>>> ep.send(getClonedMessageContext(mc));
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> As we implemented the POC it became clear that it was more
>>>>>>>>> elegant to
>>>>>>>>> be able to associate a sequence to a particular
>>>>>>>>> subscription, and
>>>>>>>>> execute that sequence before sending. This goes a bit beyond
>>>>>>>>> the
>>>>>>>>> standard WS-Eventing model, but doesn't seem to contradict
>>>>>>>>> it or be a
>>>>>>>>> bad fit.
>>>>>>>>>
>>>>>>>>> We also implemented a WS-Eventing subscribe model. Now that is
>>>>>>>>> logically separate, because there might be other ways to
>>>>>>>>> subscribe.
>>>>>>>>> For example, you might subscribe by adding an entry in a
>>>>>>>>> registry or
>>>>>>>>> using WS-Notification or your own interface. We also have
>>>>>>>>> allowed
>>>>>>>>> simple static subscriptions in the synapse.xml model too.
>>>>>>>>>
>>>>>>>>> So the mediator itself is really simple - it only needs to
>>>>>>>>> get access
>>>>>>>>> to some kind of thing that manages the subscriptions that
>>>>>>>>> can give it
>>>>>>>>> a list of subscribers. In WS-Eventing an "Event Source" is
>>>>>>>>> something
>>>>>>>>> that emits events. Effectively our mediator is therefore an
>>>>>>>>> event
>>>>>>>>> source. So effectively the event source name is how you
>>>>>>>>> reference the
>>>>>>>>> manager that gives you the list of subscribers:
>>>>>>>>>
>>>>>>>>> <sequence>
>>>>>>>>> <event-source-publisher event-source-name="name"/>
>>>>>>>>> </sequence>
>>>>>>>>>
>>>>>>>>> Now how do you define these event sources. Well we want a
>>>>>>>>> new top
>>>>>>>>> level child of <definitions> that is configured at start
>>>>>>>>> time. And
>>>>>>>>> this defines an event-source, and also configures how the
>>>>>>>>> subscriptions can happen.
>>>>>>>>>
>>>>>>>>> <definitions>
>>>>>>>>> <eventSource name="blah">
>>>>>>>>> <subscriptionManager
>>>>>>>>> class
>>>>>>>>> =
>>>>>>>>> "org
>>>>>>>>> .apache.synapse.events.DefaultInMemorySubscriptionManager">
>>>>>>>>> <property name="blah">some xml prop</property>
>>>>>>>>> <property name="other" value="some text property"/>
>>>>>>>>> </subscriptionManager>
>>>>>>>>> <subscription id="static1">
>>>>>>>>> <filter....>
>>>>>>>>> <sequence...>
>>>>>>>>> <endpoint..>
>>>>>>>>> </subscription>
>>>>>>>>> <subscription...>
>>>>>>>>> <wsEventing>
>>>>>>>>> <eventSourceService name="myEventSource">
>>>>>>>>> <same subchildren of proxy go here>
>>>>>>>>> </eventSourceService>
>>>>>>>>> <subscriptionManagerService name="myEventSubManager">
>>>>>>>>> <same subchildren of proxy go here>
>>>>>>>>> </subscriptionManager>
>>>>>>>>> </wsEventing>
>>>>>>>>> <eventSource>
>>>>>>>>>
>>>>>>>>> Lets go through this:
>>>>>>>>> Each event source has a subscription manager. This is a
>>>>>>>>> class that
>>>>>>>>> keeps track of subscriptions. Here are some examples: a
>>>>>>>>> transient in
>>>>>>>>> memory one. A database backed persistent one. A registry
>>>>>>>>> backed
>>>>>>>>> read-only one. A registry backed read-write one. The class
>>>>>>>>> must
>>>>>>>>> implement a simple interface:
>>>>>>>>> public interface SubscriptionManager {
>>>>>>>>> List<Subscription> getSubscribers();
>>>>>>>>> Subscription getSubscription(String id);
>>>>>>>>> String addSubscription(Subscription subs);
>>>>>>>>> boolean deleteSubscription(String id);
>>>>>>>>>
>>>>>>>>> }
>>>>>>>>> The subscriptionManager instance is injected with config
>>>>>>>>> properties at
>>>>>>>>> startup just like other things are (tasks, class mediators,
>>>>>>>>> etc).
>>>>>>>>> These might contain the JDBC connection parameters or the
>>>>>>>>> URL of the
>>>>>>>>> registry.
>>>>>>>>>
>>>>>>>>> Next come static subscriptions. These are added into the
>>>>>>>>> subscription
>>>>>>>>> manager by synapse. That happens once at startup.
>>>>>>>>>
>>>>>>>>> The next piece is WSEventing specific, but there could be
>>>>>>>>> other
>>>>>>>>> children for notification etc. Here I'm not 100% sure that
>>>>>>>>> we need to
>>>>>>>>> separate the EventSourceService from the
>>>>>>>>> SubscriptionManagerService.
>>>>>>>>> In WS-Eventing it says these can be the same endpoint or
>>>>>>>>> different.
>>>>>>>>> Basically the configuration of these is the same as a proxy,
>>>>>>>>> allowing
>>>>>>>>> configuration of security etc for this endpoint.
>>>>>>>>>
>>>>>>>>> We certainly haven't done everything. We haven't handled
>>>>>>>>> expiry,
>>>>>>>>> though . We haven't thought about other deliveryModes. We
>>>>>>>>> haven't
>>>>>>>>> dealt with efficiently handling evaluating multiple
>>>>>>>>> subscriptions
>>>>>>>>> against a single message at once. We have simply re-used the
>>>>>>>>> existing
>>>>>>>>> filtermediator code to implement XPath and Xpath/Regex
>>>>>>>>> filters as is
>>>>>>>>> (we can be much more efficient, for Xpaths, by e.g. using
>>>>>>>>> DanD's SXC
>>>>>>>>> code which can evaluate multiple Xpaths on a single
>>>>>>>>> message). But its
>>>>>>>>> not a bad start.
>>>>>>>>>
>>>>>>>>> I'd really appreciate if we have the right overall
>>>>>>>>> structure. I did a
>>>>>>>>> first cut of code, but Ruwan is tidying it up right now, so
>>>>>>>>> expect a
>>>>>>>>> check-in soon.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Paul
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Paul Fremantle
>>>>>>>>> Co-Founder and CTO, WSO2
>>>>>>>>> Apache Synapse PMC Chair
>>>>>>>>> OASIS WS-RX TC Co-chair
>>>>>>>>>
>>>>>>>>> blog: http://pzf.fremantle.org
>>>>>>>>> paul@wso2.com
>>>>>>>>>
>>>>>>>>> "Oxygenating the Web Service Platform", www.wso2.com
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>>>>>>>>> For additional commands, e-mail: dev-help@synapse.apache.org
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ruwan Linton
>>>>>>>> http://wso2.org - "Oxygenating the Web Services Platform"
>>>>>>>> http://ruwansblog.blogspot.com/
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@synapse.apache.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Paul Fremantle
>>>>>>> Co-Founder and CTO, WSO2
>>>>>>> Apache Synapse PMC Chair
>>>>>>> OASIS WS-RX TC Co-chair
>>>>>>>
>>>>>>> blog: http://pzf.fremantle.org
>>>>>>> paul@wso2.com
>>>>>>>
>>>>>>> "Oxygenating the Web Service Platform", www.wso2.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Paul Fremantle
>>>>>>> Co-Founder and CTO, WSO2
>>>>>>> Apache Synapse PMC Chair
>>>>>>> OASIS WS-RX TC Co-chair
>>>>>>>
>>>>>>> blog: http://pzf.fremantle.org
>>>>>>> paul@wso2.com
>>>>>>>
>>>>>>> "Oxygenating the Web Service Platform", www.wso2.com
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>>>>>>> For additional commands, e-mail: dev-help@synapse.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Ruwan Linton
>>>>>> http://wso2.org - "Oxygenating the Web Services Platform"
>>>>>> http://ruwansblog.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>>>>> For additional commands, e-mail: dev-help@synapse.apache.org
>>>>>
>>>>>
>>>>
>>>> --
>>>> Asankha C. Perera
>>>>
>>>> WSO2 - http://wso2.org
>>>> http://esbmagic.blogspot.com
>>>>
>>>
>>> --
>>> Asankha C. Perera
>>>
>>> WSO2 - http://wso2.org
>>> http://esbmagic.blogspot.com
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>> For additional commands, e-mail: dev-help@synapse.apache.org
>>
>>
>
> --
> Asankha C. Perera
>
> WSO2 - http://wso2.org
> http://esbmagic.blogspot.com
>
> <surefire-
> reports
> .zip
> >---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
> For additional commands, e-mail: dev-help@synapse.apache.org