You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@synapse.apache.org by Rajith Attapattu <ra...@gmail.com> on 2007/03/09 21:13:32 UTC
Re: Endpoints and load-balancing
Chathura, Paul,
Can these load balances identify affinity based on client properties (like
service group id or some other identifier) to make sure requests from a
particular client (or clients accessing the same service group) is always
redirected to a particular node. This will help a lot in the axis2 case, so
we can do cheap clustering w/o worrying about concurrent access to the
replicated contexts. In this case we can replicate to backup nodes blindly
bcos the load balancer will always ensure one primary.
I guess this could be another policy :)
Sorry if I am causing u trouble .
Rajith
On 2/21/07, Chathura Ekanayake <cc...@gmail.com> wrote:
>
> Hi All,
>
> Another advantage I see in the new endpoint model is that we can nest load
> balance and failover groups inside each other. One good use case for this is
> that we can load balance over set of failover groups to acheive scalability
> as well as high availability.
>
> So I propose following changes to endpoints and to the send mediator to
> support this model.
>
> We can have a Endpoint interface which is implemented by all the endpoints.
>
> public interface Endpoint {
>
> public void send(MessageContext synapseMessageContext);
> }
>
> send method sends the message according to the given endpoint object. For example a basic endpoint would send the message to the actual endpoint address and a load balance endpoint would select another endpoint according to a load balance policy and call its send method. So we can achieve the nested structure.
>
>
> Send mediator has a single endpoint object in which there can be a hierarchy of endpoints. When mediating, the send mediator just calls
> the send method of its endpoint object.
>
> Each endpoint has its own endpoint factory which implements the EndpointFactory interface.
>
>
> public interface EndpointFactory {
> public Endpoint createEndpoint(OMElement endpointConfig);
> }
>
> SendMediatorFactory will call above factory method for its immediate child endpoint. And each EndpointFactory will call factory methods for their child endpoints except for leaf level endpoints (address and WSDL endpoints). This is similar to the AbstractListMediatorFactory.
>
>
> I think we can achieve the nested endpoint model cleanly using this design. If this endpoint model and the desing is ok, I will start working on it.
>
> Comments...
>
>
> Chathura
>
>
>
> On 2/22/07, Paul Fremantle <pz...@gmail.com> wrote:
> >
> > Chathura has written some load-balancing code for Synapse, but it
> > seems like the current model for endpoints needs a little refactoring
> > to sit well with it.
> >
> > Currently an endpoint is "just" one endpoint. In the config language
> > (http://ws.apache.org/synapse/Synapse_Configuration_Language.html) it
> > says you can have several endpoints in a <send>..</send> but we've
> > never implemented that.
> >
> > If we did it that way, then you couldn't use the load-balancing with a
> > proxy without defining a sequence, because the proxy just takes a
> > single endpoint reference.
> >
> > So it seems it makes more sense to have the <endpoint> tag refer to a
> > "logical" endpoint. A logical endpoint could be defined in lots of
> > ways. For example, it could actually be a load-balancing group, or a
> > failover pair.
> >
> > So here is a potential syntax:
> >
> > <endpoint name="string">
> > (
> > <address uri="http://" [format="soap|soap11|soap12|pox"]>
> > <enableRM policy=" "/>?
> > <enableSec [policy="key"]/>?
> > <enableAddressing/>?
> > .. extensibility ..
> > </address> |
> >
> > <wsdl [uri="http://.."] [service="qname"] [port/endpoint="qname"]>
> > <wsdl:definition>...</wsdl:definition>?
> > <wsdl20:description>...</wsdl20:description>?
> > <enableRM policy=" "/>?
> > <enableSec [policy="key"]/>?
> > <enableAddressing/>?
> > .. extensibility ..
> > </wsdl> |
> >
> > <loadbalance policy="round-robin">
> > <endpoint ../>+
> > </loadbalance> |
> >
> > <failover policy="exponential-retry">
> > <endpoint ../>+
> > </failover> |
> >
> > .. extensibility .. )
> > </endpoint>
> >
> > So in this model, we can define an endpoint as either an address, or
> > using a WSDL, or as a load-balance group, or failover group. There
> > could be other types (maybe using MEX?).
> >
> > Chathura has some cool implementation thoughts to go with this, which
> > I hope he will send as well (hint, hint).
> >
> > Paul
> >
> > --
> > Paul Fremantle
> > VP/Technology, WSO2 and OASIS WS-RX TC Co-chair
> >
> > http://bloglines.com/blog/paulfremantle
> > paul@wso2.com
> >
> > "Oxygenating the Web Service Platform", www.wso2.com
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
> > For additional commands, e-mail: synapse-dev-help@ws.apache.org
> >
> >
>
Re: Endpoints and load-balancing
Posted by Chathura Ekanayake <cc...@gmail.com>.
Hi Rajith,
I think sample 200 (synapse_sample_200.xml), 201 and 205 will help you.
Sample 201 is just failover sending. So if we are using Axis2 clustering
with that we can keep only one node as the primary at a time.
We don't have to worry about concurrent access issues, but it's only for
high availability (not for scalability).
Sample 200 is state less load balancing. So either there should be no
sessions or all nodes have to be clustered with considering
the concurrent access issues.
Sample 205 would be the best fit. It binds failover groups to http sessions.
So we have cluster the nodes in failover groups using
the primary backup model.
Although the loadbalancing implementation supports service group ID based
sessions, there is an issue in Axis2 (
https://issues.apache.org/jira/browse/AXIS2-2404), which prevents soap
sessions from passing through Synapse. Therefore I didn't
include a sample with the soap session. Once this issue is resolved, setting
<session type="soap"/> would enable soap sessions.
Regards,
Chathura
Re: Endpoints and load-balancing
Posted by Chathura Ekanayake <cc...@gmail.com>.
Hi,
I have completed the synapse loadbalancing and failover implementation with
the addition of session affinity based load balancing.
So in addition to the loadbalancing syntax mentioned the configuration
language, load balancing endpoints can have a element
named <session type="soap | http | simpleClientSession"/>. The type
attribute specifies the session type, to which the load balancing
endpoints should be bound.
Currently it supports 3 session types as noted above.
1) SOAP sessions based on the service group ID
2) HTTP sessions based on HTTP cookies
3) simpleClientSession based on a SOAP header named "ClientID" set by the
client. This is added to demonstrate that loadbalancing
logic can bind sessions based on the content of the messages sent by the
client.
I have also added some samples to demonstrate this new functionality.
Sample 200 (synapse_sample_200.xml): configuration for simple load
balancing.
Sample 201: configuration for failover sending
Sample 202: configuration for simpleClientSession based load balancing
Sample 203: configuration for simpleClientSession based load balancing with
failover
Sample 204: configuration for HTTP session based load balancing
Sample 205: configuration for HTTP session based load balancing with
failover
To get a quick feeling of the new functionality, start synapse with one of
the above sample configurations and run either simple client or
multi threaded client pointing to synapse.
Below is a complete procedure for one scenario.
Extract synapse-1.0-RC1-SNAPSHOT-bin.tar.gz to a directory.
Go to synapse-1.0-RC1-SNAPSHOT/bin directory and run "sh synapse.sh -sample
200"
Go to synapse-1.0-RC1-SNAPSHOT/samples/axis2Server/src/LoadbalanceFailoverService
directory and run "ant" to build neccessary services.
Open synapse-1.0-RC1-SNAPSHOT/samples/axis2Server directory in a seperate
terminal and run "sh axis2server.sh -http 9001 -https 9005 -name Server1"
This will start a server on http port 9001 and gives it the name Server1
Similarly open two another terminals and run servers using
sh axis2server.sh -http 9002 -https 9006 -name Server2
sh axis2server.sh -http 9003 -https 9007 -name Server3
Now go to the synapse-1.0-RC1-SNAPSHOT/samples/axis2Client directory and run
"ant loadbalancefailover"
You can see how requests are distributed between Server1, Server2 and
Server3.
The multi threaded client can be used to run clients in multiple threads
pointing to the same EPR (e.g. Synapse or Axis2 Server).
So it can be used to compare the performance advantage of using the synapse
load balancing over directly invoking Axis2 server.
I will update the synapse samples documentation with the details of these.
Hope you will try this functionality and give ideas for improvement.
Thanks,
Chathura
Re: Endpoints and load-balancing
Posted by Rajith Attapattu <ra...@gmail.com>.
Excellent, this way we can combine axis2 and synapse to do clustering
effectively.
Thanks for the information.
Chathura can I try this out? Do u have a small example on how to do this?
Regards,
Rajith
On 3/11/07, Chathura Ekanayake <cc...@gmail.com> wrote:
>
> Hi Rajith,
>
> I have already implemented the session affinity based loadbalancing. It
> can bind a session id (e.g.soap session, RM session) to an endpoint, so
> that all successive messages from that session will be directed to the
> bound endpoint. In our new endpoint model, this endpoint can also
> be a failover endpoint. Therefore, a session can be bound to a failover
> group, which contains one master node (at a given time) and set of
> backup nodes. So if we consider the failover group as an Axis2 cluster, we
> can blindly replicate the state of the master node to backup nodes
> without considering the concurrency issues.
>
> I have to change my code a bit to suite the new endpoint model and NIO. I
> will commit the new code ASAP.
>
> Chathura
>
>
> On 3/10/07, Rajith Attapattu <ra...@gmail.com> wrote:
> >
> > Chathura, Paul,
> >
> > Can these load balances identify affinity based on client properties
> > (like service group id or some other identifier) to make sure requests from
> > a particular client (or clients accessing the same service group) is always
> > redirected to a particular node. This will help a lot in the axis2 case, so
> > we can do cheap clustering w/o worrying about concurrent access to the
> > replicated contexts. In this case we can replicate to backup nodes blindly
> > bcos the load balancer will always ensure one primary.
> >
> > I guess this could be another policy :)
> > Sorry if I am causing u trouble .
> >
> > Rajith
> >
> > On 2/21/07, Chathura Ekanayake < cce.axis@gmail.com > wrote:
> > >
> > > Hi All,
> > >
> > > Another advantage I see in the new endpoint model is that we can nest
> > > load balance and failover groups inside each other. One good use case for
> > > this is that we can load balance over set of failover groups to acheive
> > > scalability as well as high availability.
> > >
> > > So I propose following changes to endpoints and to the send mediator
> > > to support this model.
> > >
> > > We can have a Endpoint interface which is implemented by all the endpoints.
> > >
> > > public interface Endpoint {
> > >
> > >
> > > public void send(MessageContext synapseMessageContext);
> > > }
> > >
> > > send method sends the message according to the given endpoint object. For example a basic endpoint would send the message to the actual endpoint address and a load balance endpoint would select another endpoint according to a load balance policy and call its send method. So we can achieve the nested structure.
> > >
> > >
> > >
> > > Send mediator has a single endpoint object in which there can be a hierarchy of endpoints. When mediating, the send mediator just calls
> > > the send method of its endpoint object.
> > >
> > > Each endpoint has its own endpoint factory which implements the EndpointFactory interface.
> > >
> > >
> > >
> > > public interface EndpointFactory {
> > > public Endpoint createEndpoint(OMElement endpointConfig);
> > > }
> > >
> > > SendMediatorFactory will call above factory method for its immediate child endpoint. And each EndpointFactory will call factory methods for their child endpoints except for leaf level endpoints (address and WSDL endpoints). This is similar to the AbstractListMediatorFactory.
> > >
> > >
> > >
> > > I think we can achieve the nested endpoint model cleanly using this design. If this endpoint model and the desing is ok, I will start working on it.
> > >
> > > Comments...
> > >
> > >
> > > Chathura
> > >
> > >
> > >
> > > On 2/22/07, Paul Fremantle <pzfreo@gmail.com > wrote:
> > > >
> > > > Chathura has written some load-balancing code for Synapse, but it
> > > > seems like the current model for endpoints needs a little
> > > > refactoring
> > > > to sit well with it.
> > > >
> > > > Currently an endpoint is "just" one endpoint. In the config language
> > > >
> > > > ( http://ws.apache.org/synapse/Synapse_Configuration_Language.html)
> > > > it
> > > > says you can have several endpoints in a <send>..</send> but we've
> > > > never implemented that.
> > > >
> > > > If we did it that way, then you couldn't use the load-balancing with
> > > > a
> > > > proxy without defining a sequence, because the proxy just takes a
> > > > single endpoint reference.
> > > >
> > > > So it seems it makes more sense to have the <endpoint> tag refer to
> > > > a
> > > > "logical" endpoint. A logical endpoint could be defined in lots of
> > > > ways. For example, it could actually be a load-balancing group, or a
> > > > failover pair.
> > > >
> > > > So here is a potential syntax:
> > > >
> > > > <endpoint name="string">
> > > > (
> > > > <address uri="http://" [format="soap|soap11|soap12|pox"]>
> > > > <enableRM policy=" "/>?
> > > > <enableSec [policy="key"]/>?
> > > > <enableAddressing/>?
> > > > .. extensibility ..
> > > > </address> |
> > > >
> > > > <wsdl [uri="http://.."] [service="qname"]
> > > > [port/endpoint="qname"]>
> > > > <wsdl:definition>...</wsdl:definition>?
> > > > <wsdl20:description>...</wsdl20:description>?
> > > > <enableRM policy=" "/>?
> > > > <enableSec [policy="key"]/>?
> > > > <enableAddressing/>?
> > > > .. extensibility ..
> > > > </wsdl> |
> > > >
> > > > <loadbalance policy="round-robin">
> > > > <endpoint ../>+
> > > > </loadbalance> |
> > > >
> > > > <failover policy="exponential-retry">
> > > > <endpoint ../>+
> > > > </failover> |
> > > >
> > > > .. extensibility .. )
> > > > </endpoint>
> > > >
> > > > So in this model, we can define an endpoint as either an address, or
> > > >
> > > > using a WSDL, or as a load-balance group, or failover group. There
> > > > could be other types (maybe using MEX?).
> > > >
> > > > Chathura has some cool implementation thoughts to go with this,
> > > > which
> > > > I hope he will send as well (hint, hint).
> > > >
> > > > Paul
> > > >
> > > > --
> > > > Paul Fremantle
> > > > VP/Technology, WSO2 and OASIS WS-RX TC Co-chair
> > > >
> > > > http://bloglines.com/blog/paulfremantle
> > > > paul@wso2.com
> > > >
> > > > "Oxygenating the Web Service Platform", www.wso2.com
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
> > > > For additional commands, e-mail: synapse-dev-help@ws.apache.org
> > > >
> > > >
> > >
> >
>
Re: Endpoints and load-balancing
Posted by Chathura Ekanayake <cc...@gmail.com>.
Hi Rajith,
I have already implemented the session affinity based loadbalancing. It
can bind a session id (e.g.soap session, RM session) to an endpoint, so
that all successive messages from that session will be directed to the bound
endpoint. In our new endpoint model, this endpoint can also
be a failover endpoint. Therefore, a session can be bound to a failover
group, which contains one master node (at a given time) and set of
backup nodes. So if we consider the failover group as an Axis2 cluster, we
can blindly replicate the state of the master node to backup nodes
without considering the concurrency issues.
I have to change my code a bit to suite the new endpoint model and NIO. I
will commit the new code ASAP.
Chathura
On 3/10/07, Rajith Attapattu <ra...@gmail.com> wrote:
>
> Chathura, Paul,
>
> Can these load balances identify affinity based on client properties (like
> service group id or some other identifier) to make sure requests from a
> particular client (or clients accessing the same service group) is always
> redirected to a particular node. This will help a lot in the axis2 case, so
> we can do cheap clustering w/o worrying about concurrent access to the
> replicated contexts. In this case we can replicate to backup nodes blindly
> bcos the load balancer will always ensure one primary.
>
> I guess this could be another policy :)
> Sorry if I am causing u trouble .
>
> Rajith
>
> On 2/21/07, Chathura Ekanayake < cce.axis@gmail.com> wrote:
> >
> > Hi All,
> >
> > Another advantage I see in the new endpoint model is that we can nest
> > load balance and failover groups inside each other. One good use case for
> > this is that we can load balance over set of failover groups to acheive
> > scalability as well as high availability.
> >
> > So I propose following changes to endpoints and to the send mediator to
> > support this model.
> >
> > We can have a Endpoint interface which is implemented by all the endpoints.
> >
> > public interface Endpoint {
> >
> > public void send(MessageContext synapseMessageContext);
> > }
> >
> > send method sends the message according to the given endpoint object. For example a basic endpoint would send the message to the actual endpoint address and a load balance endpoint would select another endpoint according to a load balance policy and call its send method. So we can achieve the nested structure.
> >
> >
> > Send mediator has a single endpoint object in which there can be a hierarchy of endpoints. When mediating, the send mediator just calls
> > the send method of its endpoint object.
> >
> > Each endpoint has its own endpoint factory which implements the EndpointFactory interface.
> >
> >
> > public interface EndpointFactory {
> > public Endpoint createEndpoint(OMElement endpointConfig);
> > }
> >
> > SendMediatorFactory will call above factory method for its immediate child endpoint. And each EndpointFactory will call factory methods for their child endpoints except for leaf level endpoints (address and WSDL endpoints). This is similar to the AbstractListMediatorFactory.
> >
> >
> > I think we can achieve the nested endpoint model cleanly using this design. If this endpoint model and the desing is ok, I will start working on it.
> >
> > Comments...
> >
> >
> > Chathura
> >
> >
> >
> > On 2/22/07, Paul Fremantle <pzfreo@gmail.com > wrote:
> > >
> > > Chathura has written some load-balancing code for Synapse, but it
> > > seems like the current model for endpoints needs a little refactoring
> > > to sit well with it.
> > >
> > > Currently an endpoint is "just" one endpoint. In the config language
> > > (http://ws.apache.org/synapse/Synapse_Configuration_Language.html) it
> > > says you can have several endpoints in a <send>..</send> but we've
> > > never implemented that.
> > >
> > > If we did it that way, then you couldn't use the load-balancing with a
> > > proxy without defining a sequence, because the proxy just takes a
> > > single endpoint reference.
> > >
> > > So it seems it makes more sense to have the <endpoint> tag refer to a
> > > "logical" endpoint. A logical endpoint could be defined in lots of
> > > ways. For example, it could actually be a load-balancing group, or a
> > > failover pair.
> > >
> > > So here is a potential syntax:
> > >
> > > <endpoint name="string">
> > > (
> > > <address uri="http://" [format="soap|soap11|soap12|pox"]>
> > > <enableRM policy=" "/>?
> > > <enableSec [policy="key"]/>?
> > > <enableAddressing/>?
> > > .. extensibility ..
> > > </address> |
> > >
> > > <wsdl [uri="http://.."] [service="qname"] [port/endpoint="qname"]>
> > > <wsdl:definition>...</wsdl:definition>?
> > > <wsdl20:description>...</wsdl20:description>?
> > > <enableRM policy=" "/>?
> > > <enableSec [policy="key"]/>?
> > > <enableAddressing/>?
> > > .. extensibility ..
> > > </wsdl> |
> > >
> > > <loadbalance policy="round-robin">
> > > <endpoint ../>+
> > > </loadbalance> |
> > >
> > > <failover policy="exponential-retry">
> > > <endpoint ../>+
> > > </failover> |
> > >
> > > .. extensibility .. )
> > > </endpoint>
> > >
> > > So in this model, we can define an endpoint as either an address, or
> > > using a WSDL, or as a load-balance group, or failover group. There
> > > could be other types (maybe using MEX?).
> > >
> > > Chathura has some cool implementation thoughts to go with this, which
> > > I hope he will send as well (hint, hint).
> > >
> > > Paul
> > >
> > > --
> > > Paul Fremantle
> > > VP/Technology, WSO2 and OASIS WS-RX TC Co-chair
> > >
> > > http://bloglines.com/blog/paulfremantle
> > > paul@wso2.com
> > >
> > > "Oxygenating the Web Service Platform", www.wso2.com
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: synapse-dev-help@ws.apache.org
> > >
> > >
> >
>