You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ode.apache.org by Jeff Yu <je...@gmail.com> on 2010/04/07 04:40:10 UTC
ODE clustering support
Hi all,
I'd like to bring this topic up,
https://issues.apache.org/jira/browse/ODE-563, I am going to work on this
task, before I do this, I'd like to have a heads up at here.
Below is discussion that I found in the mail archive by Alex Boisvert, which
I think it is very good summary in this area.
"
The Axis2 integration already supports *clustering*. (Well, maybe a better
characterization is that it doesn't get in the way)
There's a few things do to to support *clustering*. First thing would be to
make the scheduler *cluster*-aware, to support work distribution (load
balancing) across different nodes and to avoid contention. I suggested
using Terracotta as a basis for this, but there may be other (and better)
ways.
If we want to get more performance out of the system, we can introduce
varying levels of node affinity to reduce data shipping and improve cache
efficiency within the *cluster*. This would probably entail some form of
distributed lock manager, or a registry to redirect messages to process
instances already loaded on *cluster* nodes. This is an area where
integration with the messaging layer (Axis2) can help since you can
establish "sessions" between clients and services to improve message routing
and, again, cache efficiency.
The persistence DAOs (OpenJPA and Hibernate) also support *clustering*today.
Both can be configured to be *cluster*-aware and both support the more
aggressive transparent *cluster* cache such as Terracotta, JBoss Cache, etc.
Hope this helps,
alex
"
>From above summary, I believe we have following sub-tasks for the
clustering.
1. make cronScheduler cluster-aware.
2. simpleScheduler module's DAO need to be implemented as JPA based, so that
it can support cluster.
3. make simple scheduler cluster-aware?
Anything I am missing or mis-understanding here??
--
Cheers,
Jeff Yu
----------------
blog: http://jeff.familyyu.net
Re: ODE clustering support
Posted by Jeff Yu <je...@gmail.com>.
Hi Milinda,
I believe below are some tasks that we need to do:
1. clustering deployment, or as you said process store synchronization. I am
wondering how did you do this, did you create a repository for its
deployment registration?
2. heart beat mechanism for detecting cluster node.
3. implement the cluster aware cron scheduler, we can try to extract the
cron scheduler API, and then try out the Quartz by leverage its clustering
feature.
For these three tasks, we will come up an API for them, so that can be
plugged into different implementation.
I am not sure about the axis2 integration already supports *clustering* in
the email that wrote by Alex Boisvert.
I thought in the front end, we would expect users to use the httpd to do the
load balancing.
Let me know what you thought, and what I am missing here.
Regards
Jeff
On Wed, Apr 7, 2010 at 2:25 PM, Milinda Pathirage <
milinda.pathirage@gmail.com> wrote:
> Hi Jeff,
>
> I did some initial work on clustering for WSO2 BPS based on clustering
> support provided by Axis2. I only implement the process store
> synchronization part. But we are planning to start work on clustering
> support end of this April. I am happy to help you on this. There are some
> nice design documents on OpenESB BPEL clustering at [1]. This may
> also helpful when designing the ODE clustering.
>
> Thanks
> Milinda
>
> [1] http://wiki.open-esb.java.net/Wiki.jsp?page=BPELSEClusteringDesign
>
> On Wed, Apr 7, 2010 at 8:10 AM, Jeff Yu <je...@gmail.com> wrote:
>
> > Hi all,
> >
> > I'd like to bring this topic up,
> > https://issues.apache.org/jira/browse/ODE-563, I am going to work on
> this
> > task, before I do this, I'd like to have a heads up at here.
> >
> > Below is discussion that I found in the mail archive by Alex Boisvert,
> > which
> > I think it is very good summary in this area.
> >
> > "
> > The Axis2 integration already supports *clustering*. (Well, maybe a
> better
> > characterization is that it doesn't get in the way)
> >
> > There's a few things do to to support *clustering*. First thing would be
> > to
> >
> > make the scheduler *cluster*-aware, to support work distribution (load
> > balancing) across different nodes and to avoid contention. I suggested
> > using Terracotta as a basis for this, but there may be other (and better)
> > ways.
> >
> > If we want to get more performance out of the system, we can introduce
> > varying levels of node affinity to reduce data shipping and improve cache
> > efficiency within the *cluster*. This would probably entail some form of
> > distributed lock manager, or a registry to redirect messages to process
> > instances already loaded on *cluster* nodes. This is an area where
> > integration with the messaging layer (Axis2) can help since you can
> > establish "sessions" between clients and services to improve message
> > routing
> >
> > and, again, cache efficiency.
> >
> > The persistence DAOs (OpenJPA and Hibernate) also support
> > *clustering*today.
> > Both can be configured to be *cluster*-aware and both support the more
> > aggressive transparent *cluster* cache such as Terracotta, JBoss Cache,
> > etc.
> >
> >
> > Hope this helps,
> > alex
> > "
> >
> > From above summary, I believe we have following sub-tasks for the
> > clustering.
> >
> > 1. make cronScheduler cluster-aware.
> > 2. simpleScheduler module's DAO need to be implemented as JPA based, so
> > that
> > it can support cluster.
> > 3. make simple scheduler cluster-aware?
> >
> > Anything I am missing or mis-understanding here??
> >
> > --
> > Cheers,
> > Jeff Yu
> >
> > ----------------
> > blog: http://jeff.familyyu.net
> >
>
>
>
> --
> Milinda Pathirage
> Senior Software Engineer & Product Manager WSO2 BPS; http://wso2.org/bps
> WSO2 <http://wso2.org/bps%0AWSO2> Inc.; http://wso2.com
> E-mail: milinda@wso2.com, milinda.pathirage@gmail.com
> Web: http://mpathirage.com
> Blog: http://blog.mpathirage.com
>
--
Cheers,
Jeff Yu
----------------
blog: http://jeff.familyyu.net
Re: ODE clustering support
Posted by Jeff Yu <je...@gmail.com>.
Hi Milinda,
It is great to hear this. ;), I will look at the url, and then continue the
discussion over here.
Regards
Jeff
On Wed, Apr 7, 2010 at 2:25 PM, Milinda Pathirage <
milinda.pathirage@gmail.com> wrote:
> Hi Jeff,
>
> I did some initial work on clustering for WSO2 BPS based on clustering
> support provided by Axis2. I only implement the process store
> synchronization part. But we are planning to start work on clustering
> support end of this April. I am happy to help you on this. There are some
> nice design documents on OpenESB BPEL clustering at [1]. This may
> also helpful when designing the ODE clustering.
>
> Thanks
> Milinda
>
> [1] http://wiki.open-esb.java.net/Wiki.jsp?page=BPELSEClusteringDesign
>
> On Wed, Apr 7, 2010 at 8:10 AM, Jeff Yu <je...@gmail.com> wrote:
>
> > Hi all,
> >
> > I'd like to bring this topic up,
> > https://issues.apache.org/jira/browse/ODE-563, I am going to work on
> this
> > task, before I do this, I'd like to have a heads up at here.
> >
> > Below is discussion that I found in the mail archive by Alex Boisvert,
> > which
> > I think it is very good summary in this area.
> >
> > "
> > The Axis2 integration already supports *clustering*. (Well, maybe a
> better
> > characterization is that it doesn't get in the way)
> >
> > There's a few things do to to support *clustering*. First thing would be
> > to
> >
> > make the scheduler *cluster*-aware, to support work distribution (load
> > balancing) across different nodes and to avoid contention. I suggested
> > using Terracotta as a basis for this, but there may be other (and better)
> > ways.
> >
> > If we want to get more performance out of the system, we can introduce
> > varying levels of node affinity to reduce data shipping and improve cache
> > efficiency within the *cluster*. This would probably entail some form of
> > distributed lock manager, or a registry to redirect messages to process
> > instances already loaded on *cluster* nodes. This is an area where
> > integration with the messaging layer (Axis2) can help since you can
> > establish "sessions" between clients and services to improve message
> > routing
> >
> > and, again, cache efficiency.
> >
> > The persistence DAOs (OpenJPA and Hibernate) also support
> > *clustering*today.
> > Both can be configured to be *cluster*-aware and both support the more
> > aggressive transparent *cluster* cache such as Terracotta, JBoss Cache,
> > etc.
> >
> >
> > Hope this helps,
> > alex
> > "
> >
> > From above summary, I believe we have following sub-tasks for the
> > clustering.
> >
> > 1. make cronScheduler cluster-aware.
> > 2. simpleScheduler module's DAO need to be implemented as JPA based, so
> > that
> > it can support cluster.
> > 3. make simple scheduler cluster-aware?
> >
> > Anything I am missing or mis-understanding here??
> >
> > --
> > Cheers,
> > Jeff Yu
> >
> > ----------------
> > blog: http://jeff.familyyu.net
> >
>
>
>
> --
> Milinda Pathirage
> Senior Software Engineer & Product Manager WSO2 BPS; http://wso2.org/bps
> WSO2 <http://wso2.org/bps%0AWSO2> Inc.; http://wso2.com
> E-mail: milinda@wso2.com, milinda.pathirage@gmail.com
> Web: http://mpathirage.com
> Blog: http://blog.mpathirage.com
>
--
Cheers,
Jeff Yu
----------------
blog: http://jeff.familyyu.net
Re: ODE clustering support
Posted by Milinda Pathirage <mi...@gmail.com>.
Hi Jeff,
I did some initial work on clustering for WSO2 BPS based on clustering
support provided by Axis2. I only implement the process store
synchronization part. But we are planning to start work on clustering
support end of this April. I am happy to help you on this. There are some
nice design documents on OpenESB BPEL clustering at [1]. This may
also helpful when designing the ODE clustering.
Thanks
Milinda
[1] http://wiki.open-esb.java.net/Wiki.jsp?page=BPELSEClusteringDesign
On Wed, Apr 7, 2010 at 8:10 AM, Jeff Yu <je...@gmail.com> wrote:
> Hi all,
>
> I'd like to bring this topic up,
> https://issues.apache.org/jira/browse/ODE-563, I am going to work on this
> task, before I do this, I'd like to have a heads up at here.
>
> Below is discussion that I found in the mail archive by Alex Boisvert,
> which
> I think it is very good summary in this area.
>
> "
> The Axis2 integration already supports *clustering*. (Well, maybe a better
> characterization is that it doesn't get in the way)
>
> There's a few things do to to support *clustering*. First thing would be
> to
>
> make the scheduler *cluster*-aware, to support work distribution (load
> balancing) across different nodes and to avoid contention. I suggested
> using Terracotta as a basis for this, but there may be other (and better)
> ways.
>
> If we want to get more performance out of the system, we can introduce
> varying levels of node affinity to reduce data shipping and improve cache
> efficiency within the *cluster*. This would probably entail some form of
> distributed lock manager, or a registry to redirect messages to process
> instances already loaded on *cluster* nodes. This is an area where
> integration with the messaging layer (Axis2) can help since you can
> establish "sessions" between clients and services to improve message
> routing
>
> and, again, cache efficiency.
>
> The persistence DAOs (OpenJPA and Hibernate) also support
> *clustering*today.
> Both can be configured to be *cluster*-aware and both support the more
> aggressive transparent *cluster* cache such as Terracotta, JBoss Cache,
> etc.
>
>
> Hope this helps,
> alex
> "
>
> From above summary, I believe we have following sub-tasks for the
> clustering.
>
> 1. make cronScheduler cluster-aware.
> 2. simpleScheduler module's DAO need to be implemented as JPA based, so
> that
> it can support cluster.
> 3. make simple scheduler cluster-aware?
>
> Anything I am missing or mis-understanding here??
>
> --
> Cheers,
> Jeff Yu
>
> ----------------
> blog: http://jeff.familyyu.net
>
--
Milinda Pathirage
Senior Software Engineer & Product Manager WSO2 BPS; http://wso2.org/bps
WSO2 Inc.; http://wso2.com
E-mail: milinda@wso2.com, milinda.pathirage@gmail.com
Web: http://mpathirage.com
Blog: http://blog.mpathirage.com
Re: ODE clustering support
Posted by Milinda Pathirage <mi...@gmail.com>.
+1 for Rafal's proposal. We must make ODE clustering possible for both
integration layers. When implementing process store synchronization I had to
use central repository to store BPEL packages. For that I did my own
implementation of Process store and process configuration, we must take
these things into consideration when designing clustering API.
Thanks
Milinda
On Wed, Apr 7, 2010 at 2:27 PM, Rafal Rusin <ra...@gmail.com> wrote:
> I played with clustering ODE a bit. I think the problem with
> clustering is that it doesn't have common public API implemented by
> various vendors.
> Instead of that, each vendor has it's own clustering extensions. For
> example, Axis2 uses Tomcat's tribes.
> So I think we need to support various cluster implementations in ODE
> in future. This will include SMX3 & 4 too.
> So let's make an interface, which will include cluster related
> operations, like noticing new node or rewriting http destination while
> making outgoing http calls from ODE. This could have pluggable
> implementations for various clustering vendors.
>
> On 7 April 2010 04:40, Jeff Yu <je...@gmail.com> wrote:
> > Hi all,
> >
> > I'd like to bring this topic up,
> > https://issues.apache.org/jira/browse/ODE-563, I am going to work on
> this
> > task, before I do this, I'd like to have a heads up at here.
> >
> > Below is discussion that I found in the mail archive by Alex Boisvert,
> which
> > I think it is very good summary in this area.
> >
> > "
> > The Axis2 integration already supports *clustering*. (Well, maybe a
> better
> > characterization is that it doesn't get in the way)
> >
> > There's a few things do to to support *clustering*. First thing would be
> to
> >
> > make the scheduler *cluster*-aware, to support work distribution (load
> > balancing) across different nodes and to avoid contention. I suggested
> > using Terracotta as a basis for this, but there may be other (and better)
> > ways.
> >
> > If we want to get more performance out of the system, we can introduce
> > varying levels of node affinity to reduce data shipping and improve cache
> > efficiency within the *cluster*. This would probably entail some form of
> > distributed lock manager, or a registry to redirect messages to process
> > instances already loaded on *cluster* nodes. This is an area where
> > integration with the messaging layer (Axis2) can help since you can
> > establish "sessions" between clients and services to improve message
> routing
> >
> > and, again, cache efficiency.
> >
> > The persistence DAOs (OpenJPA and Hibernate) also support
> *clustering*today.
> > Both can be configured to be *cluster*-aware and both support the more
> > aggressive transparent *cluster* cache such as Terracotta, JBoss Cache,
> etc.
> >
> >
> > Hope this helps,
> > alex
> > "
> >
> > From above summary, I believe we have following sub-tasks for the
> > clustering.
> >
> > 1. make cronScheduler cluster-aware.
> > 2. simpleScheduler module's DAO need to be implemented as JPA based, so
> that
> > it can support cluster.
> > 3. make simple scheduler cluster-aware?
> >
> > Anything I am missing or mis-understanding here??
> >
> > --
> > Cheers,
> > Jeff Yu
> >
> > ----------------
> > blog: http://jeff.familyyu.net
> >
>
>
> Regards,
> --
> Rafał Rusin
> http://rrusin.blogspot.com
> http://www.touk.pl
> http://top.touk.pl
>
--
Milinda Pathirage
Senior Software Engineer & Product Manager WSO2 BPS; http://wso2.org/bps
WSO2 Inc.; http://wso2.com
E-mail: milinda@wso2.com, milinda.pathirage@gmail.com
Web: http://mpathirage.com
Blog: http://blog.mpathirage.com
Re: ODE clustering support
Posted by Rafal Rusin <ra...@gmail.com>.
I played with clustering ODE a bit. I think the problem with
clustering is that it doesn't have common public API implemented by
various vendors.
Instead of that, each vendor has it's own clustering extensions. For
example, Axis2 uses Tomcat's tribes.
So I think we need to support various cluster implementations in ODE
in future. This will include SMX3 & 4 too.
So let's make an interface, which will include cluster related
operations, like noticing new node or rewriting http destination while
making outgoing http calls from ODE. This could have pluggable
implementations for various clustering vendors.
On 7 April 2010 04:40, Jeff Yu <je...@gmail.com> wrote:
> Hi all,
>
> I'd like to bring this topic up,
> https://issues.apache.org/jira/browse/ODE-563, I am going to work on this
> task, before I do this, I'd like to have a heads up at here.
>
> Below is discussion that I found in the mail archive by Alex Boisvert, which
> I think it is very good summary in this area.
>
> "
> The Axis2 integration already supports *clustering*. (Well, maybe a better
> characterization is that it doesn't get in the way)
>
> There's a few things do to to support *clustering*. First thing would be to
>
> make the scheduler *cluster*-aware, to support work distribution (load
> balancing) across different nodes and to avoid contention. I suggested
> using Terracotta as a basis for this, but there may be other (and better)
> ways.
>
> If we want to get more performance out of the system, we can introduce
> varying levels of node affinity to reduce data shipping and improve cache
> efficiency within the *cluster*. This would probably entail some form of
> distributed lock manager, or a registry to redirect messages to process
> instances already loaded on *cluster* nodes. This is an area where
> integration with the messaging layer (Axis2) can help since you can
> establish "sessions" between clients and services to improve message routing
>
> and, again, cache efficiency.
>
> The persistence DAOs (OpenJPA and Hibernate) also support *clustering*today.
> Both can be configured to be *cluster*-aware and both support the more
> aggressive transparent *cluster* cache such as Terracotta, JBoss Cache, etc.
>
>
> Hope this helps,
> alex
> "
>
> From above summary, I believe we have following sub-tasks for the
> clustering.
>
> 1. make cronScheduler cluster-aware.
> 2. simpleScheduler module's DAO need to be implemented as JPA based, so that
> it can support cluster.
> 3. make simple scheduler cluster-aware?
>
> Anything I am missing or mis-understanding here??
>
> --
> Cheers,
> Jeff Yu
>
> ----------------
> blog: http://jeff.familyyu.net
>
Regards,
--
Rafał Rusin
http://rrusin.blogspot.com
http://www.touk.pl
http://top.touk.pl
Re: ODE clustering support
Posted by sharrissf <st...@terracottatech.com>.
Hi,
I work with the Terracotta team. Let us know if there is anything we can do
to help out!
We can do a call and or give advise over e-mail if it's needed.
Cheers,
Steve
Jeff Yu wrote:
>
> Hi all,
>
> I'd like to bring this topic up,
> https://issues.apache.org/jira/browse/ODE-563, I am going to work on this
> task, before I do this, I'd like to have a heads up at here.
>
> Below is discussion that I found in the mail archive by Alex Boisvert,
> which
> I think it is very good summary in this area.
>
> "
> The Axis2 integration already supports *clustering*. (Well, maybe a
> better
> characterization is that it doesn't get in the way)
>
> There's a few things do to to support *clustering*. First thing would be
> to
>
> make the scheduler *cluster*-aware, to support work distribution (load
> balancing) across different nodes and to avoid contention. I suggested
> using Terracotta as a basis for this, but there may be other (and better)
> ways.
>
> If we want to get more performance out of the system, we can introduce
> varying levels of node affinity to reduce data shipping and improve cache
> efficiency within the *cluster*. This would probably entail some form of
> distributed lock manager, or a registry to redirect messages to process
> instances already loaded on *cluster* nodes. This is an area where
> integration with the messaging layer (Axis2) can help since you can
> establish "sessions" between clients and services to improve message
> routing
>
> and, again, cache efficiency.
>
> The persistence DAOs (OpenJPA and Hibernate) also support
> *clustering*today.
> Both can be configured to be *cluster*-aware and both support the more
> aggressive transparent *cluster* cache such as Terracotta, JBoss Cache,
> etc.
>
>
> Hope this helps,
> alex
> "
>
> From above summary, I believe we have following sub-tasks for the
> clustering.
>
> 1. make cronScheduler cluster-aware.
> 2. simpleScheduler module's DAO need to be implemented as JPA based, so
> that
> it can support cluster.
> 3. make simple scheduler cluster-aware?
>
> Anything I am missing or mis-understanding here??
>
> --
> Cheers,
> Jeff Yu
>
> ----------------
> blog: http://jeff.familyyu.net
>
>
--
View this message in context: http://old.nabble.com/ODE-clustering-support-tp28159941p28170400.html
Sent from the Apache Ode Dev mailing list archive at Nabble.com.