You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by sditlinger <st...@ditlinger.com> on 2015/02/19 23:08:25 UTC
Bundle sync in cluster with failover
ServiceMix 5.4/Karaf 2.4.1/Cellar 2.3.4
I have three karaf instances in a cluster with cellar. I have set them up
with Master-Slave-Slave with a simple file lock.
1) I start root as Master, from within root shell, I start child-1 and
child-2, both are waiting on file lock
2) I deploy a REST service in root shell using cluster:bundle-install from a
Maven repository (accessible to all instances); I cluster:bundle-start the
service
3) I can see the newly deployed service in cluster:bundle-list on root
4) I see the service in osgi:list on root.
5) I see the service listed on web page at http://localhost:8181/cxf
6) I kill root karaf process, it releases lock
7) Child-1 takes over as master and hold lock, child-2 remains waiting
8) I can see the service in cluster:bundle-list on child-1
9) I DO NOT see the service in osgi:list on child-1.
10) I DO NOT see the service listed on web page at http://localhost:8181/cxf
Same thing repeats for child-2 if I stop child-1.
So it appears there is no syncing of bundles here, even though child-1 knows
about the service (i.e. shows up in cluster:bundle-list results). Why isn't
the service syncing to child-1 from the cluster? Do all 3 nodes have to be
completely up (i.e. not waiting for lock) for the sync to take place? The
bundle sync property is set to "cluster" for all instances.
--
View this message in context: http://karaf.922171.n3.nabble.com/Bundle-sync-in-cluster-with-failover-tp4038642.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Bundle sync in cluster with failover
Posted by sditlinger <st...@ditlinger.com>.
Hmmmm, I want to believe you, but the JDBC locking also says it is
Active/Passive. Maybe you mean container level locking?
In any case, if the service bundle is listed in the cluster bundle map (i.e.
shows in the results for "cluster:bundle-list") on child-1, why doesn't it
get synced to child-1 when child-1 finishes loading up and becomes the
master? Seems like we are inches away from it working the way I want it to.
Steve
--
View this message in context: http://karaf.922171.n3.nabble.com/Bundle-sync-in-cluster-with-failover-tp4038642p4038644.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Bundle sync in cluster with failover
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi all and Steve,
(I'm back from vacation).
It sounds good to me.
Regards
JB
On 02/24/2015 12:10 AM, sditlinger wrote:
> So the proper process for a deployment on a failover-configured karaf cluster
> would be to:
> 1) stop all slave karaf instances
> 2) remove all locking (file, jdbc, etc) configuration from system properties
> files for all slave karaf instances
> 3) re-start all slave karaf instances in cluster
> 4) install and start new bundles/features to cluster
> 5) stop all slave instances
> 6) re-add the locking (file, jdbc, etc) configuration to system properties
> files for all slave karaf instances
> 7) re-start the slave karaf instances
>
> Is this correct?
>
> Thanks,
> Steve
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Bundle-sync-in-cluster-with-failover-tp4038642p4038679.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Bundle sync in cluster with failover
Posted by sditlinger <st...@ditlinger.com>.
So the proper process for a deployment on a failover-configured karaf cluster
would be to:
1) stop all slave karaf instances
2) remove all locking (file, jdbc, etc) configuration from system properties
files for all slave karaf instances
3) re-start all slave karaf instances in cluster
4) install and start new bundles/features to cluster
5) stop all slave instances
6) re-add the locking (file, jdbc, etc) configuration to system properties
files for all slave karaf instances
7) re-start the slave karaf instances
Is this correct?
Thanks,
Steve
--
View this message in context: http://karaf.922171.n3.nabble.com/Bundle-sync-in-cluster-with-failover-tp4038642p4038679.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Bundle sync in cluster with failover
Posted by Achim Nierbeck <bc...@googlemail.com>.
Hi,
simple answer yes, all three nodes must be in a state to sync each other
via cellar, as
you have a very low lock-level the other two instances can't sync with your
"master" node.
Cellar is made for a "active/active" cluster not a "active/passive"
failover.
It looks like you need a database based lock here. [1]
In this make sure your "business-bundles" are residing in a higher
startlevel, compared to what you have
configured. The default is usually startlevel 50.
Would be interesting to have a Hazelcast based lock, that shouldn't be to
hard to achieve instead of a jdbc one.
That way you'll use cellar infrastructure for the locking mechanism.
regards, Achim
[1] - http://karaf.apache.org/manual/latest/users-guide/failover.html
2015-02-19 23:08 GMT+01:00 sditlinger <st...@ditlinger.com>:
> ServiceMix 5.4/Karaf 2.4.1/Cellar 2.3.4
> I have three karaf instances in a cluster with cellar. I have set them up
> with Master-Slave-Slave with a simple file lock.
>
> 1) I start root as Master, from within root shell, I start child-1 and
> child-2, both are waiting on file lock
> 2) I deploy a REST service in root shell using cluster:bundle-install from
> a
> Maven repository (accessible to all instances); I cluster:bundle-start the
> service
> 3) I can see the newly deployed service in cluster:bundle-list on root
> 4) I see the service in osgi:list on root.
> 5) I see the service listed on web page at http://localhost:8181/cxf
> 6) I kill root karaf process, it releases lock
> 7) Child-1 takes over as master and hold lock, child-2 remains waiting
> 8) I can see the service in cluster:bundle-list on child-1
> 9) I DO NOT see the service in osgi:list on child-1.
> 10) I DO NOT see the service listed on web page at
> http://localhost:8181/cxf
>
> Same thing repeats for child-2 if I stop child-1.
>
> So it appears there is no syncing of bundles here, even though child-1
> knows
> about the service (i.e. shows up in cluster:bundle-list results). Why isn't
> the service syncing to child-1 from the cluster? Do all 3 nodes have to be
> completely up (i.e. not waiting for lock) for the sync to take place? The
> bundle sync property is set to "cluster" for all instances.
>
>
>
>
> --
> View this message in context:
> http://karaf.922171.n3.nabble.com/Bundle-sync-in-cluster-with-failover-tp4038642.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>
Software Architect / Project Manager / Scrum Master