You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Harvey Li <ha...@Objectivasoftware.com> on 2009/06/15 23:56:17 UTC
RE: Always "Waiting for CPG initialization"
Thanks. Your note is really helpful.
I installed openais and re-made the project. I could get help usage with cluster.so module.
But when I failed to start my broker by blocking on the message "notice Waiting for CPG initialization".
(Without cluster it works fine)
My command to start the broker likes "src/qpidd --load-module cluster.so --cluster-name cluster1".
What should be the cluster name? Is it arbitrary?
Should I add more options like --cluster-url?
I am still wondering about how the cluster nodes share the persistent data. They don't need to share a relational database or share a data folder? unbelievable...
-----Original Message-----
From: Gordon Sim [mailto:gsim@redhat.com]
Sent: Mon 2009-6-15 18:41
To: dev@qpid.apache.org
Cc: users@qpid.apache.org
Subject: Re: How to build the Active-Active cluster?
Harvey Li wrote:
> I didn't get details on how to build a A-A cluster in Qpid C++.
> Except in FAQ. Where can I find it? I will appreciate if someone
> points it out.
>
> Yes. I found some document about Federation and qpid-route. But I
> think federation and cluster are totally different concept, aren't
> they?
Yes they are. The cluster support should be automatically built (as a
dynamically loadable plugin, cluster.so) if you have openais and
openais-devel installed when configuring.
There is a little bit of information in the INSTALL notes distributed
with the code about this and getting ais configured and started. To
start a cluster all you should require is then to run qpidd with the
cluster module loaded and a --cluster-name option supplied.
Hope this helps a little...
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Always "Waiting for CPG initialization"
Posted by Alan Conway <ac...@redhat.com>.
Carl Trieloff wrote:
>
> if you can connect to it with qpid-tool, and list or run qpid-cluster it
> will report it's up status.
>
> In terms of the group, you can also newgrp ais, or add ais to the group
> of the qpidd process.
>
Note: for the cluster, qpidd needs to have ais as its effective group, not just
be a member. You can start qpidd with a user who's _primary_ group is ais, or
you can start with a user who is a member of ais _and_ use "newgrp ais" to make
it the effective group.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Always "Waiting for CPG initialization"
Posted by Alan Conway <ac...@redhat.com>.
Carl Trieloff wrote:
>
> if you can connect to it with qpid-tool, and list or run qpid-cluster it
> will report it's up status.
>
> In terms of the group, you can also newgrp ais, or add ais to the group
> of the qpidd process.
>
Note: for the cluster, qpidd needs to have ais as its effective group, not just
be a member. You can start qpidd with a user who's _primary_ group is ais, or
you can start with a user who is a member of ais _and_ use "newgrp ais" to make
it the effective group.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Always "Waiting for CPG initialization"
Posted by Carl Trieloff <cc...@redhat.com>.
if you can connect to it with qpid-tool, and list or run qpid-cluster it
will report it's up status.
In terms of the group, you can also newgrp ais, or add ais to the group
of the qpidd process.
regards,
Carl.
Harvey Li wrote:
> Finally I got cluster ready. It seems I have to execute Qpid start command with "sudo" as well. Or it would lock.
>
> Without you guys help it's impossible. I really appreciate it.
>
> Do you have a set of test cases to ensure it works? Or where's a document to introduce the whole mechanism of the cluster?
>
> -----Original Message-----
> From: Gordon Sim [mailto:gsim@redhat.com]
> Sent: Tue 2009-6-16 17:15
> To: dev@qpid.apache.org
> Cc: users@qpid.apache.org
> Subject: Re: Always "Waiting for CPG initialization"
>
> Harvey Li wrote:
>
>> cctrieloff@redhat.com>, <us...@qpid.apache.org> Cc:
>> <de...@qpid.apache.org>
>>
>> So I need to start openais first before starting broker? I have no
>> idea of openais before. Thanks. I will try it later.
>>
>
> Yes, have a read of the 'configuring ais' section in the INSTALL notes
> for some hints.
>
>
>> For the cluster, in a word it combine the load-balancing and failover
>> feature together. I don't need to worry anything(like message lost or
>> delay). I could add a broke or kill a broke anytime, right?
>>
>
> It provides redundancy through replicating broker state. An enqueued
> message will exist on all nodes and will thus not be lost when any node
> fails (providing it is survived by at least one other node). Until a
> message is enqueued however it is 'in-doubt' and may need to be replayed
> on failing over.
>
> Though each cluster node is active and will accept clients connections,
> the degree to which this balances load should not be overstated as each
> node will process the same sequence of inputs in order to keep the state
> consistent.
>
> --Gordon.
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>
Re: Always "Waiting for CPG initialization"
Posted by Carl Trieloff <cc...@redhat.com>.
if you can connect to it with qpid-tool, and list or run qpid-cluster it
will report it's up status.
In terms of the group, you can also newgrp ais, or add ais to the group
of the qpidd process.
regards,
Carl.
Harvey Li wrote:
> Finally I got cluster ready. It seems I have to execute Qpid start command with "sudo" as well. Or it would lock.
>
> Without you guys help it's impossible. I really appreciate it.
>
> Do you have a set of test cases to ensure it works? Or where's a document to introduce the whole mechanism of the cluster?
>
> -----Original Message-----
> From: Gordon Sim [mailto:gsim@redhat.com]
> Sent: Tue 2009-6-16 17:15
> To: dev@qpid.apache.org
> Cc: users@qpid.apache.org
> Subject: Re: Always "Waiting for CPG initialization"
>
> Harvey Li wrote:
>
>> cctrieloff@redhat.com>, <us...@qpid.apache.org> Cc:
>> <de...@qpid.apache.org>
>>
>> So I need to start openais first before starting broker? I have no
>> idea of openais before. Thanks. I will try it later.
>>
>
> Yes, have a read of the 'configuring ais' section in the INSTALL notes
> for some hints.
>
>
>> For the cluster, in a word it combine the load-balancing and failover
>> feature together. I don't need to worry anything(like message lost or
>> delay). I could add a broke or kill a broke anytime, right?
>>
>
> It provides redundancy through replicating broker state. An enqueued
> message will exist on all nodes and will thus not be lost when any node
> fails (providing it is survived by at least one other node). Until a
> message is enqueued however it is 'in-doubt' and may need to be replayed
> on failing over.
>
> Though each cluster node is active and will accept clients connections,
> the degree to which this balances load should not be overstated as each
> node will process the same sequence of inputs in order to keep the state
> consistent.
>
> --Gordon.
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
>
RE: Always "Waiting for CPG initialization"
Posted by Harvey Li <ha...@Objectivasoftware.com>.
Finally I got cluster ready. It seems I have to execute Qpid start command with "sudo" as well. Or it would lock.
Without you guys help it's impossible. I really appreciate it.
Do you have a set of test cases to ensure it works? Or where's a document to introduce the whole mechanism of the cluster?
-----Original Message-----
From: Gordon Sim [mailto:gsim@redhat.com]
Sent: Tue 2009-6-16 17:15
To: dev@qpid.apache.org
Cc: users@qpid.apache.org
Subject: Re: Always "Waiting for CPG initialization"
Harvey Li wrote:
> cctrieloff@redhat.com>, <us...@qpid.apache.org> Cc:
> <de...@qpid.apache.org>
>
> So I need to start openais first before starting broker? I have no
> idea of openais before. Thanks. I will try it later.
Yes, have a read of the 'configuring ais' section in the INSTALL notes
for some hints.
> For the cluster, in a word it combine the load-balancing and failover
> feature together. I don't need to worry anything(like message lost or
> delay). I could add a broke or kill a broke anytime, right?
It provides redundancy through replicating broker state. An enqueued
message will exist on all nodes and will thus not be lost when any node
fails (providing it is survived by at least one other node). Until a
message is enqueued however it is 'in-doubt' and may need to be replayed
on failing over.
Though each cluster node is active and will accept clients connections,
the degree to which this balances load should not be overstated as each
node will process the same sequence of inputs in order to keep the state
consistent.
--Gordon.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
RE: Always "Waiting for CPG initialization"
Posted by Harvey Li <ha...@Objectivasoftware.com>.
Finally I got cluster ready. It seems I have to execute Qpid start command with "sudo" as well. Or it would lock.
Without you guys help it's impossible. I really appreciate it.
Do you have a set of test cases to ensure it works? Or where's a document to introduce the whole mechanism of the cluster?
-----Original Message-----
From: Gordon Sim [mailto:gsim@redhat.com]
Sent: Tue 2009-6-16 17:15
To: dev@qpid.apache.org
Cc: users@qpid.apache.org
Subject: Re: Always "Waiting for CPG initialization"
Harvey Li wrote:
> cctrieloff@redhat.com>, <us...@qpid.apache.org> Cc:
> <de...@qpid.apache.org>
>
> So I need to start openais first before starting broker? I have no
> idea of openais before. Thanks. I will try it later.
Yes, have a read of the 'configuring ais' section in the INSTALL notes
for some hints.
> For the cluster, in a word it combine the load-balancing and failover
> feature together. I don't need to worry anything(like message lost or
> delay). I could add a broke or kill a broke anytime, right?
It provides redundancy through replicating broker state. An enqueued
message will exist on all nodes and will thus not be lost when any node
fails (providing it is survived by at least one other node). Until a
message is enqueued however it is 'in-doubt' and may need to be replayed
on failing over.
Though each cluster node is active and will accept clients connections,
the degree to which this balances load should not be overstated as each
node will process the same sequence of inputs in order to keep the state
consistent.
--Gordon.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Always "Waiting for CPG initialization"
Posted by Alan Conway <ac...@redhat.com>.
Gordon Sim wrote:
> Harvey Li wrote:
>> cctrieloff@redhat.com>, <us...@qpid.apache.org> Cc:
>> <de...@qpid.apache.org>
>>
>> So I need to start openais first before starting broker? I have no
>> idea of openais before. Thanks. I will try it later.
>
> Yes, have a read of the 'configuring ais' section in the INSTALL notes
> for some hints.
>
Also note that on some recent versions of linux (e.g. if you're on a recent
version of Fedora) openais has been renamed as corosync.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Always "Waiting for CPG initialization"
Posted by Alan Conway <ac...@redhat.com>.
Gordon Sim wrote:
> Harvey Li wrote:
>> cctrieloff@redhat.com>, <us...@qpid.apache.org> Cc:
>> <de...@qpid.apache.org>
>>
>> So I need to start openais first before starting broker? I have no
>> idea of openais before. Thanks. I will try it later.
>
> Yes, have a read of the 'configuring ais' section in the INSTALL notes
> for some hints.
>
Also note that on some recent versions of linux (e.g. if you're on a recent
version of Fedora) openais has been renamed as corosync.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Always "Waiting for CPG initialization"
Posted by Gordon Sim <gs...@redhat.com>.
Harvey Li wrote:
> cctrieloff@redhat.com>, <us...@qpid.apache.org> Cc:
> <de...@qpid.apache.org>
>
> So I need to start openais first before starting broker? I have no
> idea of openais before. Thanks. I will try it later.
Yes, have a read of the 'configuring ais' section in the INSTALL notes
for some hints.
> For the cluster, in a word it combine the load-balancing and failover
> feature together. I don't need to worry anything(like message lost or
> delay). I could add a broke or kill a broke anytime, right?
It provides redundancy through replicating broker state. An enqueued
message will exist on all nodes and will thus not be lost when any node
fails (providing it is survived by at least one other node). Until a
message is enqueued however it is 'in-doubt' and may need to be replayed
on failing over.
Though each cluster node is active and will accept clients connections,
the degree to which this balances load should not be overstated as each
node will process the same sequence of inputs in order to keep the state
consistent.
--Gordon.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Always "Waiting for CPG initialization"
Posted by Gordon Sim <gs...@redhat.com>.
Harvey Li wrote:
> cctrieloff@redhat.com>, <us...@qpid.apache.org> Cc:
> <de...@qpid.apache.org>
>
> So I need to start openais first before starting broker? I have no
> idea of openais before. Thanks. I will try it later.
Yes, have a read of the 'configuring ais' section in the INSTALL notes
for some hints.
> For the cluster, in a word it combine the load-balancing and failover
> feature together. I don't need to worry anything(like message lost or
> delay). I could add a broke or kill a broke anytime, right?
It provides redundancy through replicating broker state. An enqueued
message will exist on all nodes and will thus not be lost when any node
fails (providing it is survived by at least one other node). Until a
message is enqueued however it is 'in-doubt' and may need to be replayed
on failing over.
Though each cluster node is active and will accept clients connections,
the degree to which this balances load should not be overstated as each
node will process the same sequence of inputs in order to keep the state
consistent.
--Gordon.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
RE: Always "Waiting for CPG initialization"
Posted by Harvey Li <ha...@Objectivasoftware.com>.
cctrieloff@redhat.com>,
<us...@qpid.apache.org>
Cc: <de...@qpid.apache.org>
So I need to start openais first before starting broker? I have no idea of openais before. Thanks. I will try it later.
For the cluster, in a word it combine the load-balancing and failover feature together. I don't need to worry anything(like message lost or delay). I could add a broke or kill a broke anytime, right?
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
RE: Always "Waiting for CPG initialization"
Posted by Harvey Li <ha...@Objectivasoftware.com>.
cctrieloff@redhat.com>,
<us...@qpid.apache.org>
Cc: <de...@qpid.apache.org>
So I need to start openais first before starting broker? I have no idea of openais before. Thanks. I will try it later.
For the cluster, in a word it combine the load-balancing and failover feature together. I don't need to worry anything(like message lost or delay). I could add a broke or kill a broke anytime, right?
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Always "Waiting for CPG initialization"
Posted by Carl Trieloff <cc...@redhat.com>.
Harvey Li wrote:
> Thanks. Your note is really helpful.
> I installed openais and re-made the project. I could get help usage with cluster.so module.
> But when I failed to start my broker by blocking on the message "notice Waiting for CPG initialization".
> (Without cluster it works fine)
>
When you start AIS, make sure it has come up correctly in it's log.
(i.e. typical issue, is a. firewall, b. did not set the netmask)
> My command to start the broker likes "src/qpidd --load-module cluster.so --cluster-name cluster1".
> What should be the cluster name? Is it arbitrary?
>
anything you like, but the same on all node that you want to join
> Should I add more options like --cluster-url?
>
this is used if you have AIS on one set of nics for example, and the
public traffic on others. So you can limite the URL's sent to clients.
The URL list from all nodes that is specified will be
provided to the clients for failover.
>
> I am still wondering about how the cluster nodes share the persistent data. They don't need to share a relational database or share a data folder? unbelievable...
>
They don't share any data, each node in the cluster logs it's own copy.
So you can just shoot a node, disk and all if you want to.
Carl.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Always "Waiting for CPG initialization"
Posted by Carl Trieloff <cc...@redhat.com>.
Harvey Li wrote:
> Thanks. Your note is really helpful.
> I installed openais and re-made the project. I could get help usage with cluster.so module.
> But when I failed to start my broker by blocking on the message "notice Waiting for CPG initialization".
> (Without cluster it works fine)
>
When you start AIS, make sure it has come up correctly in it's log.
(i.e. typical issue, is a. firewall, b. did not set the netmask)
> My command to start the broker likes "src/qpidd --load-module cluster.so --cluster-name cluster1".
> What should be the cluster name? Is it arbitrary?
>
anything you like, but the same on all node that you want to join
> Should I add more options like --cluster-url?
>
this is used if you have AIS on one set of nics for example, and the
public traffic on others. So you can limite the URL's sent to clients.
The URL list from all nodes that is specified will be
provided to the clients for failover.
>
> I am still wondering about how the cluster nodes share the persistent data. They don't need to share a relational database or share a data folder? unbelievable...
>
They don't share any data, each node in the cluster logs it's own copy.
So you can just shoot a node, disk and all if you want to.
Carl.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org