You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by stoyan <st...@hotmail.com> on 2011/05/10 18:36:18 UTC
qpid clustering: Failed to initialize CPG
hello
i have two brokers (tag 0.8) and i want to create a cluster(using
corosync-1.3.0) including these brokers.
following the instructions on
https://cwiki.apache.org/qpid/starting-a-cluster.html (selinux is disabled)
on CentOs 5, i managed to get two machines talking in the cluster (log read
"Completed service synchronization, ready to provide service", but when i
started one broker with --cluster-name=QpidCL i could only get as far as
"Daemon startup failed: Failed to initialize CPG.: try again (6)"
(/var/log/messages doesn't give more details than that either).
what is strange to me is that the corosync.log doesn't indicate that someone
is trying to reach it. i guess this is security/permissions issue, but have
no idea how to fight it further - i tried putting the qpidd user in ais
group, and also tried creating uidgid file (/etc/corosync/uidgid.d/qpidd)
any ideas? am i missing a qpid.conf setting or something?
thanks!
stoyan
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/qpid-clustering-Failed-to-initialize-CPG-tp6348587p6348587.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid clustering: Failed to initialize CPG
Posted by stoyan <st...@hotmail.com>.
i just noticed qpid 0.10 is out and tested locally with it - got different
error log, but with the same result:
notice Initializing CPG
info Retrying cpg_join
info Retrying cpg_join
info Retrying cpg_join
info Retrying cpg_finalize
info Retrying cpg_finalize
info Retrying cpg_finalize
error Error during CPG shutdown: Cannot finalize CPG group QpidCL
so, i guess, help is still needed :(
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/qpid-clustering-Failed-to-initialize-CPG-tp6348587p6355557.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid clustering: Failed to initialize CPG
Posted by stoyan <st...@hotmail.com>.
testcpg update: testcpg failed because it wasn't run with the proper user;
once that was fixed it executed fine:
Local node id is 7364a8c0
membership list
node id 1935976640 pid 9567
Type EXIT to finish
ConfchgCallback: group 'GROUP'
joined node/pid 1935976640/9567 reason: 1
nodes in group now 1
node/pid 1935976640/9567
TotemConfchgCallback: ringid (1935976640.60)
active processors 1: 1935976640
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/qpid-clustering-Failed-to-initialize-CPG-tp6348587p6352448.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid clustering: Failed to initialize CPG
Posted by stoyan <st...@hotmail.com>.
Thank you , Alan
i am also somewhat confused about openais/corosync - which service would
qpid use?
initially i started corosync on two servers and they seemed to communicate
well (but when starting the broker on one of these servers it issued Failed
to initialize CPG).
then i tried to start openais service and it began complaining with:
aisexec: [MAIN ] parse error in config: No interfaces defined
aisexec: [MAIN ] AIS Executive exiting (reason: could not read the main
configuration file).
and since i have openais.conf (and it has proper interfaces similar to those
in corosync.conf) i assumed it was looking for cluster.conf and hence the
cman cluster thoughts.
i put user qpidd in ais group and also created /etc/corosync/uidgid.d/qpidd:
uidgid {
uid: qpidd
gid: ais
}
but the blocker seems to be that qpidd cannot find the correct cpg, so the
answer might really lie somewhere in these addresses:
here is the corosync.conf:
compatibility: whitetank
totem {
version: 2
secauth: off
threads: 0
interface {
ringnumber: 0
#my network
bindnetaddr: 192.168.100.0
mcastaddr: 226.94.1.1
mcastport: 5405
}
}
logging {
fileline: off
to_stderr: no
to_logfile: yes
to_syslog: yes
logfile: /var/log/cluster/corosync.log
debug: off
timestamp: on
logger_subsys {
subsys: AMF
debug: off
}
}
amf {
mode: disabled
}
/sbin/ifconfig:
eth0 Link encap:Ethernet HWaddr 00:E0:81:C1:00:8F
inet addr:192.168.100.115 Bcast:192.168.100.255
Mask:255.255.255.0
inet6 addr: fe80::2e0:81ff:fec1:8f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:17421920 errors:0 dropped:154 overruns:0 frame:0
TX packets:16831241 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:13205968256 (12.2 GiB) TX bytes:10379287406 (9.6 GiB)
Memory:fbce0000-fbd00000
btw just tried testcpg and it also fails with "Could not initialize Cluster
Process Group API instance error 100"
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/qpid-clustering-Failed-to-initialize-CPG-tp6348587p6352045.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid clustering: Failed to initialize CPG
Posted by Alan Conway <ac...@redhat.com>.
On 05/11/2011 08:20 AM, stoyan wrote:
> ok, initially after reading the qpid docs on setting up clustering i thought
> running the corosync service and qpidd with clustering options is enough,
> but now here's my wild idea - should the machines that i want to setup as
> qpid cluster be already in a linux cluster? i am not an admin, but while
> fighting this problem there were many mentions of aisexec, cman,
> cluster.conf, so am i right that the servers should be in a working cluster
> before attempting to run qpid brokers in cluster? after all even to me it is
> strange to run qpidd --cluster-name=BlahBlahCluster with the name
> BlahBlahCluster coming out of nowhere
>
You dont need to start cman etc. to start a qpid cluster. It is recommended you
use cman because without it the qpidd cluster will not be able to handle network
partitions, but I suggest you first get yourself running without cman. CPG can
create process groups on the fly as processes join, the cluster-name is the name
of a CPG group.
"Failed to initialize CPG.: try again (6)" sometimes indicates a problem with
the openais.conf (or corosync.conf) Have you checked that against the checklist
https://cwiki.apache.org/qpid/starting-a-cluster.html ? If you mail your
corosync.conf and the output if ifconfig I'll take a look.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid clustering: Failed to initialize CPG
Posted by stoyan <st...@hotmail.com>.
ok, initially after reading the qpid docs on setting up clustering i thought
running the corosync service and qpidd with clustering options is enough,
but now here's my wild idea - should the machines that i want to setup as
qpid cluster be already in a linux cluster? i am not an admin, but while
fighting this problem there were many mentions of aisexec, cman,
cluster.conf, so am i right that the servers should be in a working cluster
before attempting to run qpid brokers in cluster? after all even to me it is
strange to run qpidd --cluster-name=BlahBlahCluster with the name
BlahBlahCluster coming out of nowhere
please confirm
thanks!
stoyan
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/qpid-clustering-Failed-to-initialize-CPG-tp6348587p6351614.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid clustering: Failed to initialize CPG
Posted by stoyan <st...@hotmail.com>.
what you suggested is what i am doing already:
/sbin/service corosync start, then
qpidd -d --config=/etc/qpid/qpid.conf --cluster-name=QpidCL
are there other qpid options i should be using? how does qpid know how to
reach the corosync cluster? maybe it is using another config file than what
the corosync service is using? when i gdb-ed into qpid source it looped on
cpg_initialize(), i guess it is part of corosync libs and knows how it is
configured, but..
this is my qpid.conf:
port=4672
auth=yes
data-dir=/etc/qpid/data/
log-time=yes
trace=yes
log-to-file=/etc/qpid/qpidd.log
acl-file=/etc/qpid/acl_list.acl
load-module=/usr/local/lib/qpid/daemon/msgstore.so
sasl-config=/etc/qpid/sasl.conf
using 'cluster-name' inside or outside the conf makes no difference
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/qpid-clustering-Failed-to-initialize-CPG-tp6348587p6350890.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid clustering: Failed to initialize CPG
Posted by jeffin_m <je...@yahoo.in>.
Hi,
Please start corosync service first and then try to start the qpidd with
cluster options. Hope this solves you problem.
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/qpid-clustering-Failed-to-initialize-CPG-tp6348587p6350519.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: [resolved]Re: qpid clustering: Failed to initialize CPG
Posted by stoyan <st...@hotmail.com>.
i checked that, configs for corosync and openais seem to be equivalent, i
created the uidgid files and put my user in ais group, but nothing worked,
selinux is disabled.
anyways - i'm moving on with corosync - newer is better, right? :)
many thanks for the great product!
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/qpid-clustering-Failed-to-initialize-CPG-tp6348587p6368772.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: [resolved]Re: qpid clustering: Failed to initialize CPG
Posted by Alan Conway <ac...@redhat.com>.
On 05/13/2011 09:56 AM, stoyan wrote:
> after some struggle i managed to run a 0.10 broker in cluster on my local
> machine and started digging why it wouldn't start on the other servers.
> after some strace-ing i noticed that on the local server qpid attempted to
> connect(10, {sa_family=AF_FILE, path=@"corosync.ipc"}, 110) = 0
> while on the other machines it failed when trying to connect(10,
> {sa_family=AF_FILE, path=@"libais.socket"...}, 110) = -1 ECONNREFUSED
> (Connection refused)
>
> now apparently it would fail because i started corosync service, not
> openais.
>
> on the server i have openais 0.80.6 (yum installed); i manually built and
> installed corosync 1.3.0. qpid's configure found both and i don't know what
> decisions it made, but eventually i only managed to start qpid with cluster
> on a machine without ais and i subsequently i gave up starting openais
> service after a bunch of these (with valid confs in proper locations, i
> believe):
>
> aisexec: [MAIN ] AIS Executive Service RELEASE 'subrev 1887 version 0.80.6'
> aisexec: [MAIN ] Copyright (C) 2002-2006 MontaVista Software, Inc and
> contributors.
> aisexec: [MAIN ] Copyright (C) 2006 Red Hat, Inc.
> aisexec: [MAIN ] AIS Executive Service: started and ready to provide
> service.
> aisexec: [MAIN ] parse error in config: No interfaces defined
> aisexec: [MAIN ] AIS Executive exiting (reason: could not read the main
> configuration file).
>
That may be a problem in your AIS configuration. There's a checklist at
https://cwiki.apache.org/qpid/starting-a-cluster.html
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
[resolved]Re: qpid clustering: Failed to initialize CPG
Posted by stoyan <st...@hotmail.com>.
after some struggle i managed to run a 0.10 broker in cluster on my local
machine and started digging why it wouldn't start on the other servers.
after some strace-ing i noticed that on the local server qpid attempted to
connect(10, {sa_family=AF_FILE, path=@"corosync.ipc"}, 110) = 0
while on the other machines it failed when trying to connect(10,
{sa_family=AF_FILE, path=@"libais.socket"...}, 110) = -1 ECONNREFUSED
(Connection refused)
now apparently it would fail because i started corosync service, not
openais.
on the server i have openais 0.80.6 (yum installed); i manually built and
installed corosync 1.3.0. qpid's configure found both and i don't know what
decisions it made, but eventually i only managed to start qpid with cluster
on a machine without ais and i subsequently i gave up starting openais
service after a bunch of these (with valid confs in proper locations, i
believe):
aisexec: [MAIN ] AIS Executive Service RELEASE 'subrev 1887 version 0.80.6'
aisexec: [MAIN ] Copyright (C) 2002-2006 MontaVista Software, Inc and
contributors.
aisexec: [MAIN ] Copyright (C) 2006 Red Hat, Inc.
aisexec: [MAIN ] AIS Executive Service: started and ready to provide
service.
aisexec: [MAIN ] parse error in config: No interfaces defined
aisexec: [MAIN ] AIS Executive exiting (reason: could not read the main
configuration file).
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/qpid-clustering-Failed-to-initialize-CPG-tp6348587p6359515.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org