QMF creating huge durable queue causes cluster desync

[Pavel Moravec <pm...@redhat.com>]

Hi all,
it seems to me that sending QMF methods to create a queue over-passes "cluster error checking mechanism" (if a sole node detects an error, it preventive leaves the cluster).

Having a simple C++ client to create a huge durable queue:

        Connection connection("amqp:tcp:127.0.0.1:5672");
        connection.open();
        Session session = connection.createSession();
        Sender sender = session.createSender("qmf.default.direct/broker");
        Address responseQueue("#reply-queue; {create:always, node:{x-declare:{auto-delete:true}}}");
        Receiver receiver = session.createReceiver(responseQueue);

        Message message;
        Variant::Map content;
	Variant::Map OID;
	Variant::Map arguments;
	Variant::Map properties;

	properties["durable"] = "true";
        properties["qpid.file_count"] = "3000";

	arguments["type"] = "queue";
	arguments["name"] = "HugeDurableQueue";
	arguments["properties"] = properties;
	
	OID["_object_name"] = "org.apache.qpid.broker:broker:amqp-broker";
        content["_object_id"] = OID;
        content["_method_name"] = "create";
        content["_arguments"] = arguments;
	
        encode(content, message);
	message.setReplyTo(responseQueue);
	message.setProperty("x-amqp-0-10.app-id", "qmf2");
	message.setProperty("qmf.opcode", "_method_request");
	message.setContentType("amqp/map");

        sender.send(message, true);

and running the client to a clustered pair of brokers creates a durable queue with 3000 files in its journal.

However, if one of the brokers (let say node1) has insufficient free disk space, qpid on node1 shall shut down due to the "cluster error checking mechanism". At least, that I see when I create the queue via qpid-config or using proper address string and spout/drain.

But running the above QMF method leaves the cluster in state where node1 has not have the queue created while node2 has it created - a substantial cluster de-sync.

Did I do something wrong or is it a bug?

Thanks in advance for your answers.


Kind regards,
Pavel



---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

Re: QMF creating huge durable queue causes cluster desync

[Alan Conway <ac...@redhat.com>]

On 01/30/2012 07:04 AM, Pavel Moravec wrote:
> Hi all,
> it seems to me that sending QMF methods to create a queue over-passes "cluster error checking mechanism" (if a sole node detects an error, it preventive leaves the cluster).
>
> Having a simple C++ client to create a huge durable queue:
>
>          Connection connection("amqp:tcp:127.0.0.1:5672");
>          connection.open();
>          Session session = connection.createSession();
>          Sender sender = session.createSender("qmf.default.direct/broker");
>          Address responseQueue("#reply-queue; {create:always, node:{x-declare:{auto-delete:true}}}");
>          Receiver receiver = session.createReceiver(responseQueue);
>
>          Message message;
>          Variant::Map content;
> 	Variant::Map OID;
> 	Variant::Map arguments;
> 	Variant::Map properties;
>
> 	properties["durable"] = "true";
>          properties["qpid.file_count"] = "3000";
>
> 	arguments["type"] = "queue";
> 	arguments["name"] = "HugeDurableQueue";
> 	arguments["properties"] = properties;
> 	
> 	OID["_object_name"] = "org.apache.qpid.broker:broker:amqp-broker";
>          content["_object_id"] = OID;
>          content["_method_name"] = "create";
>          content["_arguments"] = arguments;
> 	
>          encode(content, message);
> 	message.setReplyTo(responseQueue);
> 	message.setProperty("x-amqp-0-10.app-id", "qmf2");
> 	message.setProperty("qmf.opcode", "_method_request");
> 	message.setContentType("amqp/map");
>
>          sender.send(message, true);
>
> and running the client to a clustered pair of brokers creates a durable queue with 3000 files in its journal.
>
> However, if one of the brokers (let say node1) has insufficient free disk space, qpid on node1 shall shut down due to the "cluster error checking mechanism". At least, that I see when I create the queue via qpid-config or using proper address string and spout/drain.
>
> But running the above QMF method leaves the cluster in state where node1 has not have the queue created while node2 has it created - a substantial cluster de-sync.
>
> Did I do something wrong or is it a bug?
>
> Thanks in advance for your answers.
>

I think that is a bug. Haven't looked at the code but I would guess the QMF 
errors are sent as QMF response messages, and not raised as AMQP errors, which 
is what the cluster code is looking out for.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org