You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by smartdog <jw...@gmail.com> on 2014/03/05 03:08:41 UTC
Does qpid federation work with Azure Service Bus
We need post some messages to Azure Service Bus and figure it would be nice
(in case Azure service bus is down) to post messages to the local qpid
broker first and then let the broker propagate messages to Azure.
I tried with
qpid-route -d queue add azureuser/password@namespace.servicebus.windows.net
127.0.0.1:5672 amq.fanout brokerqueue
and got Failed: VersionError - client: 0-10, server: 0-0
It seems related with version conflict (Azure supports amqp 1.0, while
broker is qmap 0-10)
Any idea or this just won't work? Thanks.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Does-qpid-federation-work-with-Azure-Service-Bus-tp7605139.html
Sent from the Apache Qpid developers mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by smartdog <jw...@gmail.com>.
Thanks for the reply, it is helpful. Ran into an issue with qpid-config add
outgoing my-out-link.
This is fine:
qpid-config -b 127.0.0.1:5672 add domain my-servicebus --argument
url=mynamespace.servicebus.windows.net --argument username=myusername
--argument password=mypassword= --argument sasl_mechanisms=PLAIN
This failed.
qpid-config add outgoing my-out-link --argument domain=my-servicebus
--argument source=brokerqueue --argument target=azuretopic
Logs show:
[Broker] notice Created domain my-servicebus with url
amqp:tcp:mynamespace.servicebus.windows.net:5672 from {password:mypassowrd=,
url:mynamespace.servicebus.windows.net, username:myusername}
[Broker] notice Creating interconnect my-out-link, {domain:my-servicebus,
source:brokerqueue, target:azuretopic}
[Management] debug Management object (V1) added:
org.apache.qpid.broker:connection:my-out-link
[Broker] info Using AMQP 1.0 (with SASL layer) on connect
[System] debug SENT [my-out-link]: INIT(1-0)
[Protocol] debug my-out-link read protocol header: 1-0
[Protocol] debug my-out-link Received SASL-MECHANISMS(PLAIN EXTERNAL )
[System] debug Exception constructed: No suitable mechanism!
[System] error No suitable mechanism!
[Broker] notice Interconnect deleted
--
View this message in context: http://qpid.2158936.n2.nabble.com/Re-Does-qpid-federation-work-with-Azure-Service-Bus-tp7605151p7605253.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by Gordon Sim <gs...@redhat.com>.
On 03/07/2014 06:35 PM, smartdog wrote:
> Thanks for the help.
> installed this
> sudo apt-get install libsasl2-dev
>
> Unfortunately still have the same issue:
> cmake ..
> -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
> -- Could NOT find VALGRIND (missing: VALGRIND_EXECUTABLE)
> -- Could NOT find SASL (missing: FOUND_SASL_LIB FOUND_SASL_H)
Did libsasl2-dev install header files? If so where did it install them to?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by smartdog <jw...@gmail.com>.
It turns out the /etc/sasl2/qpidd.conf comes from the apt-get install qpidd
pointing to a different db location. Change that address the SASL issue.
But still I cannot get the link working. I end up writing a shovel program
that fetches messages from local queue and resend them to Azure.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Re-Does-qpid-federation-work-with-Azure-Service-Bus-tp7605151p7605665.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by Gordon Sim <gs...@redhat.com>.
On 03/12/2014 10:53 PM, smartdog wrote:
> Thanks for the reply. I managed to get SASL start with the broker by
> rebuilding. However, when I test with proton, it failed with these logs.
>
> my endpoint
> amqp://test:test1@127.0.0.1:5672/myqueue
>
> send exe output:
> recv: Connection refused
> [0x99e830]:ERROR[-2] SASL header mismatch: ''
>
> I do have a user
> sasldblistusers2 -f /var/lib/qpidd/qpidd.sasldb
> test@QPID: userPassword
What does your /etc/sasl2/qpidd.conf have in it? I wonder if qpidd is
not finding the password db. (Also, just to check, which versions are
you using for qpidd and proton?)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by smartdog <jw...@gmail.com>.
Thanks for the reply. I managed to get SASL start with the broker by
rebuilding. However, when I test with proton, it failed with these logs.
my endpoint
amqp://test:test1@127.0.0.1:5672/myqueue
send exe output:
recv: Connection refused
[0x99e830]:ERROR[-2] SASL header mismatch: ''
I do have a user
sasldblistusers2 -f /var/lib/qpidd/qpidd.sasldb
test@QPID: userPassword
Start qpid
sudo src/qpidd --daemon --config /etc/qpid/qpidd.conf --realm QPID
[Store] warning Message store plugin: No storage providers available.
[Security] info SASL: no config path set - using default.
[Broker] info SASL enabled
[Network] debug No Socket fd specified
[Security] info Policy file not specified. ACL Disabled, no ACL checking
being done!
[Network] debug Using interface:
[Network] info Listening to: 0.0.0.0:5672
[Network] debug Listened to: 5672
[Network] notice Listening on TCP/TCP6 port 5672
[Broker] debug Daemon ready on port: 5672
[Broker] notice Broker running
[Network] info Set TCP_NODELAY on connection to 127.0.0.1:50786
[System] debug RECV [qpid.127.0.0.1:5672-127.0.0.1:50786]: INIT(1-0)
[Broker] info Using AMQP 1.0 (with SASL layer)
[Security] debug External ssf=0 and auth=
[Security] debug min_ssf: 0, max_ssf: 256, external_ssf: 0
[Model] trace Mgmt create connection. id:qpid.127.0.0.1:5672-127.0.0.1:50786
[Management] debug Management object (V1) added:
org.apache.qpid.broker:connection:qpid.127.0.0.1:5672-127.0.0.1:50786
[Security] info SASL: Mechanism list: GS2-IAKERB GS2-KRB5 SCRAM-SHA-1 GSSAPI
DIGEST-MD5 NTLM CRAM-MD5 LOGIN PLAIN ANONYMOUS
[Security] trace Completed encoding of frame of 110 bytes
[Protocol] debug qpid.127.0.0.1:5672-127.0.0.1:50786 Sent
SASL-MECHANISMS(GS2-IAKERB GS2-KRB5 SCRAM-SHA-1 GSSAPI DIGEST-MD5 NTLM
CRAM-MD5 LOGIN PLAIN ANONYMOUS) 110
[Security] trace Reading SASL frame of size 40
[Security] trace Reading SASL-INIT
[Protocol] debug qpid.127.0.0.1:5672-127.0.0.1:50786 Received
SASL-INIT(PLAIN, \x00test\x00test1)
[Security] info SASL: Starting authentication with mechanism: PLAIN
[Security] trace Completed encoding of frame of 16 bytes
[Protocol] debug qpid.127.0.0.1:5672-127.0.0.1:50786 Sent SASL-OUTCOME(1)
126
[Security] info qpid.127.0.0.1:5672-127.0.0.1:50786 Failed to authenticate
--
View this message in context: http://qpid.2158936.n2.nabble.com/Re-Does-qpid-federation-work-with-Azure-Service-Bus-tp7605151p7605543.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by Andrew Stitcher <as...@redhat.com>.
On Fri, 2014-03-07 at 10:35 -0800, smartdog wrote:
> Thanks for the help.
> installed this
> sudo apt-get install libsasl2-dev
>
> Unfortunately still have the same issue:
> cmake ..
> -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
> -- Could NOT find VALGRIND (missing: VALGRIND_EXECUTABLE)
> -- Could NOT find SASL (missing: FOUND_SASL_LIB FOUND_SASL_H)
>
Did you start with a new CMake build directory this time or was there an
existing CMakeCache.txt file? I think that this detection in our cmake
script will keep on failing if you don't remove the CMakeCache file and
start cmake with no prior configuration.
We should fix this.
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by smartdog <jw...@gmail.com>.
Thanks for the help.
installed this
sudo apt-get install libsasl2-dev
Unfortunately still have the same issue:
cmake ..
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Could NOT find VALGRIND (missing: VALGRIND_EXECUTABLE)
-- Could NOT find SASL (missing: FOUND_SASL_LIB FOUND_SASL_H)
sudo src/qpidd --daemon --config /etc/qpidd.conf --load-module amqp.so
--log-to-stdout yes --log-to-syslog yes --log-enable debug+ --data-dir
/var/qpiddata --realm QPID
Daemon startup failed: Requested authentication but SASL unavailable
2014-03-07 10:35:28 [Broker] critical Unexpected error: Daemon startup
failed: Requested authentication but SASL unavailable
--
View this message in context: http://qpid.2158936.n2.nabble.com/Re-Does-qpid-federation-work-with-Azure-Service-Bus-tp7605151p7605314.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by Gordon Sim <gs...@redhat.com>.
On 03/07/2014 01:18 AM, smartdog wrote:
> This may be related:
>
> When I turn on authN
> /etc/qpidd.conf
> auth=yes
> realm=QPID
>
> I got
> sudo src/qpidd --daemon --config /etc/qpidd.conf --load-module amqp.so
> --log-to-stdout yes --log-to-syslog yes --log-enable info+ --data-dir
> /var/qpiddata
> [Broker] critical Unexpected error: Requested authentication but SASL
> unavailable
>
> I do have a user.
> sasldblistusers2 -f /var/lib/qpidd/qpidd.sasldb
> test@QPID: userPassword
>
> I installed:
> sudo apt-get install sasl2-bin
> cyrus-sasl-2.1.26
>
> ~/qpid-cpp-0.26/build$ cmake ..
> -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
> -- Could NOT find VALGRIND (missing: VALGRIND_EXECUTABLE)
> -- Could NOT find SASL (missing: FOUND_SASL_LIB FOUND_SASL_H)
>
> I think I am missing SASL but how can I get it?
On fedora, the package with the headers is called cyrus-sasl-devel. I
think its called 'libsasl2-dev' on debian (or debian based)
distributions[1](?) You will also need libsasl2-modules.
[1] Based on https://packages.debian.org/source/wheezy/i386/cyrus-sasl2
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by smartdog <jw...@gmail.com>.
This may be related:
When I turn on authN
/etc/qpidd.conf
auth=yes
realm=QPID
I got
sudo src/qpidd --daemon --config /etc/qpidd.conf --load-module amqp.so
--log-to-stdout yes --log-to-syslog yes --log-enable info+ --data-dir
/var/qpiddata
[Broker] critical Unexpected error: Requested authentication but SASL
unavailable
I do have a user.
sasldblistusers2 -f /var/lib/qpidd/qpidd.sasldb
test@QPID: userPassword
I installed:
sudo apt-get install sasl2-bin
cyrus-sasl-2.1.26
~/qpid-cpp-0.26/build$ cmake ..
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Could NOT find VALGRIND (missing: VALGRIND_EXECUTABLE)
-- Could NOT find SASL (missing: FOUND_SASL_LIB FOUND_SASL_H)
I think I am missing SASL but how can I get it?
Thanks.
--
View this message in context: http://qpid.2158936.n2.nabble.com/Re-Does-qpid-federation-work-with-Azure-Service-Bus-tp7605151p7605254.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by Gordon Sim <gs...@redhat.com>.
On 03/05/2014 02:08 AM, smartdog wrote:
> We need post some messages to Azure Service Bus and figure it would be nice
> (in case Azure service bus is down) to post messages to the local qpid
> broker first and then let the broker propagate messages to Azure.
>
> I tried with
>
> qpid-route -d queue add azureuser/password@namespace.servicebus.windows.net
> 127.0.0.1:5672 amq.fanout brokerqueue
>
> and got Failed: VersionError - client: 0-10, server: 0-0
>
> It seems related with version conflict (Azure supports amqp 1.0, while
> broker is qmap 0-10)
>
> Any idea or this just won't work? Thanks.
The federation features controlled by qpid-route are based on AMQP 0-10.
There is a similar mechanism for AMQP 1.0 but I haven't ever tested it
against ServiceBus, though it hasn't been used much yet.
To establish links (in either direction), you first need to create a
'domain' defining the connection details for the process you want to
connect to (in your case the service bus). This of course also assumes
that the AMQP 1.0 module is loaded.
If you are using qpid-config from 0.26 you can do:
qpid-config -b 127.0.0.1:5672 add domain my-servicebus --argument
url=namespace.servicebus.windows.net --argument username=azureuser
--argument password=password --argument sasl_mechanisms=PLAIN
If you can't use the latest qpid-config, you can use the qpidt testing
tool from 0.24[1].
Now you have a domain you can create incoming or outgoing links to the
service identified by the domain. Again you can use qpid-config for this
since 0.26, or qpidt if you don't have the latest version of that.
qpid-config add incoming my-in-link --argument domain=my-servicebus
--argument source=node-on-servicebus --argument target=node-on-qpidd
Which should cause messages from node-on-servicebus to be pulled into
qpidd, into a node (queue or exchange) called node-on-qpidd.
You can also create links in the other direction, where the type is
outgoing rather than incoming. (You can also create senders or receivers
on a qpidd session to/from e.g. a-node@my-servicebus, and qpid will
establish links to the node on service bus over which messages, flow and
dispositions are relayed. I.e. you can send to or receive from any node
on the service bus, via qpidd).
One current limitation of this 1.0 based mechanism is that it won't try
to re-establish any failed connections.
--Gordon
[1]
http://svn.apache.org/viewvc/qpid/branches/0.24/qpid/cpp/src/tests/qpidt?view=markup,
with this tool the format is slightly different, e.g.
qpidt create domain my-servicebus url=namespace.servicebus.windows.net
username=azureuser password=password sasl_mechanisms=PLAIN
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does qpid federation work with Azure Service Bus
Posted by Gordon Sim <gs...@redhat.com>.
On 03/05/2014 02:08 AM, smartdog wrote:
> We need post some messages to Azure Service Bus and figure it would be nice
> (in case Azure service bus is down) to post messages to the local qpid
> broker first and then let the broker propagate messages to Azure.
>
> I tried with
>
> qpid-route -d queue add azureuser/password@namespace.servicebus.windows.net
> 127.0.0.1:5672 amq.fanout brokerqueue
>
> and got Failed: VersionError - client: 0-10, server: 0-0
>
> It seems related with version conflict (Azure supports amqp 1.0, while
> broker is qmap 0-10)
>
> Any idea or this just won't work? Thanks.
The federation features controlled by qpid-route are based on AMQP 0-10.
There is a similar mechanism for AMQP 1.0 but I haven't ever tested it
against ServiceBus, though it hasn't been used much yet.
To establish links (in either direction), you first need to create a
'domain' defining the connection details for the process you want to
connect to (in your case the service bus). This of course also assumes
that the AMQP 1.0 module is loaded.
If you are using qpid-config from 0.26 you can do:
qpid-config -b 127.0.0.1:5672 add domain my-servicebus --argument
url=namespace.servicebus.windows.net --argument username=azureuser
--argument password=password --argument sasl_mechanisms=PLAIN
If you can't use the latest qpid-config, you can use the qpidt testing
tool from 0.24[1].
Now you have a domain you can create incoming or outgoing links to the
service identified by the domain. Again you can use qpid-config for this
since 0.26, or qpidt if you don't have the latest version of that.
qpid-config add incoming my-in-link --argument domain=my-servicebus
--argument source=node-on-servicebus --argument target=node-on-qpidd
Which should cause messages from node-on-servicebus to be pulled into
qpidd, into a node (queue or exchange) called node-on-qpidd.
You can also create links in the other direction, where the type is
outgoing rather than incoming. (You can also create senders or receivers
on a qpidd session to/from e.g. a-node@my-servicebus, and qpid will
establish links to the node on service bus over which messages, flow and
dispositions are relayed. I.e. you can send to or receive from any node
on the service bus, via qpidd).
One current limitation of this 1.0 based mechanism is that it won't try
to re-establish any failed connections.
--Gordon
[1]
http://svn.apache.org/viewvc/qpid/branches/0.24/qpid/cpp/src/tests/qpidt?view=markup,
with this tool the format is slightly different, e.g.
qpidt create domain my-servicebus url=namespace.servicebus.windows.net
username=azureuser password=password sasl_mechanisms=PLAIN
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org