You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Keith W <ke...@gmail.com> on 2016/12/19 15:57:02 UTC
AMQP 1.0/Python - Question re. encoding of maps and lists : failing
qpid_tests.broker_1_0.translation.TranslationTests.test_translate* against
Java Broker
Hi all,
The qpid_tests.broker_1_0.translation.TranslationTests.test_translate*
tests fail against the Java Broker. This is not a new problem. The
tests have been excluded against the Java Broker since I first enabled
the Jenkins job about six months ago (pending investigation)
The test I am focusing on is:
qpid_tests.broker_1_0.translation.TranslationTests.test_translate_map_2
This tests the ability of the Broker to translate a map sent from a
client publishing with AMQP 1.0 to a receiver using AMQP 0-10. When
the test is run against the Java Broker, the test fails as follows:
fail
Error during test: Traceback (most recent call last):
File "/home/keith/tmp/py/bin/qpid-python-test", line 340, in run
phase()
File "/home/keith/src/qpid-python/qpid_tests/broker_1_0/translation.py",
line 81, in test_translate_map_2
self.send_receive_map("amqp1.0", "amqp0-10", '#')
File "/home/keith/src/qpid-python/qpid_tests/broker_1_0/translation.py",
line 48, in send_receive_map
self.send_receive_messages([Message(content={'s':'abc','i':10})],
send_version, receive_version, address)
File "/home/keith/src/qpid-python/qpid_tests/broker_1_0/translation.py",
line 38, in send_receive_messages
assert msg.content == expected.content, (msg.content, expected.content)
AssertionError:
('\x00\x00\x00\x17\x00\x00\x00\x02\x01i1\x00\x00\x00\x00\x00\x00\x00\n\x01s\x90\x00\x03abc',
{'i': 10L, 's': 'abc'})
I have turned PN_TRACE_FRM=true on client side and I see:
2016-12-19 16:49:13 [Messaging] trace tcp:Oslo.local:5672 encode(65535)
2016-12-19 16:49:13 [Protocol] trace
[14df4d0e-1eb5-45d8-96b9-fffbc15e2665]: 0 -> @transfer(20) [handle=0,
delivery-id=0, delivery-tag=b"\x00\x00\x00\x00", message-format=0,
settled=false, more=false] (75)
"\x00Sp\xc0\x04\x02BP\x00\x00Ss\xd0\x00\x00\x00\x14\x00\x00\x00\x07@@@@@@\xa3\x08amqp/map\x00St\xc1\x01\x00\x00Su\xa0\x1b\x00\x00\x00\x17\x00\x00\x00\x02\x01i1\x00\x00\x00\x00\x00\x00\x00\x0a\x01s\x90\x00\x03abc"
2016-12-19 16:49:13 [Network] debug tcp:Oslo.local:5672 encoded 111
bytes from 65535
By debugging the Java Broker's translation module, I can see that the
AMQP 1.0 publishing end is sending the map encoded within a
DataSection. This surprises me - I was expecting the application-data
to be an amqp-value containing the map. The Java Broker doesn't know
how to translate this for the 0-10 and hence the test fails.
Why does Qpid Python / swigged client encode the map this way?
cheers Keith.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: AMQP 1.0/Python - Question re. encoding of maps and lists :
failing qpid_tests.broker_1_0.translation.TranslationTests.test_translate*
against Java Broker
Posted by Keith W <ke...@gmail.com>.
>> https://qpid.apache.org/components/messaging-api/index.html
>
>
> The qpid.messaging python client (pure python) is of course supported.
> However at present it only speak AMQP 0-10 and no-one has indicated any
> plans to add 1.0 support to it.
>
> The swigged version of the c++ equivalent is a different library (though
> they look very similar obviously), and that is the one we have never pushed
> as an actual end-usable client library (because there already was a pure
> python equivalent).
Ok - I understand the distinction now.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: AMQP 1.0/Python - Question re. encoding of maps and lists :
failing qpid_tests.broker_1_0.translation.TranslationTests.test_translate*
against Java Broker
Posted by Gordon Sim <gs...@redhat.com>.
On 23/12/16 08:34, Keith W wrote:
> On 22 December 2016 at 12:11, Gordon Sim <gs...@redhat.com> wrote:
>> On 21/12/16 17:51, Keith W wrote:
>>>
>>> I believed that Python Messaging derived its AMQP 1.0 support by way
>>> of the swigged client. Are we saying this is deprecated?
>>
>>
>> It is more that it has never been considered a recommended client. It was
>> just something we used in testing.
>
> Ok - I see. I think we should re-visit this page. As an end user I
> would believe the opposite.
>
> https://qpid.apache.org/components/messaging-api/index.html
The qpid.messaging python client (pure python) is of course supported.
However at present it only speak AMQP 0-10 and no-one has indicated any
plans to add 1.0 support to it.
The swigged version of the c++ equivalent is a different library (though
they look very similar obviously), and that is the one we have never
pushed as an actual end-usable client library (because there already was
a pure python equivalent).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: AMQP 1.0/Python - Question re. encoding of maps and lists :
failing qpid_tests.broker_1_0.translation.TranslationTests.test_translate*
against Java Broker
Posted by Keith W <ke...@gmail.com>.
On 22 December 2016 at 12:11, Gordon Sim <gs...@redhat.com> wrote:
> On 21/12/16 17:51, Keith W wrote:
>>
>> I believed that Python Messaging derived its AMQP 1.0 support by way
>> of the swigged client. Are we saying this is deprecated?
>
>
> It is more that it has never been considered a recommended client. It was
> just something we used in testing.
Ok - I see. I think we should re-visit this page. As an end user I
would believe the opposite.
https://qpid.apache.org/components/messaging-api/index.html
>
>> If so,
>> what is the preferred Qpid Python client for AMQP 1.0?
>
>
> The proton client:
> https://qpid.apache.org/releases/qpid-proton-0.16.0/proton/python/book/overview.html
>
>
Understood. Thanks for the clarification.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: AMQP 1.0/Python - Question re. encoding of maps and lists :
failing qpid_tests.broker_1_0.translation.TranslationTests.test_translate*
against Java Broker
Posted by Gordon Sim <gs...@redhat.com>.
On 21/12/16 17:51, Keith W wrote:
> I believed that Python Messaging derived its AMQP 1.0 support by way
> of the swigged client. Are we saying this is deprecated?
It is more that it has never been considered a recommended client. It
was just something we used in testing.
> If so,
> what is the preferred Qpid Python client for AMQP 1.0?
The proton client:
https://qpid.apache.org/releases/qpid-proton-0.16.0/proton/python/book/overview.html
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: AMQP 1.0/Python - Question re. encoding of maps and lists :
failing qpid_tests.broker_1_0.translation.TranslationTests.test_translate*
against Java Broker
Posted by Keith W <ke...@gmail.com>.
On 20 December 2016 at 18:34, Gordon Sim <gs...@redhat.com> wrote:
> On 20/12/16 18:20, Keith W wrote:
>>
>> On 20 December 2016 at 10:14, Gordon Sim <gs...@redhat.com> wrote:
>>>
>>> On 19/12/16 15:57, Keith W wrote:
>>>>
>>>>
>>>> By debugging the Java Broker's translation module, I can see that the
>>>> AMQP 1.0 publishing end is sending the map encoded within a
>>>> DataSection. This surprises me - I was expecting the application-data
>>>> to be an amqp-value containing the map. The Java Broker doesn't know
>>>> how to translate this for the 0-10 and hence the test fails.
>>>>
>>>> Why does Qpid Python / swigged client encode the map this way?
>>>
>>>
>>>
>>> The swig layer has some special logic for managing message content that
>>> predates 1.0 and works the same way for both 1.0 and 0-10. The
>>> content-type
>>> is used to indicate the type in both cases.
>>>
>>> 1.0 required some changes to the messaging API (set-/get-
>>> ContentObject())
>>> to allow value sections to be used. I suspect the swig binding was just
>>> never changed to use those changes.
>>>
>>
>> Thanks Gordon for the swift reply. That all makes sense and I was
>> able to find the code concerned. It matches your suspicions.
>> Do you know if the bindings are due to have their AMQP 1.0 support
>> refreshed?
>
>
> I don't know of any such plan. The swigged client is really only used for
> tests at present.
>
I believed that Python Messaging derived its AMQP 1.0 support by way
of the swigged client. Are we saying this is deprecated? If so,
what is the preferred Qpid Python client for AMQP 1.0?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: AMQP 1.0/Python - Question re. encoding of maps and lists :
failing qpid_tests.broker_1_0.translation.TranslationTests.test_translate*
against Java Broker
Posted by Gordon Sim <gs...@redhat.com>.
On 20/12/16 18:20, Keith W wrote:
> On 20 December 2016 at 10:14, Gordon Sim <gs...@redhat.com> wrote:
>> On 19/12/16 15:57, Keith W wrote:
>>>
>>> By debugging the Java Broker's translation module, I can see that the
>>> AMQP 1.0 publishing end is sending the map encoded within a
>>> DataSection. This surprises me - I was expecting the application-data
>>> to be an amqp-value containing the map. The Java Broker doesn't know
>>> how to translate this for the 0-10 and hence the test fails.
>>>
>>> Why does Qpid Python / swigged client encode the map this way?
>>
>>
>> The swig layer has some special logic for managing message content that
>> predates 1.0 and works the same way for both 1.0 and 0-10. The content-type
>> is used to indicate the type in both cases.
>>
>> 1.0 required some changes to the messaging API (set-/get- ContentObject())
>> to allow value sections to be used. I suspect the swig binding was just
>> never changed to use those changes.
>>
>
> Thanks Gordon for the swift reply. That all makes sense and I was
> able to find the code concerned. It matches your suspicions.
> Do you know if the bindings are due to have their AMQP 1.0 support refreshed?
I don't know of any such plan. The swigged client is really only used
for tests at present.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: AMQP 1.0/Python - Question re. encoding of maps and lists :
failing qpid_tests.broker_1_0.translation.TranslationTests.test_translate*
against Java Broker
Posted by Keith W <ke...@gmail.com>.
On 20 December 2016 at 10:14, Gordon Sim <gs...@redhat.com> wrote:
> On 19/12/16 15:57, Keith W wrote:
>>
>> By debugging the Java Broker's translation module, I can see that the
>> AMQP 1.0 publishing end is sending the map encoded within a
>> DataSection. This surprises me - I was expecting the application-data
>> to be an amqp-value containing the map. The Java Broker doesn't know
>> how to translate this for the 0-10 and hence the test fails.
>>
>> Why does Qpid Python / swigged client encode the map this way?
>
>
> The swig layer has some special logic for managing message content that
> predates 1.0 and works the same way for both 1.0 and 0-10. The content-type
> is used to indicate the type in both cases.
>
> 1.0 required some changes to the messaging API (set-/get- ContentObject())
> to allow value sections to be used. I suspect the swig binding was just
> never changed to use those changes.
>
Thanks Gordon for the swift reply. That all makes sense and I was
able to find the code concerned. It matches your suspicions.
Do you know if the bindings are due to have their AMQP 1.0 support refreshed?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
> For additional commands, e-mail: dev-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: AMQP 1.0/Python - Question re. encoding of maps and lists :
failing qpid_tests.broker_1_0.translation.TranslationTests.test_translate*
against Java Broker
Posted by Gordon Sim <gs...@redhat.com>.
On 19/12/16 15:57, Keith W wrote:
> By debugging the Java Broker's translation module, I can see that the
> AMQP 1.0 publishing end is sending the map encoded within a
> DataSection. This surprises me - I was expecting the application-data
> to be an amqp-value containing the map. The Java Broker doesn't know
> how to translate this for the 0-10 and hence the test fails.
>
> Why does Qpid Python / swigged client encode the map this way?
The swig layer has some special logic for managing message content that
predates 1.0 and works the same way for both 1.0 and 0-10. The
content-type is used to indicate the type in both cases.
1.0 required some changes to the messaging API (set-/get-
ContentObject()) to allow value sections to be used. I suspect the swig
binding was just never changed to use those changes.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org