You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Ted Ross (JIRA)" <qp...@incubator.apache.org> on 2010/03/17 23:45:27 UTC
[jira] Created: (QPID-2452) Inconsistent handling on strings
between C++ and Python messaging APIs
Inconsistent handling on strings between C++ and Python messaging APIs
----------------------------------------------------------------------
Key: QPID-2452
URL: https://issues.apache.org/jira/browse/QPID-2452
Project: Qpid
Issue Type: Bug
Components: C++ Client, Python Client
Reporter: Ted Ross
Fix For: 0.7
Description of problem:
This bug is in reference to the new messaging APIs.
The handling of strings is different between the C++ and Python messaging APIs.
The Python API assumes strings on-the-wire are UTF-8 encoded. The C++ API
apparently uses raw, unencoded octet arrays.
If a binary string of octets (with some octets > 0x7F), is encoded by a C++
client and received by a Python client, the Python client will throw an
exception.
Version-Release number of selected component (if applicable):
SVN revision 924529 and prior.
How reproducible:
100%
Steps to Reproduce:
1. Use the C++ API (qpid::messaging) to produce a map-message (using
MapContent). One of the map entries should have a string value and should
contain a sequence such as "!E\xf9\xf5\xdf\x89d\x011\xc0\xc8$7H\x99T"
2. Use the python client to receive the message.
Actual results:
The Python client will throw an exception when it tries to UTF8-decode the
string.
Expected results:
Either Python should use raw-octet encoding or C++ should use UTF8. I expect
the string seen by the Python receiver to be identical to that sent by the C++
producer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Assigned: (QPID-2452) Inconsistent handling on strings
between C++ and Python messaging APIs
Posted by "Gordon Sim (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gordon Sim reassigned QPID-2452:
--------------------------------
Assignee: Gordon Sim
> Inconsistent handling on strings between C++ and Python messaging APIs
> ----------------------------------------------------------------------
>
> Key: QPID-2452
> URL: https://issues.apache.org/jira/browse/QPID-2452
> Project: Qpid
> Issue Type: Bug
> Components: C++ Client, Python Client
> Reporter: Ted Ross
> Assignee: Gordon Sim
> Fix For: 0.7
>
>
> Description of problem:
> This bug is in reference to the new messaging APIs.
> The handling of strings is different between the C++ and Python messaging APIs.
> The Python API assumes strings on-the-wire are UTF-8 encoded. The C++ API
> apparently uses raw, unencoded octet arrays.
> If a binary string of octets (with some octets > 0x7F), is encoded by a C++
> client and received by a Python client, the Python client will throw an
> exception.
> Version-Release number of selected component (if applicable):
> SVN revision 924529 and prior.
> How reproducible:
> 100%
> Steps to Reproduce:
> 1. Use the C++ API (qpid::messaging) to produce a map-message (using
> MapContent). One of the map entries should have a string value and should
> contain a sequence such as "!E\xf9\xf5\xdf\x89d\x011\xc0\xc8$7H\x99T"
> 2. Use the python client to receive the message.
> Actual results:
> The Python client will throw an exception when it tries to UTF8-decode the
> string.
> Expected results:
> Either Python should use raw-octet encoding or C++ should use UTF8. I expect
> the string seen by the Python receiver to be identical to that sent by the C++
> producer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Resolved: (QPID-2452) Inconsistent handling on strings
between C++ and Python messaging APIs
Posted by "Gordon Sim (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gordon Sim resolved QPID-2452.
------------------------------
Resolution: Fixed
> Inconsistent handling on strings between C++ and Python messaging APIs
> ----------------------------------------------------------------------
>
> Key: QPID-2452
> URL: https://issues.apache.org/jira/browse/QPID-2452
> Project: Qpid
> Issue Type: Bug
> Components: C++ Client, Python Client
> Reporter: Ted Ross
> Assignee: Gordon Sim
> Fix For: 0.7
>
>
> Description of problem:
> This bug is in reference to the new messaging APIs.
> The handling of strings is different between the C++ and Python messaging APIs.
> The Python API assumes strings on-the-wire are UTF-8 encoded. The C++ API
> apparently uses raw, unencoded octet arrays.
> If a binary string of octets (with some octets > 0x7F), is encoded by a C++
> client and received by a Python client, the Python client will throw an
> exception.
> Version-Release number of selected component (if applicable):
> SVN revision 924529 and prior.
> How reproducible:
> 100%
> Steps to Reproduce:
> 1. Use the C++ API (qpid::messaging) to produce a map-message (using
> MapContent). One of the map entries should have a string value and should
> contain a sequence such as "!E\xf9\xf5\xdf\x89d\x011\xc0\xc8$7H\x99T"
> 2. Use the python client to receive the message.
> Actual results:
> The Python client will throw an exception when it tries to UTF8-decode the
> string.
> Expected results:
> Either Python should use raw-octet encoding or C++ should use UTF8. I expect
> the string seen by the Python receiver to be identical to that sent by the C++
> producer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-2452) Inconsistent handling on strings
between C++ and Python messaging APIs
Posted by "Ted Ross (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12846894#action_12846894 ]
Ted Ross commented on QPID-2452:
--------------------------------
You are correct. The tagging on the wire is 0x95 (which is called str16 and is, according to the spec, utf8 encoded).
The C++ API does not distinguish between str16 and vbin16. It only supplies a VAR_STRING type. So, if it is going to encode strings into str16 values, it must utf8 encode. Alternatively, it could use the vbin16 tag.
-Ted
> Inconsistent handling on strings between C++ and Python messaging APIs
> ----------------------------------------------------------------------
>
> Key: QPID-2452
> URL: https://issues.apache.org/jira/browse/QPID-2452
> Project: Qpid
> Issue Type: Bug
> Components: C++ Client, Python Client
> Reporter: Ted Ross
> Assignee: Gordon Sim
> Fix For: 0.7
>
>
> Description of problem:
> This bug is in reference to the new messaging APIs.
> The handling of strings is different between the C++ and Python messaging APIs.
> The Python API assumes strings on-the-wire are UTF-8 encoded. The C++ API
> apparently uses raw, unencoded octet arrays.
> If a binary string of octets (with some octets > 0x7F), is encoded by a C++
> client and received by a Python client, the Python client will throw an
> exception.
> Version-Release number of selected component (if applicable):
> SVN revision 924529 and prior.
> How reproducible:
> 100%
> Steps to Reproduce:
> 1. Use the C++ API (qpid::messaging) to produce a map-message (using
> MapContent). One of the map entries should have a string value and should
> contain a sequence such as "!E\xf9\xf5\xdf\x89d\x011\xc0\xc8$7H\x99T"
> 2. Use the python client to receive the message.
> Actual results:
> The Python client will throw an exception when it tries to UTF8-decode the
> string.
> Expected results:
> Either Python should use raw-octet encoding or C++ should use UTF8. I expect
> the string seen by the Python receiver to be identical to that sent by the C++
> producer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-2452) Inconsistent handling on strings
between C++ and Python messaging APIs
Posted by "Rafael H. Schloming (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12846666#action_12846666 ]
Rafael H. Schloming commented on QPID-2452:
-------------------------------------------
The python client doesn't assume anything about the strings on the wire, it just looks at the typecode. If the typecode says it is UTF-8 data then it will decode as UTF-8. If the tyepcode says it is raw binary then it will decode as raw binary. From what you're describing it sounds to me like we're somehow putting raw octets onto the wire but incorrectly tagging them as UTF-8.
> Inconsistent handling on strings between C++ and Python messaging APIs
> ----------------------------------------------------------------------
>
> Key: QPID-2452
> URL: https://issues.apache.org/jira/browse/QPID-2452
> Project: Qpid
> Issue Type: Bug
> Components: C++ Client, Python Client
> Reporter: Ted Ross
> Fix For: 0.7
>
>
> Description of problem:
> This bug is in reference to the new messaging APIs.
> The handling of strings is different between the C++ and Python messaging APIs.
> The Python API assumes strings on-the-wire are UTF-8 encoded. The C++ API
> apparently uses raw, unencoded octet arrays.
> If a binary string of octets (with some octets > 0x7F), is encoded by a C++
> client and received by a Python client, the Python client will throw an
> exception.
> Version-Release number of selected component (if applicable):
> SVN revision 924529 and prior.
> How reproducible:
> 100%
> Steps to Reproduce:
> 1. Use the C++ API (qpid::messaging) to produce a map-message (using
> MapContent). One of the map entries should have a string value and should
> contain a sequence such as "!E\xf9\xf5\xdf\x89d\x011\xc0\xc8$7H\x99T"
> 2. Use the python client to receive the message.
> Actual results:
> The Python client will throw an exception when it tries to UTF8-decode the
> string.
> Expected results:
> Either Python should use raw-octet encoding or C++ should use UTF8. I expect
> the string seen by the Python receiver to be identical to that sent by the C++
> producer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-2452) Inconsistent handling on strings
between C++ and Python messaging APIs
Posted by "Gordon Sim (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847059#action_12847059 ]
Gordon Sim commented on QPID-2452:
----------------------------------
The amqp 0-10 codec now correctly looks at the encoding of a string valued variant in deciding which type to encode it as. By default it will send it as vbin16 (vbin32 if too large fr that). However by setting the encoding to "utf8", "utf16" or "iso-8859-15" the user can alter that if desired. However the user is responsible for ensuring that the string is indeed valid for the specified encoding. No conversion is done at present.
> Inconsistent handling on strings between C++ and Python messaging APIs
> ----------------------------------------------------------------------
>
> Key: QPID-2452
> URL: https://issues.apache.org/jira/browse/QPID-2452
> Project: Qpid
> Issue Type: Bug
> Components: C++ Client, Python Client
> Reporter: Ted Ross
> Assignee: Gordon Sim
> Fix For: 0.7
>
>
> Description of problem:
> This bug is in reference to the new messaging APIs.
> The handling of strings is different between the C++ and Python messaging APIs.
> The Python API assumes strings on-the-wire are UTF-8 encoded. The C++ API
> apparently uses raw, unencoded octet arrays.
> If a binary string of octets (with some octets > 0x7F), is encoded by a C++
> client and received by a Python client, the Python client will throw an
> exception.
> Version-Release number of selected component (if applicable):
> SVN revision 924529 and prior.
> How reproducible:
> 100%
> Steps to Reproduce:
> 1. Use the C++ API (qpid::messaging) to produce a map-message (using
> MapContent). One of the map entries should have a string value and should
> contain a sequence such as "!E\xf9\xf5\xdf\x89d\x011\xc0\xc8$7H\x99T"
> 2. Use the python client to receive the message.
> Actual results:
> The Python client will throw an exception when it tries to UTF8-decode the
> string.
> Expected results:
> Either Python should use raw-octet encoding or C++ should use UTF8. I expect
> the string seen by the Python receiver to be identical to that sent by the C++
> producer.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org