You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@esme.apache.org by Richard Hirsch <hi...@gmail.com> on 2010/07/07 10:02:11 UTC
Problem with using metadata for API2
I'm been working on the use of metadata in api2 and have got it
working but the problem is that 1) all "s are removed when using JSON
(see below) or 2) all XML tags are removed when posting with XML.
I assume that this problem is linked to the following code in Message.scala
lazy val metadata: String = originalXml \ "metadata" text
lazy val metaData: String = {
val org = originalXml
(org \ "metadata").map(_.text).first
}
Anyone have any idea what the problem is?
There is also a JIRA item for thsi:
https://issues.apache.org/jira/browse/ESME-242
D.
------------------------
curl -b headers -d "message=test70&metadata=<metadata>"
meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince"
,"songtitle":"Never+Let+Me+Down"}}]</metadata>" http://localhost:8080/api2/user/
messages
<?xml version="1.0" encoding="UTF-8"?>
<api><message>
<id>47</id>
<date>Wed, 7 Jul 2010 07:57:03 UTC</date>
<source>api2</source>
<body>test70</body>
<metadata>meta:[{place:{place_type:city,region:CA }},{song:{artist:Prince,song
title:Never Let Me Down}}]</metadata>
<author><nickname>bob1</nickname><id>1</id></author>
<tags></tags><replyto></replyto><conversation></conversation>
</message></api>
Re: Problem with using metadata for API2
Posted by Vassil Dichev <vd...@apache.org>.
> But what happens when the user wants to have an xml stream in the
> metadata? For example when a bot is using the ESME api and wants to
> store additional material in the message? I was trying to duplicate
> the Twitter Annotations behavior:
> http://apiwiki.twitter.com/Annotations-Overview
Currently when the client gets the metadata as a string it should be
unescaped, which should give the expected result parsable both in the
JSON or XML case.
The correct thing for a client to do is to also URL encode the request
when posting, which is also a requirement for the Twitter API, and it
will not have to deal with shell escaping issues such as when you are
using curl from the command-line.
Re: Problem with using metadata for API2
Posted by Richard Hirsch <hi...@gmail.com>.
But what happens when the user wants to have an xml stream in the
metadata? For example when a bot is using the ESME api and wants to
store additional material in the message? I was trying to duplicate
the Twitter Annotations behavior:
http://apiwiki.twitter.com/Annotations-Overview
On Sun, Jul 11, 2010 at 2:07 PM, Vassil Dichev <vd...@apache.org> wrote:
> Dick, sorry about the delay, I had no Internet access last week.
No problems
>
> The problem with JSON you're experiencing is because the shell removes
> quotation marks, as they are used to quote the message data so that
> special characters like "&" are not interpreted by the shell. You can
> try using single quotation marks around the posted string:
>
> curl -b headers -d
> 'message=test70&metadata=<metadata>"meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}]</metadata>'
> http://localhost:8080/api2/user/messages
>
> This does work, but of course everything inside the metadata is
> escaped according to XML rules, so the response will be something
> like:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <api><message>
> <id>16</id>
> <date>Sun, 11 Jul 2010 11:15:19 UTC</date>
> <source>api2</source>
> <body>test70</body>
> <metadata>"meta":[{"place":{"place_type":"city","region":"CA
> "}},{"song":{"artist":"Prince","songtitle":"Never
> Let Me Down"}}]</metadata>
> <author><nickname>vdichev</nickname><id>1</id></author>
> <tags></tags><replyto></replyto><conversation></conversation>
> </message></api>
>
>
> You are correct that currently only the text elements are extracted
> from the metadata XML. I think this is the correct behaviour,
> otherwise one could mess with the internal XML message representation
> by putting in some of the used elements like <body>, <tags>, another
> <metadata>, etc. Probably a workaround could be to somehow escape the
> XML contained within the metadata. The problem is that apart from
> escaping XML and quoting the shell, the string must also be
> URL-encoded! So something like <a/> would probably turn to
> "<a/>" and then urlencoded would become "%26lt;a/%26gt;".
>
> Cheers,
> Vassil
>
>
> On Wed, Jul 7, 2010 at 11:02 AM, Richard Hirsch <hi...@gmail.com> wrote:
>> I'm been working on the use of metadata in api2 and have got it
>> working but the problem is that 1) all "s are removed when using JSON
>> (see below) or 2) all XML tags are removed when posting with XML.
>>
>> I assume that this problem is linked to the following code in Message.scala
>>
>> lazy val metadata: String = originalXml \ "metadata" text
>>
>> lazy val metaData: String = {
>> val org = originalXml
>>
>> (org \ "metadata").map(_.text).first
>> }
>>
>> Anyone have any idea what the problem is?
>>
>> There is also a JIRA item for thsi:
>> https://issues.apache.org/jira/browse/ESME-242
>>
>> D.
>>
>>
>> ------------------------
>>
>> curl -b headers -d "message=test70&metadata=<metadata>"
>> meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince"
>> ,"songtitle":"Never+Let+Me+Down"}}]</metadata>" http://localhost:8080/api2/user/
>> messages
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <api><message>
>> <id>47</id>
>> <date>Wed, 7 Jul 2010 07:57:03 UTC</date>
>> <source>api2</source>
>> <body>test70</body>
>> <metadata>meta:[{place:{place_type:city,region:CA }},{song:{artist:Prince,song
>> title:Never Let Me Down}}]</metadata>
>> <author><nickname>bob1</nickname><id>1</id></author>
>> <tags></tags><replyto></replyto><conversation></conversation>
>> </message></api>
>>
>
Re: Problem with using metadata for API2
Posted by Vassil Dichev <vd...@apache.org>.
Dick, sorry about the delay, I had no Internet access last week.
The problem with JSON you're experiencing is because the shell removes
quotation marks, as they are used to quote the message data so that
special characters like "&" are not interpreted by the shell. You can
try using single quotation marks around the posted string:
curl -b headers -d
'message=test70&metadata=<metadata>"meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince","songtitle":"Never+Let+Me+Down"}}]</metadata>'
http://localhost:8080/api2/user/messages
This does work, but of course everything inside the metadata is
escaped according to XML rules, so the response will be something
like:
<?xml version="1.0" encoding="UTF-8"?>
<api><message>
<id>16</id>
<date>Sun, 11 Jul 2010 11:15:19 UTC</date>
<source>api2</source>
<body>test70</body>
<metadata>"meta":[{"place":{"place_type":"city","region":"CA
"}},{"song":{"artist":"Prince","songtitle":"Never
Let Me Down"}}]</metadata>
<author><nickname>vdichev</nickname><id>1</id></author>
<tags></tags><replyto></replyto><conversation></conversation>
</message></api>
You are correct that currently only the text elements are extracted
from the metadata XML. I think this is the correct behaviour,
otherwise one could mess with the internal XML message representation
by putting in some of the used elements like <body>, <tags>, another
<metadata>, etc. Probably a workaround could be to somehow escape the
XML contained within the metadata. The problem is that apart from
escaping XML and quoting the shell, the string must also be
URL-encoded! So something like <a/> would probably turn to
"<a/>" and then urlencoded would become "%26lt;a/%26gt;".
Cheers,
Vassil
On Wed, Jul 7, 2010 at 11:02 AM, Richard Hirsch <hi...@gmail.com> wrote:
> I'm been working on the use of metadata in api2 and have got it
> working but the problem is that 1) all "s are removed when using JSON
> (see below) or 2) all XML tags are removed when posting with XML.
>
> I assume that this problem is linked to the following code in Message.scala
>
> lazy val metadata: String = originalXml \ "metadata" text
>
> lazy val metaData: String = {
> val org = originalXml
>
> (org \ "metadata").map(_.text).first
> }
>
> Anyone have any idea what the problem is?
>
> There is also a JIRA item for thsi:
> https://issues.apache.org/jira/browse/ESME-242
>
> D.
>
>
> ------------------------
>
> curl -b headers -d "message=test70&metadata=<metadata>"
> meta":[{"place":{"place_type":"city","region":"CA+"}},{"song":{"artist":"Prince"
> ,"songtitle":"Never+Let+Me+Down"}}]</metadata>" http://localhost:8080/api2/user/
> messages
>
> <?xml version="1.0" encoding="UTF-8"?>
> <api><message>
> <id>47</id>
> <date>Wed, 7 Jul 2010 07:57:03 UTC</date>
> <source>api2</source>
> <body>test70</body>
> <metadata>meta:[{place:{place_type:city,region:CA }},{song:{artist:Prince,song
> title:Never Let Me Down}}]</metadata>
> <author><nickname>bob1</nickname><id>1</id></author>
> <tags></tags><replyto></replyto><conversation></conversation>
> </message></api>
>