You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Tolga Tarhan <to...@netbrains.com> on 2011/07/02 02:44:19 UTC
Re: URI Escaping in HTTP (and other Producers)
All,
I hate to reply to my own post, but I'm shocked that nobody else sees
this as an issue. Sending an HTTP request is obviously a common use
case. As soon as you leave the world of unit-tests and enter
production-code, URL encoding is inevitably a requirement. Am I really
misunderstanding something here? I'm willing to help implement the
changes required to support my use-case, but I need some validation
that it really doesn't exist today before I go duplicating work.
Thanks,
Tolga
On Wed, May 25, 2011 at 8:29 PM, Tolga Tarhan <to...@netbrains.com> wrote:
>
> All,
>
> I've scoured the internet looking for an answer to something that I
> believe should be very simple with Camel: I want to take several
> headers and compose them into an HTTP query string in a safe way. The
> only examples I've found either use constant(), which isn't useful for
> building dynamic query strings, or they use simple() which doesn't
> offer URL escaping.
>
> For example, take the following snippet right from HTTP component's
> documentation:
>
> from("direct:start")
> .setHeader(Exchange.HTTP_QUERY, constant("order=123&detail=short"))
> .to("http://oldhost");
>
> This is 90% of the way there, but what if you don't always want order
> id 123? We'd like to be able to substitute a header value here. So,
> the next logical version of this is to switch to simple:
>
> from("direct:start")
> .setHeader(Exchange.HTTP_QUERY,
> simple("order=${header.orderId}&detail=short"))
> .to("http://oldhost");
>
> But this has the major issue of not being URL encoded. This means that
> a space (or any reserved character) in header.orderId results in an
> exception thrown by the HTTP component for an invalid query string.
>
> So the only way that's left is to use JavaScript, which is very
> verbose for something like this, or to write a custom processor. It
> seems like this should be something that's built-in, so I'm asking
> here to see if I'm missing an obvious/normal way to do what I'm
> looking for here?
>
> Thanks for the help,
> Tolga
Re: URI Escaping in HTTP (and other Producers)
Posted by Willem Jiang <wi...@gmail.com>.
On 7/7/11 9:06 AM, Tolga Tarhan wrote:
> It seems like something like the Map-based feature that the CXFRS component
> provides makes a lot of sense here -- that is, we take a Map<String,Object>
> and make a query string out of it automatically. It could support GET or
> POST (in the case of POST, the URL-encoded string becomes the POST body,
> in application/x-www-form-urlencoded format).
>
> If it makes more sense to build this as a processor which runs in the
> pipeline before the HTTP producer, that's also an option. The processor
> would take the Map and do the appropriate encoding, and then either set the
> resulting string in the Exchange.HTTPQuery header (for GET), or as the
> message body (for POST).
Good suggestion, it could be more easy for use to generate a right
Exchange.HTTPQuery header with a right URL encoding.
>
> Would such a processor be considered for inclusion as part of the
> distribution? I'll provide the code. The only thing I haven't looked into is
> how to extend the DSL and/or XML to support it in an fluent way. Any hints?
>
It could be some build in method in the camel-http component, as it is
not wildly used across other components, user can do it by applying the
header in the processor or customer bean method.
The DSL just come into my mind is setHeader() with some expression to
take the value form the query parameter map.
> Thanks,
> Tolga
>
> On Mon, Jul 4, 2011 at 1:23 AM, Willem Jiang<wi...@gmail.com> wrote:
>
>> It's hard to tell the query string by calling the URLEncoder in side of
>> camel http producer, as there are some character means something to the URI
>> like "=","?","&".
>> Ben just showed us a way to build up the Query string by encoding the
>> parameter, but the header should be set like this
>>
>> exchange.getIn().setHeader(**Exchange.HTTPQuery,
>> "order="+encoded+"&detail=**short");
>>
>>
>>
>> On 7/2/11 12:06 PM, boday wrote:
>>
>>> I can't find a DSL API to do this either. And yes, the value in the
>>> HTTP_QUERY header is assumed to be encoded by design (so it doesn't double
>>> encode, etc).
>>>
>>> Log a JIRA (and submit a patch if you'd like)...and I'll take a look.
>>>
>>> For now, just throw in a simple processor and be done with it...
>>>
>>> .process( new Processor() {
>>> public void process(Exchange exchange) throws Exception {
>>> String encoded =
>>> URLEncoder.encode((String)**exchange.getIn().getHeader("**orderId"),
>>> "UTF-8"));
>>> exchange.getIn().setHeader(**Exchange.HTTP_QUERY,encoded);
>>> }
>>> })
>>>
>>>
>>> TTar wrote:
>>>
>>>>
>>>> All,
>>>>
>>>> I hate to reply to my own post, but I'm shocked that nobody else sees
>>>> this as an issue. Sending an HTTP request is obviously a common use
>>>> case. As soon as you leave the world of unit-tests and enter
>>>> production-code, URL encoding is inevitably a requirement. Am I really
>>>> misunderstanding something here? I'm willing to help implement the
>>>> changes required to support my use-case, but I need some validation
>>>> that it really doesn't exist today before I go duplicating work.
>>>>
>>>> Thanks,
>>>> Tolga
>>>>
>>>> On Wed, May 25, 2011 at 8:29 PM, Tolga Tarhan<tolga@netbrains.com&**
>>>> gt;
>>>> wrote:
>>>>
>>>>>
>>>>> All,
>>>>>
>>>>> I've scoured the internet looking for an answer to something that I
>>>>> believe should be very simple with Camel: I want to take several
>>>>> headers and compose them into an HTTP query string in a safe way. The
>>>>> only examples I've found either use constant(), which isn't useful for
>>>>> building dynamic query strings, or they use simple() which doesn't
>>>>> offer URL escaping.
>>>>>
>>>>> For example, take the following snippet right from HTTP component's
>>>>> documentation:
>>>>>
>>>>> from("direct:start")
>>>>> .setHeader(Exchange.HTTP_**QUERY, constant("order=123&detail=**
>>>>> short"))
>>>>> .to("http://oldhost");
>>>>>
>>>>> This is 90% of the way there, but what if you don't always want order
>>>>> id 123? We'd like to be able to substitute a header value here. So,
>>>>> the next logical version of this is to switch to simple:
>>>>>
>>>>> from("direct:start")
>>>>> .setHeader(Exchange.HTTP_**QUERY,
>>>>> simple("order=${header.**orderId}&detail=short"))
>>>>> .to("http://oldhost");
>>>>>
>>>>> But this has the major issue of not being URL encoded. This means that
>>>>> a space (or any reserved character) in header.orderId results in an
>>>>> exception thrown by the HTTP component for an invalid query string.
>>>>>
>>>>> So the only way that's left is to use JavaScript, which is very
>>>>> verbose for something like this, or to write a custom processor. It
>>>>> seems like this should be something that's built-in, so I'm asking
>>>>> here to see if I'm missing an obvious/normal way to do what I'm
>>>>> looking for here?
>>>>>
>>>>> Thanks for the help,
>>>>> Tolga
>>>>>
>>>>
>>>>
>>>
>>> -----
>>> Ben O'Day
>>> IT Consultant -http://consulting-notes.com
>>>
>>> --
>>> View this message in context: http://camel.465427.n5.nabble.**
>>> com/URI-Escaping-in-HTTP-and-**other-Producers-**tp4427457p4543958.html<http://camel.465427.n5.nabble.com/URI-Escaping-in-HTTP-and-other-Producers-tp4427457p4543958.html>
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>> --
>> Willem
>> ------------------------------**----
>> FuseSource
>> Web: http://www.fusesource.com
>> Blog: http://willemjiang.blogspot.**com<http://willemjiang.blogspot.com>(English)
>> http://jnn.javaeye.com (Chinese)
>> Twitter: willemjiang
>> Weibo: willemjiang
>>
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Weibo: willemjiang
Re: URI Escaping in HTTP (and other Producers)
Posted by Tolga Tarhan <to...@netbrains.com>.
It seems like something like the Map-based feature that the CXFRS component
provides makes a lot of sense here -- that is, we take a Map<String,Object>
and make a query string out of it automatically. It could support GET or
POST (in the case of POST, the URL-encoded string becomes the POST body,
in application/x-www-form-urlencoded format).
If it makes more sense to build this as a processor which runs in the
pipeline before the HTTP producer, that's also an option. The processor
would take the Map and do the appropriate encoding, and then either set the
resulting string in the Exchange.HTTPQuery header (for GET), or as the
message body (for POST).
Would such a processor be considered for inclusion as part of the
distribution? I'll provide the code. The only thing I haven't looked into is
how to extend the DSL and/or XML to support it in an fluent way. Any hints?
Thanks,
Tolga
On Mon, Jul 4, 2011 at 1:23 AM, Willem Jiang <wi...@gmail.com> wrote:
> It's hard to tell the query string by calling the URLEncoder in side of
> camel http producer, as there are some character means something to the URI
> like "=","?","&".
> Ben just showed us a way to build up the Query string by encoding the
> parameter, but the header should be set like this
>
> exchange.getIn().setHeader(**Exchange.HTTPQuery,
> "order="+encoded+"&detail=**short");
>
>
>
> On 7/2/11 12:06 PM, boday wrote:
>
>> I can't find a DSL API to do this either. And yes, the value in the
>> HTTP_QUERY header is assumed to be encoded by design (so it doesn't double
>> encode, etc).
>>
>> Log a JIRA (and submit a patch if you'd like)...and I'll take a look.
>>
>> For now, just throw in a simple processor and be done with it...
>>
>> .process( new Processor() {
>> public void process(Exchange exchange) throws Exception {
>> String encoded =
>> URLEncoder.encode((String)**exchange.getIn().getHeader("**orderId"),
>> "UTF-8"));
>> exchange.getIn().setHeader(**Exchange.HTTP_QUERY,encoded);
>> }
>> })
>>
>>
>> TTar wrote:
>>
>>>
>>> All,
>>>
>>> I hate to reply to my own post, but I'm shocked that nobody else sees
>>> this as an issue. Sending an HTTP request is obviously a common use
>>> case. As soon as you leave the world of unit-tests and enter
>>> production-code, URL encoding is inevitably a requirement. Am I really
>>> misunderstanding something here? I'm willing to help implement the
>>> changes required to support my use-case, but I need some validation
>>> that it really doesn't exist today before I go duplicating work.
>>>
>>> Thanks,
>>> Tolga
>>>
>>> On Wed, May 25, 2011 at 8:29 PM, Tolga Tarhan<tolga@netbrains.com&**
>>> gt;
>>> wrote:
>>>
>>>>
>>>> All,
>>>>
>>>> I've scoured the internet looking for an answer to something that I
>>>> believe should be very simple with Camel: I want to take several
>>>> headers and compose them into an HTTP query string in a safe way. The
>>>> only examples I've found either use constant(), which isn't useful for
>>>> building dynamic query strings, or they use simple() which doesn't
>>>> offer URL escaping.
>>>>
>>>> For example, take the following snippet right from HTTP component's
>>>> documentation:
>>>>
>>>> from("direct:start")
>>>> .setHeader(Exchange.HTTP_**QUERY, constant("order=123&detail=**
>>>> short"))
>>>> .to("http://oldhost");
>>>>
>>>> This is 90% of the way there, but what if you don't always want order
>>>> id 123? We'd like to be able to substitute a header value here. So,
>>>> the next logical version of this is to switch to simple:
>>>>
>>>> from("direct:start")
>>>> .setHeader(Exchange.HTTP_**QUERY,
>>>> simple("order=${header.**orderId}&detail=short"))
>>>> .to("http://oldhost");
>>>>
>>>> But this has the major issue of not being URL encoded. This means that
>>>> a space (or any reserved character) in header.orderId results in an
>>>> exception thrown by the HTTP component for an invalid query string.
>>>>
>>>> So the only way that's left is to use JavaScript, which is very
>>>> verbose for something like this, or to write a custom processor. It
>>>> seems like this should be something that's built-in, so I'm asking
>>>> here to see if I'm missing an obvious/normal way to do what I'm
>>>> looking for here?
>>>>
>>>> Thanks for the help,
>>>> Tolga
>>>>
>>>
>>>
>>
>> -----
>> Ben O'Day
>> IT Consultant -http://consulting-notes.com
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.**
>> com/URI-Escaping-in-HTTP-and-**other-Producers-**tp4427457p4543958.html<http://camel.465427.n5.nabble.com/URI-Escaping-in-HTTP-and-other-Producers-tp4427457p4543958.html>
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
> --
> Willem
> ------------------------------**----
> FuseSource
> Web: http://www.fusesource.com
> Blog: http://willemjiang.blogspot.**com<http://willemjiang.blogspot.com>(English)
> http://jnn.javaeye.com (Chinese)
> Twitter: willemjiang
> Weibo: willemjiang
>
Re: URI Escaping in HTTP (and other Producers)
Posted by Willem Jiang <wi...@gmail.com>.
It's hard to tell the query string by calling the URLEncoder in side of
camel http producer, as there are some character means something to the
URI like "=","?","&".
Ben just showed us a way to build up the Query string by encoding the
parameter, but the header should be set like this
exchange.getIn().setHeader(Exchange.HTTPQuery,
"order="+encoded+"&detail=short");
On 7/2/11 12:06 PM, boday wrote:
> I can't find a DSL API to do this either. And yes, the value in the
> HTTP_QUERY header is assumed to be encoded by design (so it doesn't double
> encode, etc).
>
> Log a JIRA (and submit a patch if you'd like)...and I'll take a look.
>
> For now, just throw in a simple processor and be done with it...
>
> .process( new Processor() {
> public void process(Exchange exchange) throws Exception {
> String encoded =
> URLEncoder.encode((String)exchange.getIn().getHeader("orderId"), "UTF-8"));
> exchange.getIn().setHeader(Exchange.HTTP_QUERY,encoded);
> }
> })
>
>
> TTar wrote:
>>
>> All,
>>
>> I hate to reply to my own post, but I'm shocked that nobody else sees
>> this as an issue. Sending an HTTP request is obviously a common use
>> case. As soon as you leave the world of unit-tests and enter
>> production-code, URL encoding is inevitably a requirement. Am I really
>> misunderstanding something here? I'm willing to help implement the
>> changes required to support my use-case, but I need some validation
>> that it really doesn't exist today before I go duplicating work.
>>
>> Thanks,
>> Tolga
>>
>> On Wed, May 25, 2011 at 8:29 PM, Tolga Tarhan<tolga@netbrains.com>
>> wrote:
>>>
>>> All,
>>>
>>> I've scoured the internet looking for an answer to something that I
>>> believe should be very simple with Camel: I want to take several
>>> headers and compose them into an HTTP query string in a safe way. The
>>> only examples I've found either use constant(), which isn't useful for
>>> building dynamic query strings, or they use simple() which doesn't
>>> offer URL escaping.
>>>
>>> For example, take the following snippet right from HTTP component's
>>> documentation:
>>>
>>> from("direct:start")
>>> .setHeader(Exchange.HTTP_QUERY, constant("order=123&detail=short"))
>>> .to("http://oldhost");
>>>
>>> This is 90% of the way there, but what if you don't always want order
>>> id 123? We'd like to be able to substitute a header value here. So,
>>> the next logical version of this is to switch to simple:
>>>
>>> from("direct:start")
>>> .setHeader(Exchange.HTTP_QUERY,
>>> simple("order=${header.orderId}&detail=short"))
>>> .to("http://oldhost");
>>>
>>> But this has the major issue of not being URL encoded. This means that
>>> a space (or any reserved character) in header.orderId results in an
>>> exception thrown by the HTTP component for an invalid query string.
>>>
>>> So the only way that's left is to use JavaScript, which is very
>>> verbose for something like this, or to write a custom processor. It
>>> seems like this should be something that's built-in, so I'm asking
>>> here to see if I'm missing an obvious/normal way to do what I'm
>>> looking for here?
>>>
>>> Thanks for the help,
>>> Tolga
>>
>
>
> -----
> Ben O'Day
> IT Consultant -http://consulting-notes.com
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/URI-Escaping-in-HTTP-and-other-Producers-tp4427457p4543958.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Weibo: willemjiang
Re: URI Escaping in HTTP (and other Producers)
Posted by boday <be...@initekconsulting.com>.
I can't find a DSL API to do this either. And yes, the value in the
HTTP_QUERY header is assumed to be encoded by design (so it doesn't double
encode, etc).
Log a JIRA (and submit a patch if you'd like)...and I'll take a look.
For now, just throw in a simple processor and be done with it...
.process( new Processor() {
public void process(Exchange exchange) throws Exception {
String encoded =
URLEncoder.encode((String)exchange.getIn().getHeader("orderId"), "UTF-8"));
exchange.getIn().setHeader(Exchange.HTTP_QUERY,encoded);
}
})
TTar wrote:
>
> All,
>
> I hate to reply to my own post, but I'm shocked that nobody else sees
> this as an issue. Sending an HTTP request is obviously a common use
> case. As soon as you leave the world of unit-tests and enter
> production-code, URL encoding is inevitably a requirement. Am I really
> misunderstanding something here? I'm willing to help implement the
> changes required to support my use-case, but I need some validation
> that it really doesn't exist today before I go duplicating work.
>
> Thanks,
> Tolga
>
> On Wed, May 25, 2011 at 8:29 PM, Tolga Tarhan <tolga@netbrains.com>
> wrote:
>>
>> All,
>>
>> I've scoured the internet looking for an answer to something that I
>> believe should be very simple with Camel: I want to take several
>> headers and compose them into an HTTP query string in a safe way. The
>> only examples I've found either use constant(), which isn't useful for
>> building dynamic query strings, or they use simple() which doesn't
>> offer URL escaping.
>>
>> For example, take the following snippet right from HTTP component's
>> documentation:
>>
>> from("direct:start")
>> .setHeader(Exchange.HTTP_QUERY, constant("order=123&detail=short"))
>> .to("http://oldhost");
>>
>> This is 90% of the way there, but what if you don't always want order
>> id 123? We'd like to be able to substitute a header value here. So,
>> the next logical version of this is to switch to simple:
>>
>> from("direct:start")
>> .setHeader(Exchange.HTTP_QUERY,
>> simple("order=${header.orderId}&detail=short"))
>> .to("http://oldhost");
>>
>> But this has the major issue of not being URL encoded. This means that
>> a space (or any reserved character) in header.orderId results in an
>> exception thrown by the HTTP component for an invalid query string.
>>
>> So the only way that's left is to use JavaScript, which is very
>> verbose for something like this, or to write a custom processor. It
>> seems like this should be something that's built-in, so I'm asking
>> here to see if I'm missing an obvious/normal way to do what I'm
>> looking for here?
>>
>> Thanks for the help,
>> Tolga
>
-----
Ben O'Day
IT Consultant -http://consulting-notes.com
--
View this message in context: http://camel.465427.n5.nabble.com/URI-Escaping-in-HTTP-and-other-Producers-tp4427457p4543958.html
Sent from the Camel - Users mailing list archive at Nabble.com.