You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Claus Ibsen <cl...@gmail.com> on 2009/08/08 12:42:18 UTC
[DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Hi
For background see ticket CAMEL-1886
https://issues.apache.org/activemq/browse/CAMEL-1886
The issue is that headers is stored using keys that are case
sensitive, and that leads to a few problems and differences in Camel:
- lookup is cumbersome as you must match case 100%
- errors in camel components that is unable to realize a header
already exists however with a different (case)
- some protocols are agnostic to header cases, such as HTTP (and all
its different transports using HTTP), Mail, and whatnot?
- camel-mail stores notoriously headers in lower keys only
- etc.
And most importantly
- no one wants to send different values using a key that have
different case, such as:
Foo=cheese
FOO=cake
foo=beer
So James and I had a chat about it and to see if would could do
something about it.
The real reason that I picked up this issue is the tuning experiment
with Camel 2.x
http://camel.apache.org/camel-2x-speed-optimizations.html
Where we want to take a few short cuts and offer header lookup on the
source directly to avoid the "tiny" overhead of extracting headers
from the source
and populate the camel message. This is useful if you do message
routing that only uses headers. The camel-jms component did already
offer this but
it also had some inconsistency in this relation.
I have attempted a resolution in the tuning branch but discovered an
issue, that it does take some more grunt work to keep the lower case
index in sync etc.
So I wanted to try a different approach
CaseInsenstiveMap
===============
I have implemented a small case insenstive map so we can store headers
in 1 map instead of keeping 2 maps in sync.
I am still amazed why such a Map does not exists directly in the JDK itself.
What does this offer then
- lookup is easy as you can lookup using any case
and in the future we could also offer lookup for special keys people
spell differently such as: ContentType, content-type, content-type,
content_type and whatnot.
- put will also ensure that it replaces existing values
- and most importantly that it offers this as well when end users work
directly on the map using:
exchange.getIn().getHeaders().put("foo", "beer")
That will be able to replace existing keys if they are stored using:
Foo, FOO or what not
This is not possible today and or with a 2nd map that tracks lower
case indexes
So how to retain the original keys.
Well I implemented that as well, so when you e.g. copy the map to
another or extract from it using keySet it exposes a Set that uses the
original key cases.
So if you have stored using
exchange.getIn().setHeader("Foo", "cheese")
Then the key in the ketSet is the original key = Foo.
But you can do lookup using: foo, FOO, FoO, FOo, and whatnot :)
Using this implementation I actually find a few bugs in existing unit
tests and camel components that double add headers using different
cases.
For example camel-cxf in the new RS support.
I will later attach the code to the CAMEL-1886 ticket for people of
the community to take a look and review.
Here is a snippet from one of the test methods to give an idea
public void testLookupCaseAgnosticAddHeaderRemoveHeader() {
Map<String, Object> map = new CaseInsensitiveMap();
assertNull(map.get("foo"));
map.put("foo", "cheese");
assertEquals("cheese", map.get("foo"));
assertEquals("cheese", map.get("Foo"));
assertEquals("cheese", map.get("FOO"));
assertNull(map.get("unknown"));
map.put("bar", "beer");
assertEquals("beer", map.get("bar"));
assertEquals("beer", map.get("Bar"));
assertEquals("beer", map.get("BAR"));
assertNull(map.get("unknown"));
map.remove("bar");
assertNull(map.get("bar"));
assertNull(map.get("unknown"));
}
And when using the Message API
public void testRemoveWithDifferentCase() {
Message msg = new DefaultMessage();
assertNull(msg.getHeader("foo"));
msg.setHeader("foo", "cheese");
msg.setHeader("Foo", "bar");
assertEquals("bar", msg.getHeader("FOO"));
assertEquals("bar", msg.getHeader("foo"));
assertEquals("bar", msg.getHeader("Foo"));
msg.removeHeader("FOO");
assertEquals(null, msg.getHeader("foo"));
assertEquals(null, msg.getHeader("Foo"));
assertEquals(null, msg.getHeader("FOO"));
assertTrue(msg.getHeaders().isEmpty());
}
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Aug 10, 2009 at 1:59 PM, Roman
Kalukiewicz<ro...@gmail.com> wrote:
> Just few general comments on the topic here:
>
> 1) My impression is that we are trying to solve problem that doesn't
> exist. No one would expect that getting "foo" header should retrieve
> what was put under "FOO" key. If we want to have guarantee that it
> works properly people should use constants for it. It is the way that
> Java world solves those problems
One think is theory another is the world we live in.
It is in fact not only peoples one header keys but the various
protocols that affect the keys that is the problem.
If people consume emails and want to get the To header and use "To"
they will not get it as they have to use lowercase "to".
But when you send an email you can set the To using a upper case T.
I think all this should be easy and transparent for Camel end users.
Whether or not they use To/to/TO etc.
And in case of the HTTP protocols you could end up with double headers
for the same semantic entry.
ContentType and contenttype can be stored twice. For example camel-cxf
have this error already.
> 2) If we create CaseInsenstiveMap class we will fail to obey Map
> contract. Let's take a look at Map.containsKey() method's contract in
> javadoc. And my impression is that we want to treat headers as a Map,
> don't we?
They are treated as a Map but you can lookup easily using case insensitive keys.
If the JDK already had such a Map we would have used that from day zero.
>
> In protocols that has case-insensitive headers (like http) I believe
> component should take care of it by (for example) setting headers
> using lowercase keys only (for example) and it should be clearly
> stated in component's documentation, as in Camel components are
> responsible for handling protocol-specific features.
>
See above.
Will be confusing if some components changes cases of all headers.
Where as other wont.
Then on the receiver side you newer know whether or not keys are in which case.
Maybe a route only optionally invokes an external HTTP service and
when it does suddenly all keys are in lower case
when its received in the end.
> By having headers (that are Map basically) case-insensitive I believe
> we change Camel internals so it conforms to specific features of
> specific protocols. We did it already by having faults as separate
> messages, by having attachments in the Camel message - I believe those
> things were wrong and we already fixed some of them.
>
Yeah keeping it internally is a great abstraction. End users that uses
getHeaders() sees the headers just as they have been stored, using the
cases the keys have.
> Roman
>
> 2009/8/10 James Strachan <ja...@gmail.com>:
>> FWIW I'm liking the fact that lookups become case insensitive; as its
>> what folks expect in HTTP/Mail type scenarios. Using a custom Map also
>> makes it possible for a component/endpoint to disable this feature if
>> it wishes (or use a different Map implementation if it can think of a
>> good reason to do so).
>>
>> 2009/8/10 Claus Ibsen <cl...@gmail.com>:
>>> Hi
>>>
>>> Had a chat James as well.
>>>
>>> Went for the solution without a KeyValuePair object as it causes pain
>>> with dynamic languages and template engines. And the camel-jms
>>> component had some obscure errors with it also.
>>>
>>> Its very easy to revert to old behavior as its just to change to use a
>>> java.util.HashMap in DefaultMessage.
>>>
>>>
>>> On Sat, Aug 8, 2009 at 9:53 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>>> Hi Claus,
>>>>
>>>> I added a comment on the Jira as well.
>>>>
>>>> Why not use a Map<lowercase-header, Pair<header, Object>> ? I believe this
>>>> to be a simple solution. Lookups should always be on the lowercase version
>>>> of the header, on the wire the original version should go.
>>>>
>>>> What do you think?
>>>> Hadrian
>>>>
>>>> On Aug 8, 2009, at 9:42 AM, Claus Ibsen wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> I have just attached a much better implementation of the map, this
>>>>> time all code is in a single class and it does not use holder objects
>>>>> such as a KeyValuePair when storing the value.
>>>>>
>>>>> It also works out of the box with scripting engines that can access
>>>>> and manipulate the headers such as Freemarker and Velocity.
>>>>>
>>>>> The changes needed in DefaultMessage is basically to use this map
>>>>> instead of a regular HashMap.
>>>>> And when its copied to another map the original keys is automatic
>>>>> retained.
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Aug 8, 2009 at 12:42 PM, Claus Ibsen<cl...@gmail.com> wrote:
>>>>>>
>>>>>> Hi
>>>>>>
>>>>>> For background see ticket CAMEL-1886
>>>>>> https://issues.apache.org/activemq/browse/CAMEL-1886
>>>>>>
>>>>>> The issue is that headers is stored using keys that are case
>>>>>> sensitive, and that leads to a few problems and differences in Camel:
>>>>>>
>>>>>> - lookup is cumbersome as you must match case 100%
>>>>>> - errors in camel components that is unable to realize a header
>>>>>> already exists however with a different (case)
>>>>>> - some protocols are agnostic to header cases, such as HTTP (and all
>>>>>> its different transports using HTTP), Mail, and whatnot?
>>>>>> - camel-mail stores notoriously headers in lower keys only
>>>>>> - etc.
>>>>>>
>>>>>> And most importantly
>>>>>> - no one wants to send different values using a key that have
>>>>>> different case, such as:
>>>>>> Foo=cheese
>>>>>> FOO=cake
>>>>>> foo=beer
>>>>>>
>>>>>> So James and I had a chat about it and to see if would could do
>>>>>> something about it.
>>>>>>
>>>>>> The real reason that I picked up this issue is the tuning experiment
>>>>>> with Camel 2.x
>>>>>> http://camel.apache.org/camel-2x-speed-optimizations.html
>>>>>>
>>>>>> Where we want to take a few short cuts and offer header lookup on the
>>>>>> source directly to avoid the "tiny" overhead of extracting headers
>>>>>> from the source
>>>>>> and populate the camel message. This is useful if you do message
>>>>>> routing that only uses headers. The camel-jms component did already
>>>>>> offer this but
>>>>>> it also had some inconsistency in this relation.
>>>>>>
>>>>>>
>>>>>> I have attempted a resolution in the tuning branch but discovered an
>>>>>> issue, that it does take some more grunt work to keep the lower case
>>>>>> index in sync etc.
>>>>>> So I wanted to try a different approach
>>>>>>
>>>>>> CaseInsenstiveMap
>>>>>> ===============
>>>>>>
>>>>>> I have implemented a small case insenstive map so we can store headers
>>>>>> in 1 map instead of keeping 2 maps in sync.
>>>>>> I am still amazed why such a Map does not exists directly in the JDK
>>>>>> itself.
>>>>>>
>>>>>> What does this offer then
>>>>>> - lookup is easy as you can lookup using any case
>>>>>> and in the future we could also offer lookup for special keys people
>>>>>> spell differently such as: ContentType, content-type, content-type,
>>>>>> content_type and whatnot.
>>>>>> - put will also ensure that it replaces existing values
>>>>>> - and most importantly that it offers this as well when end users work
>>>>>> directly on the map using:
>>>>>> exchange.getIn().getHeaders().put("foo", "beer")
>>>>>> That will be able to replace existing keys if they are stored using:
>>>>>> Foo, FOO or what not
>>>>>> This is not possible today and or with a 2nd map that tracks lower
>>>>>> case indexes
>>>>>>
>>>>>> So how to retain the original keys.
>>>>>>
>>>>>> Well I implemented that as well, so when you e.g. copy the map to
>>>>>> another or extract from it using keySet it exposes a Set that uses the
>>>>>> original key cases.
>>>>>> So if you have stored using
>>>>>> exchange.getIn().setHeader("Foo", "cheese")
>>>>>>
>>>>>> Then the key in the ketSet is the original key = Foo.
>>>>>> But you can do lookup using: foo, FOO, FoO, FOo, and whatnot :)
>>>>>>
>>>>>> Using this implementation I actually find a few bugs in existing unit
>>>>>> tests and camel components that double add headers using different
>>>>>> cases.
>>>>>> For example camel-cxf in the new RS support.
>>>>>>
>>>>>>
>>>>>> I will later attach the code to the CAMEL-1886 ticket for people of
>>>>>> the community to take a look and review.
>>>>>>
>>>>>>
>>>>>> Here is a snippet from one of the test methods to give an idea
>>>>>>
>>>>>> public void testLookupCaseAgnosticAddHeaderRemoveHeader() {
>>>>>> Map<String, Object> map = new CaseInsensitiveMap();
>>>>>> assertNull(map.get("foo"));
>>>>>>
>>>>>> map.put("foo", "cheese");
>>>>>>
>>>>>> assertEquals("cheese", map.get("foo"));
>>>>>> assertEquals("cheese", map.get("Foo"));
>>>>>> assertEquals("cheese", map.get("FOO"));
>>>>>> assertNull(map.get("unknown"));
>>>>>>
>>>>>> map.put("bar", "beer");
>>>>>>
>>>>>> assertEquals("beer", map.get("bar"));
>>>>>> assertEquals("beer", map.get("Bar"));
>>>>>> assertEquals("beer", map.get("BAR"));
>>>>>> assertNull(map.get("unknown"));
>>>>>>
>>>>>> map.remove("bar");
>>>>>> assertNull(map.get("bar"));
>>>>>> assertNull(map.get("unknown"));
>>>>>> }
>>>>>>
>>>>>>
>>>>>> And when using the Message API
>>>>>>
>>>>>> public void testRemoveWithDifferentCase() {
>>>>>> Message msg = new DefaultMessage();
>>>>>> assertNull(msg.getHeader("foo"));
>>>>>>
>>>>>> msg.setHeader("foo", "cheese");
>>>>>> msg.setHeader("Foo", "bar");
>>>>>>
>>>>>> assertEquals("bar", msg.getHeader("FOO"));
>>>>>> assertEquals("bar", msg.getHeader("foo"));
>>>>>> assertEquals("bar", msg.getHeader("Foo"));
>>>>>>
>>>>>> msg.removeHeader("FOO");
>>>>>>
>>>>>> assertEquals(null, msg.getHeader("foo"));
>>>>>> assertEquals(null, msg.getHeader("Foo"));
>>>>>> assertEquals(null, msg.getHeader("FOO"));
>>>>>>
>>>>>> assertTrue(msg.getHeaders().isEmpty());
>>>>>> }
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by Roman Kalukiewicz <ro...@gmail.com>.
2009/8/10 Claus Ibsen <cl...@gmail.com>:
> Its definitely not strange. In fact the old behavior is more strange,
> as you end up having duplicate headers by mistake.
>
> Imagine integrating to a remote system you have no control over and if
> you are lucky using JMS. Then you have been told that they will add a
> JMS property with the key: CustomerId that is very important for you
> to hold of.
>
> Now for some reason its a mainframe system with all uppercased so they
> send it as CUSTOMERID. And you application is now toasted.
CustomerId is different than CUSTOMERID as it is different than
Customer_Id as it is different than Customer-Id. Properties in JMS are
case sensitive, so if you want to retrieve a property you have to know
its case-sensitive name so this argument doesn't convince me at all..
We just have to agree that we don't allow headers that differ only in
case. If it is OK for us/clients (for some protocol it is perfectly
fine, for others it is not, but we don't expect anyone to use such
headers in practice). It is a limitation - I also guess it is a
limitation we can live with.
To summarize - I'm convinced ;)
There will be some behaviors that are strange like
msg.getHeaders().keySet().contains("FOO") == false
and at the same time
msg.getHeader("FOO") != null
msg.getHeaders().get("FOO") != null
but I believe if we make the Map implementation good enough we can
fulfill Map contract anyway. It will be just.. Map with not typical
behavior.
Roman
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Aug 10, 2009 at 2:51 PM, Roman
Kalukiewicz<ro...@gmail.com> wrote:
> 2009/8/10 Claus Ibsen <cl...@gmail.com>:
>> On Mon, Aug 10, 2009 at 2:02 PM, James Strachan<ja...@gmail.com> wrote:
>>> 2009/8/10 Roman Kalukiewicz <ro...@gmail.com>:
>>>> Just few general comments on the topic here:
>>>>
>>>> 1) My impression is that we are trying to solve problem that doesn't
>>>> exist. No one would expect that getting "foo" header should retrieve
>>>> what was put under "FOO" key. If we want to have guarantee that it
>>>> works properly people should use constants for it. It is the way that
>>>> Java world solves those problems
>>>> 2) If we create CaseInsenstiveMap class we will fail to obey Map
>>>> contract. Let's take a look at Map.containsKey() method's contract in
>>>> javadoc. And my impression is that we want to treat headers as a Map,
>>>> don't we?
>>>>
>>>> In protocols that has case-insensitive headers (like http) I believe
>>>> component should take care of it by (for example) setting headers
>>>> using lowercase keys only (for example) and it should be clearly
>>>> stated in component's documentation, as in Camel components are
>>>> responsible for handling protocol-specific features.
>>>
>>> I think camel should preserve what its given by default though. e.g.
>>> passing requests from HTTP or email into Camel shouldn't rename
>>> headers IMHO
>>>
>>
>> And this is exactly what it does now. If a header is stored as "Foo"
>> then it stays that way.
>> However you can also look it up using "foo".
>
> OK - sounds good. What I don't like is that headers would be strange
> Map implementation.
Its definitely not strange. In fact the old behavior is more strange,
as you end up having duplicate headers by mistake.
Imagine integrating to a remote system you have no control over and if
you are lucky using JMS. Then you have been told that they will add a
JMS property with the key: CustomerId that is very important for you
to hold of.
Now for some reason its a mainframe system with all uppercased so they
send it as CUSTOMERID. And you application is now toasted.
> What I would prefer is special Map view on the standard headers, where
> keys are lower-cased. And it should be read-only.
> If it is not read-only we violate Map contract and strange things
> start to happen like
>
Strange things? There is only 1 key for a given entity so that cant be
strange. Its stored as the case it was given, eg Foo.
So if you use matching keys all the time (eg Foo always) no strange
thing happens.
Lookup for Camel end users should be dead easy, so using the existing
getHeader(name) should not require them
to obtain some read-only view and having to get it from this view.
> map.put(foo, value1)
> map.size() == 1
> map.containsKey(foo) == true
> map.containsKey(FOO) == true // I guess it will return true
> map.keys().contains(FOO) == false // as keys() are lowercase I guess.
> Or maybe special case-insensitive Set?
> map.put(FOO, value2)
> map.size() == 1
>
I have just committed a test that using the new map that demos that
using the map it would work as expected with no surprises.
> The thing that might be discussed is if header() in DSL should use
> case sensitive or insensitive map.
> Another thing to discuss is if we would like to add
> getCaseInsensitiveHeader(s)() method to the message that would return
> such map. But once again - having read-write headers not conforming to
> standard Map contract is a bad idea I think.
>
Would you rather have that we invented our own IgnoreCaseString type
as key, maybe naming it HeaderKey
Map<HeaderKey, Object> headers = exchange.getIn().getHeaders()
That would truly make lookup cumbersome as its not String based keys.
If you want the regular map back, just copy it over in a regular
HashMap and you are set
Map myOtherMap = new HashMap(exchange.getIn().getHeaders());
> Roman
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by Roman Kalukiewicz <ro...@gmail.com>.
2009/8/10 Claus Ibsen <cl...@gmail.com>:
> On Mon, Aug 10, 2009 at 2:02 PM, James Strachan<ja...@gmail.com> wrote:
>> 2009/8/10 Roman Kalukiewicz <ro...@gmail.com>:
>>> Just few general comments on the topic here:
>>>
>>> 1) My impression is that we are trying to solve problem that doesn't
>>> exist. No one would expect that getting "foo" header should retrieve
>>> what was put under "FOO" key. If we want to have guarantee that it
>>> works properly people should use constants for it. It is the way that
>>> Java world solves those problems
>>> 2) If we create CaseInsenstiveMap class we will fail to obey Map
>>> contract. Let's take a look at Map.containsKey() method's contract in
>>> javadoc. And my impression is that we want to treat headers as a Map,
>>> don't we?
>>>
>>> In protocols that has case-insensitive headers (like http) I believe
>>> component should take care of it by (for example) setting headers
>>> using lowercase keys only (for example) and it should be clearly
>>> stated in component's documentation, as in Camel components are
>>> responsible for handling protocol-specific features.
>>
>> I think camel should preserve what its given by default though. e.g.
>> passing requests from HTTP or email into Camel shouldn't rename
>> headers IMHO
>>
>
> And this is exactly what it does now. If a header is stored as "Foo"
> then it stays that way.
> However you can also look it up using "foo".
OK - sounds good. What I don't like is that headers would be strange
Map implementation.
What I would prefer is special Map view on the standard headers, where
keys are lower-cased. And it should be read-only.
If it is not read-only we violate Map contract and strange things
start to happen like
map.put(foo, value1)
map.size() == 1
map.containsKey(foo) == true
map.containsKey(FOO) == true // I guess it will return true
map.keys().contains(FOO) == false // as keys() are lowercase I guess.
Or maybe special case-insensitive Set?
map.put(FOO, value2)
map.size() == 1
The thing that might be discussed is if header() in DSL should use
case sensitive or insensitive map.
Another thing to discuss is if we would like to add
getCaseInsensitiveHeader(s)() method to the message that would return
such map. But once again - having read-write headers not conforming to
standard Map contract is a bad idea I think.
Roman
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Aug 10, 2009 at 2:02 PM, James Strachan<ja...@gmail.com> wrote:
> 2009/8/10 Roman Kalukiewicz <ro...@gmail.com>:
>> Just few general comments on the topic here:
>>
>> 1) My impression is that we are trying to solve problem that doesn't
>> exist. No one would expect that getting "foo" header should retrieve
>> what was put under "FOO" key. If we want to have guarantee that it
>> works properly people should use constants for it. It is the way that
>> Java world solves those problems
>> 2) If we create CaseInsenstiveMap class we will fail to obey Map
>> contract. Let's take a look at Map.containsKey() method's contract in
>> javadoc. And my impression is that we want to treat headers as a Map,
>> don't we?
>>
>> In protocols that has case-insensitive headers (like http) I believe
>> component should take care of it by (for example) setting headers
>> using lowercase keys only (for example) and it should be clearly
>> stated in component's documentation, as in Camel components are
>> responsible for handling protocol-specific features.
>
> I think camel should preserve what its given by default though. e.g.
> passing requests from HTTP or email into Camel shouldn't rename
> headers IMHO
>
And this is exactly what it does now. If a header is stored as "Foo"
then it stays that way.
However you can also look it up using "foo".
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by James Strachan <ja...@gmail.com>.
2009/8/10 Roman Kalukiewicz <ro...@gmail.com>:
> Just few general comments on the topic here:
>
> 1) My impression is that we are trying to solve problem that doesn't
> exist. No one would expect that getting "foo" header should retrieve
> what was put under "FOO" key. If we want to have guarantee that it
> works properly people should use constants for it. It is the way that
> Java world solves those problems
> 2) If we create CaseInsenstiveMap class we will fail to obey Map
> contract. Let's take a look at Map.containsKey() method's contract in
> javadoc. And my impression is that we want to treat headers as a Map,
> don't we?
>
> In protocols that has case-insensitive headers (like http) I believe
> component should take care of it by (for example) setting headers
> using lowercase keys only (for example) and it should be clearly
> stated in component's documentation, as in Camel components are
> responsible for handling protocol-specific features.
I think camel should preserve what its given by default though. e.g.
passing requests from HTTP or email into Camel shouldn't rename
headers IMHO
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://fusesource.com/
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by Roman Kalukiewicz <ro...@gmail.com>.
Just few general comments on the topic here:
1) My impression is that we are trying to solve problem that doesn't
exist. No one would expect that getting "foo" header should retrieve
what was put under "FOO" key. If we want to have guarantee that it
works properly people should use constants for it. It is the way that
Java world solves those problems
2) If we create CaseInsenstiveMap class we will fail to obey Map
contract. Let's take a look at Map.containsKey() method's contract in
javadoc. And my impression is that we want to treat headers as a Map,
don't we?
In protocols that has case-insensitive headers (like http) I believe
component should take care of it by (for example) setting headers
using lowercase keys only (for example) and it should be clearly
stated in component's documentation, as in Camel components are
responsible for handling protocol-specific features.
By having headers (that are Map basically) case-insensitive I believe
we change Camel internals so it conforms to specific features of
specific protocols. We did it already by having faults as separate
messages, by having attachments in the Camel message - I believe those
things were wrong and we already fixed some of them.
Roman
2009/8/10 James Strachan <ja...@gmail.com>:
> FWIW I'm liking the fact that lookups become case insensitive; as its
> what folks expect in HTTP/Mail type scenarios. Using a custom Map also
> makes it possible for a component/endpoint to disable this feature if
> it wishes (or use a different Map implementation if it can think of a
> good reason to do so).
>
> 2009/8/10 Claus Ibsen <cl...@gmail.com>:
>> Hi
>>
>> Had a chat James as well.
>>
>> Went for the solution without a KeyValuePair object as it causes pain
>> with dynamic languages and template engines. And the camel-jms
>> component had some obscure errors with it also.
>>
>> Its very easy to revert to old behavior as its just to change to use a
>> java.util.HashMap in DefaultMessage.
>>
>>
>> On Sat, Aug 8, 2009 at 9:53 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
>>> Hi Claus,
>>>
>>> I added a comment on the Jira as well.
>>>
>>> Why not use a Map<lowercase-header, Pair<header, Object>> ? I believe this
>>> to be a simple solution. Lookups should always be on the lowercase version
>>> of the header, on the wire the original version should go.
>>>
>>> What do you think?
>>> Hadrian
>>>
>>> On Aug 8, 2009, at 9:42 AM, Claus Ibsen wrote:
>>>
>>>> Hi
>>>>
>>>> I have just attached a much better implementation of the map, this
>>>> time all code is in a single class and it does not use holder objects
>>>> such as a KeyValuePair when storing the value.
>>>>
>>>> It also works out of the box with scripting engines that can access
>>>> and manipulate the headers such as Freemarker and Velocity.
>>>>
>>>> The changes needed in DefaultMessage is basically to use this map
>>>> instead of a regular HashMap.
>>>> And when its copied to another map the original keys is automatic
>>>> retained.
>>>>
>>>>
>>>>
>>>> On Sat, Aug 8, 2009 at 12:42 PM, Claus Ibsen<cl...@gmail.com> wrote:
>>>>>
>>>>> Hi
>>>>>
>>>>> For background see ticket CAMEL-1886
>>>>> https://issues.apache.org/activemq/browse/CAMEL-1886
>>>>>
>>>>> The issue is that headers is stored using keys that are case
>>>>> sensitive, and that leads to a few problems and differences in Camel:
>>>>>
>>>>> - lookup is cumbersome as you must match case 100%
>>>>> - errors in camel components that is unable to realize a header
>>>>> already exists however with a different (case)
>>>>> - some protocols are agnostic to header cases, such as HTTP (and all
>>>>> its different transports using HTTP), Mail, and whatnot?
>>>>> - camel-mail stores notoriously headers in lower keys only
>>>>> - etc.
>>>>>
>>>>> And most importantly
>>>>> - no one wants to send different values using a key that have
>>>>> different case, such as:
>>>>> Foo=cheese
>>>>> FOO=cake
>>>>> foo=beer
>>>>>
>>>>> So James and I had a chat about it and to see if would could do
>>>>> something about it.
>>>>>
>>>>> The real reason that I picked up this issue is the tuning experiment
>>>>> with Camel 2.x
>>>>> http://camel.apache.org/camel-2x-speed-optimizations.html
>>>>>
>>>>> Where we want to take a few short cuts and offer header lookup on the
>>>>> source directly to avoid the "tiny" overhead of extracting headers
>>>>> from the source
>>>>> and populate the camel message. This is useful if you do message
>>>>> routing that only uses headers. The camel-jms component did already
>>>>> offer this but
>>>>> it also had some inconsistency in this relation.
>>>>>
>>>>>
>>>>> I have attempted a resolution in the tuning branch but discovered an
>>>>> issue, that it does take some more grunt work to keep the lower case
>>>>> index in sync etc.
>>>>> So I wanted to try a different approach
>>>>>
>>>>> CaseInsenstiveMap
>>>>> ===============
>>>>>
>>>>> I have implemented a small case insenstive map so we can store headers
>>>>> in 1 map instead of keeping 2 maps in sync.
>>>>> I am still amazed why such a Map does not exists directly in the JDK
>>>>> itself.
>>>>>
>>>>> What does this offer then
>>>>> - lookup is easy as you can lookup using any case
>>>>> and in the future we could also offer lookup for special keys people
>>>>> spell differently such as: ContentType, content-type, content-type,
>>>>> content_type and whatnot.
>>>>> - put will also ensure that it replaces existing values
>>>>> - and most importantly that it offers this as well when end users work
>>>>> directly on the map using:
>>>>> exchange.getIn().getHeaders().put("foo", "beer")
>>>>> That will be able to replace existing keys if they are stored using:
>>>>> Foo, FOO or what not
>>>>> This is not possible today and or with a 2nd map that tracks lower
>>>>> case indexes
>>>>>
>>>>> So how to retain the original keys.
>>>>>
>>>>> Well I implemented that as well, so when you e.g. copy the map to
>>>>> another or extract from it using keySet it exposes a Set that uses the
>>>>> original key cases.
>>>>> So if you have stored using
>>>>> exchange.getIn().setHeader("Foo", "cheese")
>>>>>
>>>>> Then the key in the ketSet is the original key = Foo.
>>>>> But you can do lookup using: foo, FOO, FoO, FOo, and whatnot :)
>>>>>
>>>>> Using this implementation I actually find a few bugs in existing unit
>>>>> tests and camel components that double add headers using different
>>>>> cases.
>>>>> For example camel-cxf in the new RS support.
>>>>>
>>>>>
>>>>> I will later attach the code to the CAMEL-1886 ticket for people of
>>>>> the community to take a look and review.
>>>>>
>>>>>
>>>>> Here is a snippet from one of the test methods to give an idea
>>>>>
>>>>> public void testLookupCaseAgnosticAddHeaderRemoveHeader() {
>>>>> Map<String, Object> map = new CaseInsensitiveMap();
>>>>> assertNull(map.get("foo"));
>>>>>
>>>>> map.put("foo", "cheese");
>>>>>
>>>>> assertEquals("cheese", map.get("foo"));
>>>>> assertEquals("cheese", map.get("Foo"));
>>>>> assertEquals("cheese", map.get("FOO"));
>>>>> assertNull(map.get("unknown"));
>>>>>
>>>>> map.put("bar", "beer");
>>>>>
>>>>> assertEquals("beer", map.get("bar"));
>>>>> assertEquals("beer", map.get("Bar"));
>>>>> assertEquals("beer", map.get("BAR"));
>>>>> assertNull(map.get("unknown"));
>>>>>
>>>>> map.remove("bar");
>>>>> assertNull(map.get("bar"));
>>>>> assertNull(map.get("unknown"));
>>>>> }
>>>>>
>>>>>
>>>>> And when using the Message API
>>>>>
>>>>> public void testRemoveWithDifferentCase() {
>>>>> Message msg = new DefaultMessage();
>>>>> assertNull(msg.getHeader("foo"));
>>>>>
>>>>> msg.setHeader("foo", "cheese");
>>>>> msg.setHeader("Foo", "bar");
>>>>>
>>>>> assertEquals("bar", msg.getHeader("FOO"));
>>>>> assertEquals("bar", msg.getHeader("foo"));
>>>>> assertEquals("bar", msg.getHeader("Foo"));
>>>>>
>>>>> msg.removeHeader("FOO");
>>>>>
>>>>> assertEquals(null, msg.getHeader("foo"));
>>>>> assertEquals(null, msg.getHeader("Foo"));
>>>>> assertEquals(null, msg.getHeader("FOO"));
>>>>>
>>>>> assertTrue(msg.getHeaders().isEmpty());
>>>>> }
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by James Strachan <ja...@gmail.com>.
FWIW I'm liking the fact that lookups become case insensitive; as its
what folks expect in HTTP/Mail type scenarios. Using a custom Map also
makes it possible for a component/endpoint to disable this feature if
it wishes (or use a different Map implementation if it can think of a
good reason to do so).
2009/8/10 Claus Ibsen <cl...@gmail.com>:
> Hi
>
> Had a chat James as well.
>
> Went for the solution without a KeyValuePair object as it causes pain
> with dynamic languages and template engines. And the camel-jms
> component had some obscure errors with it also.
>
> Its very easy to revert to old behavior as its just to change to use a
> java.util.HashMap in DefaultMessage.
>
>
> On Sat, Aug 8, 2009 at 9:53 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
>> Hi Claus,
>>
>> I added a comment on the Jira as well.
>>
>> Why not use a Map<lowercase-header, Pair<header, Object>> ? I believe this
>> to be a simple solution. Lookups should always be on the lowercase version
>> of the header, on the wire the original version should go.
>>
>> What do you think?
>> Hadrian
>>
>> On Aug 8, 2009, at 9:42 AM, Claus Ibsen wrote:
>>
>>> Hi
>>>
>>> I have just attached a much better implementation of the map, this
>>> time all code is in a single class and it does not use holder objects
>>> such as a KeyValuePair when storing the value.
>>>
>>> It also works out of the box with scripting engines that can access
>>> and manipulate the headers such as Freemarker and Velocity.
>>>
>>> The changes needed in DefaultMessage is basically to use this map
>>> instead of a regular HashMap.
>>> And when its copied to another map the original keys is automatic
>>> retained.
>>>
>>>
>>>
>>> On Sat, Aug 8, 2009 at 12:42 PM, Claus Ibsen<cl...@gmail.com> wrote:
>>>>
>>>> Hi
>>>>
>>>> For background see ticket CAMEL-1886
>>>> https://issues.apache.org/activemq/browse/CAMEL-1886
>>>>
>>>> The issue is that headers is stored using keys that are case
>>>> sensitive, and that leads to a few problems and differences in Camel:
>>>>
>>>> - lookup is cumbersome as you must match case 100%
>>>> - errors in camel components that is unable to realize a header
>>>> already exists however with a different (case)
>>>> - some protocols are agnostic to header cases, such as HTTP (and all
>>>> its different transports using HTTP), Mail, and whatnot?
>>>> - camel-mail stores notoriously headers in lower keys only
>>>> - etc.
>>>>
>>>> And most importantly
>>>> - no one wants to send different values using a key that have
>>>> different case, such as:
>>>> Foo=cheese
>>>> FOO=cake
>>>> foo=beer
>>>>
>>>> So James and I had a chat about it and to see if would could do
>>>> something about it.
>>>>
>>>> The real reason that I picked up this issue is the tuning experiment
>>>> with Camel 2.x
>>>> http://camel.apache.org/camel-2x-speed-optimizations.html
>>>>
>>>> Where we want to take a few short cuts and offer header lookup on the
>>>> source directly to avoid the "tiny" overhead of extracting headers
>>>> from the source
>>>> and populate the camel message. This is useful if you do message
>>>> routing that only uses headers. The camel-jms component did already
>>>> offer this but
>>>> it also had some inconsistency in this relation.
>>>>
>>>>
>>>> I have attempted a resolution in the tuning branch but discovered an
>>>> issue, that it does take some more grunt work to keep the lower case
>>>> index in sync etc.
>>>> So I wanted to try a different approach
>>>>
>>>> CaseInsenstiveMap
>>>> ===============
>>>>
>>>> I have implemented a small case insenstive map so we can store headers
>>>> in 1 map instead of keeping 2 maps in sync.
>>>> I am still amazed why such a Map does not exists directly in the JDK
>>>> itself.
>>>>
>>>> What does this offer then
>>>> - lookup is easy as you can lookup using any case
>>>> and in the future we could also offer lookup for special keys people
>>>> spell differently such as: ContentType, content-type, content-type,
>>>> content_type and whatnot.
>>>> - put will also ensure that it replaces existing values
>>>> - and most importantly that it offers this as well when end users work
>>>> directly on the map using:
>>>> exchange.getIn().getHeaders().put("foo", "beer")
>>>> That will be able to replace existing keys if they are stored using:
>>>> Foo, FOO or what not
>>>> This is not possible today and or with a 2nd map that tracks lower
>>>> case indexes
>>>>
>>>> So how to retain the original keys.
>>>>
>>>> Well I implemented that as well, so when you e.g. copy the map to
>>>> another or extract from it using keySet it exposes a Set that uses the
>>>> original key cases.
>>>> So if you have stored using
>>>> exchange.getIn().setHeader("Foo", "cheese")
>>>>
>>>> Then the key in the ketSet is the original key = Foo.
>>>> But you can do lookup using: foo, FOO, FoO, FOo, and whatnot :)
>>>>
>>>> Using this implementation I actually find a few bugs in existing unit
>>>> tests and camel components that double add headers using different
>>>> cases.
>>>> For example camel-cxf in the new RS support.
>>>>
>>>>
>>>> I will later attach the code to the CAMEL-1886 ticket for people of
>>>> the community to take a look and review.
>>>>
>>>>
>>>> Here is a snippet from one of the test methods to give an idea
>>>>
>>>> public void testLookupCaseAgnosticAddHeaderRemoveHeader() {
>>>> Map<String, Object> map = new CaseInsensitiveMap();
>>>> assertNull(map.get("foo"));
>>>>
>>>> map.put("foo", "cheese");
>>>>
>>>> assertEquals("cheese", map.get("foo"));
>>>> assertEquals("cheese", map.get("Foo"));
>>>> assertEquals("cheese", map.get("FOO"));
>>>> assertNull(map.get("unknown"));
>>>>
>>>> map.put("bar", "beer");
>>>>
>>>> assertEquals("beer", map.get("bar"));
>>>> assertEquals("beer", map.get("Bar"));
>>>> assertEquals("beer", map.get("BAR"));
>>>> assertNull(map.get("unknown"));
>>>>
>>>> map.remove("bar");
>>>> assertNull(map.get("bar"));
>>>> assertNull(map.get("unknown"));
>>>> }
>>>>
>>>>
>>>> And when using the Message API
>>>>
>>>> public void testRemoveWithDifferentCase() {
>>>> Message msg = new DefaultMessage();
>>>> assertNull(msg.getHeader("foo"));
>>>>
>>>> msg.setHeader("foo", "cheese");
>>>> msg.setHeader("Foo", "bar");
>>>>
>>>> assertEquals("bar", msg.getHeader("FOO"));
>>>> assertEquals("bar", msg.getHeader("foo"));
>>>> assertEquals("bar", msg.getHeader("Foo"));
>>>>
>>>> msg.removeHeader("FOO");
>>>>
>>>> assertEquals(null, msg.getHeader("foo"));
>>>> assertEquals(null, msg.getHeader("Foo"));
>>>> assertEquals(null, msg.getHeader("FOO"));
>>>>
>>>> assertTrue(msg.getHeaders().isEmpty());
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://fusesource.com/
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Had a chat James as well.
Went for the solution without a KeyValuePair object as it causes pain
with dynamic languages and template engines. And the camel-jms
component had some obscure errors with it also.
Its very easy to revert to old behavior as its just to change to use a
java.util.HashMap in DefaultMessage.
On Sat, Aug 8, 2009 at 9:53 PM, Hadrian Zbarcea<hz...@gmail.com> wrote:
> Hi Claus,
>
> I added a comment on the Jira as well.
>
> Why not use a Map<lowercase-header, Pair<header, Object>> ? I believe this
> to be a simple solution. Lookups should always be on the lowercase version
> of the header, on the wire the original version should go.
>
> What do you think?
> Hadrian
>
> On Aug 8, 2009, at 9:42 AM, Claus Ibsen wrote:
>
>> Hi
>>
>> I have just attached a much better implementation of the map, this
>> time all code is in a single class and it does not use holder objects
>> such as a KeyValuePair when storing the value.
>>
>> It also works out of the box with scripting engines that can access
>> and manipulate the headers such as Freemarker and Velocity.
>>
>> The changes needed in DefaultMessage is basically to use this map
>> instead of a regular HashMap.
>> And when its copied to another map the original keys is automatic
>> retained.
>>
>>
>>
>> On Sat, Aug 8, 2009 at 12:42 PM, Claus Ibsen<cl...@gmail.com> wrote:
>>>
>>> Hi
>>>
>>> For background see ticket CAMEL-1886
>>> https://issues.apache.org/activemq/browse/CAMEL-1886
>>>
>>> The issue is that headers is stored using keys that are case
>>> sensitive, and that leads to a few problems and differences in Camel:
>>>
>>> - lookup is cumbersome as you must match case 100%
>>> - errors in camel components that is unable to realize a header
>>> already exists however with a different (case)
>>> - some protocols are agnostic to header cases, such as HTTP (and all
>>> its different transports using HTTP), Mail, and whatnot?
>>> - camel-mail stores notoriously headers in lower keys only
>>> - etc.
>>>
>>> And most importantly
>>> - no one wants to send different values using a key that have
>>> different case, such as:
>>> Foo=cheese
>>> FOO=cake
>>> foo=beer
>>>
>>> So James and I had a chat about it and to see if would could do
>>> something about it.
>>>
>>> The real reason that I picked up this issue is the tuning experiment
>>> with Camel 2.x
>>> http://camel.apache.org/camel-2x-speed-optimizations.html
>>>
>>> Where we want to take a few short cuts and offer header lookup on the
>>> source directly to avoid the "tiny" overhead of extracting headers
>>> from the source
>>> and populate the camel message. This is useful if you do message
>>> routing that only uses headers. The camel-jms component did already
>>> offer this but
>>> it also had some inconsistency in this relation.
>>>
>>>
>>> I have attempted a resolution in the tuning branch but discovered an
>>> issue, that it does take some more grunt work to keep the lower case
>>> index in sync etc.
>>> So I wanted to try a different approach
>>>
>>> CaseInsenstiveMap
>>> ===============
>>>
>>> I have implemented a small case insenstive map so we can store headers
>>> in 1 map instead of keeping 2 maps in sync.
>>> I am still amazed why such a Map does not exists directly in the JDK
>>> itself.
>>>
>>> What does this offer then
>>> - lookup is easy as you can lookup using any case
>>> and in the future we could also offer lookup for special keys people
>>> spell differently such as: ContentType, content-type, content-type,
>>> content_type and whatnot.
>>> - put will also ensure that it replaces existing values
>>> - and most importantly that it offers this as well when end users work
>>> directly on the map using:
>>> exchange.getIn().getHeaders().put("foo", "beer")
>>> That will be able to replace existing keys if they are stored using:
>>> Foo, FOO or what not
>>> This is not possible today and or with a 2nd map that tracks lower
>>> case indexes
>>>
>>> So how to retain the original keys.
>>>
>>> Well I implemented that as well, so when you e.g. copy the map to
>>> another or extract from it using keySet it exposes a Set that uses the
>>> original key cases.
>>> So if you have stored using
>>> exchange.getIn().setHeader("Foo", "cheese")
>>>
>>> Then the key in the ketSet is the original key = Foo.
>>> But you can do lookup using: foo, FOO, FoO, FOo, and whatnot :)
>>>
>>> Using this implementation I actually find a few bugs in existing unit
>>> tests and camel components that double add headers using different
>>> cases.
>>> For example camel-cxf in the new RS support.
>>>
>>>
>>> I will later attach the code to the CAMEL-1886 ticket for people of
>>> the community to take a look and review.
>>>
>>>
>>> Here is a snippet from one of the test methods to give an idea
>>>
>>> public void testLookupCaseAgnosticAddHeaderRemoveHeader() {
>>> Map<String, Object> map = new CaseInsensitiveMap();
>>> assertNull(map.get("foo"));
>>>
>>> map.put("foo", "cheese");
>>>
>>> assertEquals("cheese", map.get("foo"));
>>> assertEquals("cheese", map.get("Foo"));
>>> assertEquals("cheese", map.get("FOO"));
>>> assertNull(map.get("unknown"));
>>>
>>> map.put("bar", "beer");
>>>
>>> assertEquals("beer", map.get("bar"));
>>> assertEquals("beer", map.get("Bar"));
>>> assertEquals("beer", map.get("BAR"));
>>> assertNull(map.get("unknown"));
>>>
>>> map.remove("bar");
>>> assertNull(map.get("bar"));
>>> assertNull(map.get("unknown"));
>>> }
>>>
>>>
>>> And when using the Message API
>>>
>>> public void testRemoveWithDifferentCase() {
>>> Message msg = new DefaultMessage();
>>> assertNull(msg.getHeader("foo"));
>>>
>>> msg.setHeader("foo", "cheese");
>>> msg.setHeader("Foo", "bar");
>>>
>>> assertEquals("bar", msg.getHeader("FOO"));
>>> assertEquals("bar", msg.getHeader("foo"));
>>> assertEquals("bar", msg.getHeader("Foo"));
>>>
>>> msg.removeHeader("FOO");
>>>
>>> assertEquals(null, msg.getHeader("foo"));
>>> assertEquals(null, msg.getHeader("Foo"));
>>> assertEquals(null, msg.getHeader("FOO"));
>>>
>>> assertTrue(msg.getHeaders().isEmpty());
>>> }
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by Hadrian Zbarcea <hz...@gmail.com>.
Hi Claus,
I added a comment on the Jira as well.
Why not use a Map<lowercase-header, Pair<header, Object>> ? I believe
this to be a simple solution. Lookups should always be on the
lowercase version of the header, on the wire the original version
should go.
What do you think?
Hadrian
On Aug 8, 2009, at 9:42 AM, Claus Ibsen wrote:
> Hi
>
> I have just attached a much better implementation of the map, this
> time all code is in a single class and it does not use holder objects
> such as a KeyValuePair when storing the value.
>
> It also works out of the box with scripting engines that can access
> and manipulate the headers such as Freemarker and Velocity.
>
> The changes needed in DefaultMessage is basically to use this map
> instead of a regular HashMap.
> And when its copied to another map the original keys is automatic
> retained.
>
>
>
> On Sat, Aug 8, 2009 at 12:42 PM, Claus Ibsen<cl...@gmail.com>
> wrote:
>> Hi
>>
>> For background see ticket CAMEL-1886
>> https://issues.apache.org/activemq/browse/CAMEL-1886
>>
>> The issue is that headers is stored using keys that are case
>> sensitive, and that leads to a few problems and differences in Camel:
>>
>> - lookup is cumbersome as you must match case 100%
>> - errors in camel components that is unable to realize a header
>> already exists however with a different (case)
>> - some protocols are agnostic to header cases, such as HTTP (and all
>> its different transports using HTTP), Mail, and whatnot?
>> - camel-mail stores notoriously headers in lower keys only
>> - etc.
>>
>> And most importantly
>> - no one wants to send different values using a key that have
>> different case, such as:
>> Foo=cheese
>> FOO=cake
>> foo=beer
>>
>> So James and I had a chat about it and to see if would could do
>> something about it.
>>
>> The real reason that I picked up this issue is the tuning experiment
>> with Camel 2.x
>> http://camel.apache.org/camel-2x-speed-optimizations.html
>>
>> Where we want to take a few short cuts and offer header lookup on the
>> source directly to avoid the "tiny" overhead of extracting headers
>> from the source
>> and populate the camel message. This is useful if you do message
>> routing that only uses headers. The camel-jms component did already
>> offer this but
>> it also had some inconsistency in this relation.
>>
>>
>> I have attempted a resolution in the tuning branch but discovered an
>> issue, that it does take some more grunt work to keep the lower case
>> index in sync etc.
>> So I wanted to try a different approach
>>
>> CaseInsenstiveMap
>> ===============
>>
>> I have implemented a small case insenstive map so we can store
>> headers
>> in 1 map instead of keeping 2 maps in sync.
>> I am still amazed why such a Map does not exists directly in the
>> JDK itself.
>>
>> What does this offer then
>> - lookup is easy as you can lookup using any case
>> and in the future we could also offer lookup for special keys people
>> spell differently such as: ContentType, content-type, content-type,
>> content_type and whatnot.
>> - put will also ensure that it replaces existing values
>> - and most importantly that it offers this as well when end users
>> work
>> directly on the map using:
>> exchange.getIn().getHeaders().put("foo", "beer")
>> That will be able to replace existing keys if they are stored using:
>> Foo, FOO or what not
>> This is not possible today and or with a 2nd map that tracks lower
>> case indexes
>>
>> So how to retain the original keys.
>>
>> Well I implemented that as well, so when you e.g. copy the map to
>> another or extract from it using keySet it exposes a Set that uses
>> the
>> original key cases.
>> So if you have stored using
>> exchange.getIn().setHeader("Foo", "cheese")
>>
>> Then the key in the ketSet is the original key = Foo.
>> But you can do lookup using: foo, FOO, FoO, FOo, and whatnot :)
>>
>> Using this implementation I actually find a few bugs in existing unit
>> tests and camel components that double add headers using different
>> cases.
>> For example camel-cxf in the new RS support.
>>
>>
>> I will later attach the code to the CAMEL-1886 ticket for people of
>> the community to take a look and review.
>>
>>
>> Here is a snippet from one of the test methods to give an idea
>>
>> public void testLookupCaseAgnosticAddHeaderRemoveHeader() {
>> Map<String, Object> map = new CaseInsensitiveMap();
>> assertNull(map.get("foo"));
>>
>> map.put("foo", "cheese");
>>
>> assertEquals("cheese", map.get("foo"));
>> assertEquals("cheese", map.get("Foo"));
>> assertEquals("cheese", map.get("FOO"));
>> assertNull(map.get("unknown"));
>>
>> map.put("bar", "beer");
>>
>> assertEquals("beer", map.get("bar"));
>> assertEquals("beer", map.get("Bar"));
>> assertEquals("beer", map.get("BAR"));
>> assertNull(map.get("unknown"));
>>
>> map.remove("bar");
>> assertNull(map.get("bar"));
>> assertNull(map.get("unknown"));
>> }
>>
>>
>> And when using the Message API
>>
>> public void testRemoveWithDifferentCase() {
>> Message msg = new DefaultMessage();
>> assertNull(msg.getHeader("foo"));
>>
>> msg.setHeader("foo", "cheese");
>> msg.setHeader("Foo", "bar");
>>
>> assertEquals("bar", msg.getHeader("FOO"));
>> assertEquals("bar", msg.getHeader("foo"));
>> assertEquals("bar", msg.getHeader("Foo"));
>>
>> msg.removeHeader("FOO");
>>
>> assertEquals(null, msg.getHeader("foo"));
>> assertEquals(null, msg.getHeader("Foo"));
>> assertEquals(null, msg.getHeader("FOO"));
>>
>> assertTrue(msg.getHeaders().isEmpty());
>> }
>>
>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
Re: [DISCUSS] - Camel 2.0 - Case insensitive headers on Camel Message
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I have just attached a much better implementation of the map, this
time all code is in a single class and it does not use holder objects
such as a KeyValuePair when storing the value.
It also works out of the box with scripting engines that can access
and manipulate the headers such as Freemarker and Velocity.
The changes needed in DefaultMessage is basically to use this map
instead of a regular HashMap.
And when its copied to another map the original keys is automatic retained.
On Sat, Aug 8, 2009 at 12:42 PM, Claus Ibsen<cl...@gmail.com> wrote:
> Hi
>
> For background see ticket CAMEL-1886
> https://issues.apache.org/activemq/browse/CAMEL-1886
>
> The issue is that headers is stored using keys that are case
> sensitive, and that leads to a few problems and differences in Camel:
>
> - lookup is cumbersome as you must match case 100%
> - errors in camel components that is unable to realize a header
> already exists however with a different (case)
> - some protocols are agnostic to header cases, such as HTTP (and all
> its different transports using HTTP), Mail, and whatnot?
> - camel-mail stores notoriously headers in lower keys only
> - etc.
>
> And most importantly
> - no one wants to send different values using a key that have
> different case, such as:
> Foo=cheese
> FOO=cake
> foo=beer
>
> So James and I had a chat about it and to see if would could do
> something about it.
>
> The real reason that I picked up this issue is the tuning experiment
> with Camel 2.x
> http://camel.apache.org/camel-2x-speed-optimizations.html
>
> Where we want to take a few short cuts and offer header lookup on the
> source directly to avoid the "tiny" overhead of extracting headers
> from the source
> and populate the camel message. This is useful if you do message
> routing that only uses headers. The camel-jms component did already
> offer this but
> it also had some inconsistency in this relation.
>
>
> I have attempted a resolution in the tuning branch but discovered an
> issue, that it does take some more grunt work to keep the lower case
> index in sync etc.
> So I wanted to try a different approach
>
> CaseInsenstiveMap
> ===============
>
> I have implemented a small case insenstive map so we can store headers
> in 1 map instead of keeping 2 maps in sync.
> I am still amazed why such a Map does not exists directly in the JDK itself.
>
> What does this offer then
> - lookup is easy as you can lookup using any case
> and in the future we could also offer lookup for special keys people
> spell differently such as: ContentType, content-type, content-type,
> content_type and whatnot.
> - put will also ensure that it replaces existing values
> - and most importantly that it offers this as well when end users work
> directly on the map using:
> exchange.getIn().getHeaders().put("foo", "beer")
> That will be able to replace existing keys if they are stored using:
> Foo, FOO or what not
> This is not possible today and or with a 2nd map that tracks lower
> case indexes
>
> So how to retain the original keys.
>
> Well I implemented that as well, so when you e.g. copy the map to
> another or extract from it using keySet it exposes a Set that uses the
> original key cases.
> So if you have stored using
> exchange.getIn().setHeader("Foo", "cheese")
>
> Then the key in the ketSet is the original key = Foo.
> But you can do lookup using: foo, FOO, FoO, FOo, and whatnot :)
>
> Using this implementation I actually find a few bugs in existing unit
> tests and camel components that double add headers using different
> cases.
> For example camel-cxf in the new RS support.
>
>
> I will later attach the code to the CAMEL-1886 ticket for people of
> the community to take a look and review.
>
>
> Here is a snippet from one of the test methods to give an idea
>
> public void testLookupCaseAgnosticAddHeaderRemoveHeader() {
> Map<String, Object> map = new CaseInsensitiveMap();
> assertNull(map.get("foo"));
>
> map.put("foo", "cheese");
>
> assertEquals("cheese", map.get("foo"));
> assertEquals("cheese", map.get("Foo"));
> assertEquals("cheese", map.get("FOO"));
> assertNull(map.get("unknown"));
>
> map.put("bar", "beer");
>
> assertEquals("beer", map.get("bar"));
> assertEquals("beer", map.get("Bar"));
> assertEquals("beer", map.get("BAR"));
> assertNull(map.get("unknown"));
>
> map.remove("bar");
> assertNull(map.get("bar"));
> assertNull(map.get("unknown"));
> }
>
>
> And when using the Message API
>
> public void testRemoveWithDifferentCase() {
> Message msg = new DefaultMessage();
> assertNull(msg.getHeader("foo"));
>
> msg.setHeader("foo", "cheese");
> msg.setHeader("Foo", "bar");
>
> assertEquals("bar", msg.getHeader("FOO"));
> assertEquals("bar", msg.getHeader("foo"));
> assertEquals("bar", msg.getHeader("Foo"));
>
> msg.removeHeader("FOO");
>
> assertEquals(null, msg.getHeader("foo"));
> assertEquals(null, msg.getHeader("Foo"));
> assertEquals(null, msg.getHeader("FOO"));
>
> assertTrue(msg.getHeaders().isEmpty());
> }
>
>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus