You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by waterback <ma...@innoq.com> on 2010/01/27 12:53:53 UTC
Heavyweight Converter: Caching?
I have a big question regarding TypeConverters - how to cache a
heavyweight-converter:
Actually i have a TypeConverter using a Pojo for converting a
COBOL-Object-Dump (don't ask!:), which is not as reentrant as the
TypeConverter-Semantic requests it.
I have concurreny issues.
The thing is - that Pojo is too heavyweight to make a new instance every
time i want to use it:
It does a lot of initializing stuff...
Well that Pojo was used in a EJB-Environment before - therefore it was
instantiated for every instance of a SLSB and this worked fine since every
thread had his own instance and the ejb container managed the caching of it
How can i solve this in camel? I could imagine using a bean, or a Processor,
but i can't see a solution without creating a new instance for every
request.
Can i cache Processors or Beans and direct camel to use a different
cache-object for every request?
Thanx for any hint...
regards,
martin
--
View this message in context: http://old.nabble.com/Heavyweight-Converter%3A-Caching--tp27337830p27337830.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Heavyweight Converter: Caching?
Posted by waterback <ma...@innoq.com>.
Hi Willem,
threadlocal does the trick...
Thank you all for helping...
best wishes,
martin
willem.jiang wrote:
>
> Hi,
>
> Is your MidBuffer2CanonicXML a thread safe object?
> If so you can change your process method to be static.
> If not, you can try to cache the object as thread local object.
>
> Willem
>
> waterback wrote:
>> Thanx for helping...
>>
>> Well i put here my Converter's code... As you can see i had the
>> "Transformer" as
>> an instance variable, but there go the concurrency issues....
>> Right now i've commented it and make an instance every time right before
>> transformation.
>> This MidBuffer2CanonicXML-Class is the heavyweight one...
>>
>> i have read in documentation of the type converters that they get cached
>> when defined not static...
>> But can i influence camel to use a new cached instance when a new
>> exchange
>> gets worked at?
>>
>> @Converter
>> public class ObjectNet2Document {
>>
>> // protected Transformer<MidBufferHeader, Document> transformer =
>>
>> new MidBuffer2CanonicXML();
>> @Converter
>> public Document process(byte[] raw) throws Exception {
>>
>> int rawSize = raw.length;
>> MidBufferHeader mbh = new MidBufferHeader(raw, 0, rawSize);
>> Transformer<MidBufferHeader, Document> transformer = new
>> MidBuffer2CanonicXML();
>> Document document = transformer.transform(mbh);
>> return document;
>> }
>>
>> }
>>
>> Martin
>>
>>
>> willem.jiang wrote:
>>> How did you user the TypeConverter?
>>> If you can cache the CamelContext, I think you will create a new
>>> TypeConverter each time.
>>> BTW,
>>> With the recent change of CAMEL-2392[1], CamelContext will load the
>>> TypeConverter when the CamelContext is started.
>>>
>>> [1] https://issues.apache.org/activemq/browse/CAMEL-2392
>>>
>>> Willem
>>>
>>
>
>
>
--
View this message in context: http://old.nabble.com/Heavyweight-Converter%3A-Caching--tp27337830p27340210.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Heavyweight Converter: Caching?
Posted by Willem Jiang <wi...@gmail.com>.
Hi,
Is your MidBuffer2CanonicXML a thread safe object?
If so you can change your process method to be static.
If not, you can try to cache the object as thread local object.
Willem
waterback wrote:
> Thanx for helping...
>
> Well i put here my Converter's code... As you can see i had the
> "Transformer" as
> an instance variable, but there go the concurrency issues....
> Right now i've commented it and make an instance every time right before
> transformation.
> This MidBuffer2CanonicXML-Class is the heavyweight one...
>
> i have read in documentation of the type converters that they get cached
> when defined not static...
> But can i influence camel to use a new cached instance when a new exchange
> gets worked at?
>
> @Converter
> public class ObjectNet2Document {
>
> // protected Transformer<MidBufferHeader, Document> transformer =
>
> new MidBuffer2CanonicXML();
> @Converter
> public Document process(byte[] raw) throws Exception {
>
> int rawSize = raw.length;
> MidBufferHeader mbh = new MidBufferHeader(raw, 0, rawSize);
> Transformer<MidBufferHeader, Document> transformer = new
> MidBuffer2CanonicXML();
> Document document = transformer.transform(mbh);
> return document;
> }
>
> }
>
> Martin
>
>
> willem.jiang wrote:
>> How did you user the TypeConverter?
>> If you can cache the CamelContext, I think you will create a new
>> TypeConverter each time.
>> BTW,
>> With the recent change of CAMEL-2392[1], CamelContext will load the
>> TypeConverter when the CamelContext is started.
>>
>> [1] https://issues.apache.org/activemq/browse/CAMEL-2392
>>
>> Willem
>>
>
Re: Heavyweight Converter: Caching?
Posted by waterback <ma...@innoq.com>.
Thanx for helping...
Well i put here my Converter's code... As you can see i had the
"Transformer" as
an instance variable, but there go the concurrency issues....
Right now i've commented it and make an instance every time right before
transformation.
This MidBuffer2CanonicXML-Class is the heavyweight one...
i have read in documentation of the type converters that they get cached
when defined not static...
But can i influence camel to use a new cached instance when a new exchange
gets worked at?
@Converter
public class ObjectNet2Document {
// protected Transformer<MidBufferHeader, Document> transformer =
new MidBuffer2CanonicXML();
@Converter
public Document process(byte[] raw) throws Exception {
int rawSize = raw.length;
MidBufferHeader mbh = new MidBufferHeader(raw, 0, rawSize);
Transformer<MidBufferHeader, Document> transformer = new
MidBuffer2CanonicXML();
Document document = transformer.transform(mbh);
return document;
}
}
Martin
willem.jiang wrote:
>
> How did you user the TypeConverter?
> If you can cache the CamelContext, I think you will create a new
> TypeConverter each time.
> BTW,
> With the recent change of CAMEL-2392[1], CamelContext will load the
> TypeConverter when the CamelContext is started.
>
> [1] https://issues.apache.org/activemq/browse/CAMEL-2392
>
> Willem
>
--
View this message in context: http://old.nabble.com/Heavyweight-Converter%3A-Caching--tp27337830p27338187.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Heavyweight Converter: Caching?
Posted by Willem Jiang <wi...@gmail.com>.
How did you user the TypeConverter?
If you can cache the CamelContext, I think you will create a new
TypeConverter each time.
BTW,
With the recent change of CAMEL-2392[1], CamelContext will load the
TypeConverter when the CamelContext is started.
[1] https://issues.apache.org/activemq/browse/CAMEL-2392
Willem
waterback wrote:
> I have a big question regarding TypeConverters - how to cache a
> heavyweight-converter:
>
> Actually i have a TypeConverter using a Pojo for converting a
> COBOL-Object-Dump (don't ask!:), which is not as reentrant as the
> TypeConverter-Semantic requests it.
>
> I have concurreny issues.
>
> The thing is - that Pojo is too heavyweight to make a new instance every
> time i want to use it:
> It does a lot of initializing stuff...
>
> Well that Pojo was used in a EJB-Environment before - therefore it was
> instantiated for every instance of a SLSB and this worked fine since every
> thread had his own instance and the ejb container managed the caching of it
>
> How can i solve this in camel? I could imagine using a bean, or a Processor,
> but i can't see a solution without creating a new instance for every
> request.
>
> Can i cache Processors or Beans and direct camel to use a different
> cache-object for every request?
>
> Thanx for any hint...
>
> regards,
> martin
>
Re: Heavyweight Converter: Caching?
Posted by Ashwin Karpe <as...@progress.com>.
Hi,
No, the JNDI Registry is not designed for caching objects but to store
object references (TypeConverter instances included..). What I was trying to
say was that if it is expensive to constantly create instances of the object
you need, you could store the reference(s) of all said
TypeConverters(assuming there are not too many of them :)) and then pull it
up and use it on demand without incurring the cost of object creation.
If you need a true cache for your solution to work, there is a Camel Cache
component that allows any objects that is capable of storing serializable
objects. If your Type converters implement Serializable, they could be
stored in the Cache and then used via Camel processors on demand.
Cheers,
Ashwin...
http://camel.apache.org/cache.html http://camel.apache.org/cache.html
waterback wrote:
>
> I quite don't get the idea of it... Is there some sort of caching involved
> then using that JndiRegistry
> (assuming i understand that enough, to make it work :-).
>
> When i just get the reference and it is just one object, i don't win
> anything, or am i wrong?
>
> Thank you Ashwin,
> Good Idea: Cheers,
> Martin
>
>
>
> Ashwin Karpe wrote:
>>
>> Hi,
>>
>> The most straightforward way I can think of is to load the Object
>> reference into a JNDI Registry and associate it with the CamelContext
>> during instantiation or by using setters methods at a later point.
>>
>> The reference can then be looked up as and when needed to get a reference
>> to the TypeConverter.
>>
>> Check out the following link to see how this may be done.
>>
>> http://camel.apache.org/lucene.html http://camel.apache.org/lucene.html
>>
>> Cheers,
>>
>> Ashwin...
>>
>> Hope this helps.
>>
>> Cheers,
>>
>> Ashwin...
>>
>>
>
>
-----
---
Ashwin Karpe, Principal Consultant, PS - Opensource Center of Competence
Progress Software Corporation
14 Oak Park Drive
Bedford, MA 01730
---
+1-972-304-9084 (Office)
+1-972-971-1700 (Mobile)
----
Blog: http://opensourceknowledge.blogspot.com/
--
View this message in context: http://old.nabble.com/Heavyweight-Converter%3A-Caching--tp27337830p27349593.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Heavyweight Converter: Caching?
Posted by Ashwin Karpe <as...@progress.com>.
Hi,
There is no caching involved in using the JNDI registry. What I was
suggesting was that if this POJO or set of POJO's (assuming these are in the
1-15 unique objects range) can be created upfront and stored in a registry.
The required POJO/bean could then be pulled up in a processor as follows by
referring to the registry
public class MyProcessor implements Processor {
public void process(Exchange exchange) throws Exception {
MyPojo pojo = (MyPojo)
(exchange.getContext().getRegistry().lookup("ObjectName");
}
}
Looks like Willem has resolved some of this already using threadLocal
storage.
I am just providing a generic way to cut the cost of object creation and
using the object quickly without involving expensive caching strategies.
Sorry about any confusion... Hope this helps...
Cheers,
Ashwin...
waterback wrote:
>
> I quite don't get the idea of it... Is there some sort of caching involved
> then using that JndiRegistry
> (assuming i understand that enough, to make it work :-).
>
> When i just get the reference and it is just one object, i don't win
> anything, or am i wrong?
>
> Thank you Ashwin,
> Good Idea: Cheers,
> Martin
>
>
>
> Ashwin Karpe wrote:
>>
>> Hi,
>>
>> The most straightforward way I can think of is to load the Object
>> reference into a JNDI Registry and associate it with the CamelContext
>> during instantiation or by using setters methods at a later point.
>>
>> The reference can then be looked up as and when needed to get a reference
>> to the TypeConverter.
>>
>> Check out the following link to see how this may be done.
>>
>> http://camel.apache.org/lucene.html http://camel.apache.org/lucene.html
>>
>> Cheers,
>>
>> Ashwin...
>>
>> Hope this helps.
>>
>> Cheers,
>>
>> Ashwin...
>>
>>
>
>
-----
---
Ashwin Karpe, Principal Consultant, PS - Opensource Center of Competence
Progress Software Corporation
14 Oak Park Drive
Bedford, MA 01730
---
+1-972-304-9084 (Office)
+1-972-971-1700 (Mobile)
----
Blog: http://opensourceknowledge.blogspot.com/
--
View this message in context: http://old.nabble.com/Heavyweight-Converter%3A-Caching--tp27337830p27349826.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Heavyweight Converter: Caching?
Posted by waterback <ma...@innoq.com>.
I quite don't get the idea of it... Is there some sort of caching involved
then using that JndiRegistry
(assuming i understand that enough, to make it work :-).
When i just get the reference and it is just one object, i don't win
anything, or am i wrong?
Thank you Ashwin,
Good Idea: Cheers,
Martin
Ashwin Karpe wrote:
>
> Hi,
>
> The most straightforward way I can think of is to load the Object
> reference into a JNDI Registry and associate it with the CamelContext
> during instantiation or by using setters methods at a later point.
>
> The reference can then be looked up as and when needed to get a reference
> to the TypeConverter.
>
> Check out the following link to see how this may be done.
>
> http://camel.apache.org/lucene.html http://camel.apache.org/lucene.html
>
> Cheers,
>
> Ashwin...
>
> Hope this helps.
>
> Cheers,
>
> Ashwin...
>
>
--
View this message in context: http://old.nabble.com/Heavyweight-Converter%3A-Caching--tp27337830p27338518.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Heavyweight Converter: Caching?
Posted by Ashwin Karpe <as...@progress.com>.
Hi,
The most straightforward way I can think of is to load the Object reference
into a JNDI Registry and associated with the CamelContext during
instantiation of using setters.
The reference can then be looked up as a when needed to get a reference to
the TypeConverter.
Check out the following link to see how this may be done.
http://camel.apache.org/lucene.html http://camel.apache.org/lucene.html
Cheers,
Ashwin...
Hope this helps.
Cheers,
Ashwin...
waterback wrote:
>
> I have a big question regarding TypeConverters - how to cache a
> heavyweight-converter:
>
> Actually i have a TypeConverter using a Pojo for converting a
> COBOL-Object-Dump (don't ask!:), which is not as reentrant as the
> TypeConverter-Semantic requests it.
>
> I have concurreny issues.
>
> The thing is - that Pojo is too heavyweight to make a new instance every
> time i want to use it:
> It does a lot of initializing stuff...
>
> Well that Pojo was used in a EJB-Environment before - therefore it was
> instantiated for every instance of a SLSB and this worked fine since every
> thread had his own instance and the ejb container managed the caching of
> it
>
> How can i solve this in camel? I could imagine using a bean, or a
> Processor, but i can't see a solution without creating a new instance for
> every request.
>
> Can i cache Processors or Beans and direct camel to use a different
> cache-object for every request?
>
> Thanx for any hint...
>
> regards,
> martin
>
>
-----
---
Ashwin Karpe, Principal Consultant, PS - Opensource Center of Competence
Progress Software Corporation
14 Oak Park Drive
Bedford, MA 01730
---
+1-972-304-9084 (Office)
+1-972-971-1700 (Mobile)
----
Blog: http://opensourceknowledge.blogspot.com/
--
View this message in context: http://old.nabble.com/Heavyweight-Converter%3A-Caching--tp27337830p27338179.html
Sent from the Camel - Users mailing list archive at Nabble.com.