Splitting a List and deleting it at the end

[Yann <ya...@gmail.com>]

Hi,

I'm splitting a list containing 100 000 objects. Each object is sent to a
queue after a short delay.
If I don't clear my list, my JVM will explode (java heap space...). How
could I get my initial list after splitting it ?
Below my route : 
from("bean:bean?method=read")
	 // List split
	 .split(body(List.class))
         // delay
         .delay(1)
         // body conversion
	.convertBodyTo(Person.class)
         // queue
	.to("activemq:queue");
        // this is where I should be able to clear the list!

Thanks for any help !
Yann
-- 
View this message in context: http://camel.465427.n5.nabble.com/Splitting-a-List-and-deleting-it-at-the-end-tp2256727p2256727.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Splitting a List and deleting it at the end

[ext2 <xu...@tongtech.com>]

Backup it in a message header before entering split, then you can use it.

But if memory cost is a critical problem for you, you'd better to redesign
your 
from endpoint's implementation, which should not loading all objects in
memory at once, but return a iterator which will sequential load the objects
to memory while iterating. 

Also you should enable "stream" options of split pattern, it will use you
iterator to iterating object;

Finally , you must using try-finally to guarantee the iterator being closed;

----original-----
Sender: Yann [mailto:yann.bouffant@gmail.com] 
Date: 2010/7/29 21:23
Receiver: users@camel.apache.org
Subject: Splitting a List and deleting it at the end


Hi,

I'm splitting a list containing 100 000 objects. Each object is sent to a
queue after a short delay.
If I don't clear my list, my JVM will explode (java heap space...). How
could I get my initial list after splitting it ?
Below my route : 
from("bean:bean?method=read")
	 // List split
	 .split(body(List.class))
         // delay
         .delay(1)
         // body conversion
	.convertBodyTo(Person.class)
         // queue
	.to("activemq:queue");
        // this is where I should be able to clear the list!

Thanks for any help !
Yann
-- 
View this message in context:
http://camel.465427.n5.nabble.com/Splitting-a-List-and-deleting-it-at-the-en
d-tp2256727p2256727.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Splitting a List and deleting it at the end

[Claus Ibsen <cl...@gmail.com>]

Hi

Use streaming mode which can then iterate the payload.

So if your payload can be converted into an Iterator then you dont
need to have all the 100 000 objects loaded into memory.



On Thu, Jul 29, 2010 at 3:22 PM, Yann <ya...@gmail.com> wrote:
>
> Hi,
>
> I'm splitting a list containing 100 000 objects. Each object is sent to a
> queue after a short delay.
> If I don't clear my list, my JVM will explode (java heap space...). How
> could I get my initial list after splitting it ?
> Below my route :
> from("bean:bean?method=read")
>         // List split
>         .split(body(List.class))
>         // delay
>         .delay(1)
>         // body conversion
>        .convertBodyTo(Person.class)
>         // queue
>        .to("activemq:queue");
>        // this is where I should be able to clear the list!
>
> Thanks for any help !
> Yann
> --
> View this message in context: http://camel.465427.n5.nabble.com/Splitting-a-List-and-deleting-it-at-the-end-tp2256727p2256727.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus