You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by kikou1984 <hi...@atos.net> on 2015/12/09 13:41:08 UTC
Using Stream Cache With Apache Camel
I m using stream caching with this example below.
I have three routes.
First, I used the split streaming with a token "\n", reading a file splitted
by line.
<route id="SPLIT-FLOW" streamCache="true">
<from uri="file:src/data/forSplitCaching?noop=true"/>
<split streaming="true">
<tokenize token="\n"/>
<to uri="direct:PROCESS-BUSINESS"/>
</split>
</route>
Then I made a business process :
<route id="PROCESS-BUSINESS" streamCache="true">
<from uri="direct:PROCESS-BUSINESS"/>
<bean ref="ProcessBusiness" method="dealRecord"/>
<to uri="direct:STREAM-CACHING"/>
</route>
public class ProcessBusiness {
public String dealRecord(@Body String body){
System.out.println(body);
return body;
}
And Finally,
<route id="STREAM-CACHING">
<from uri="direct:STREAM-CACHING"/>
<bean ref="ProcessStreamCaching" method="usingStream"/>
<setHeader headerName="CamelFileName">
<simple>${header.CamelFileName}.${header.CamelSplitIndex}
</simple>
</setHeader>
<to uri="file:src/out"/>
</route>
public class ProcessStreamCaching {
public String usingStream(Exchange exchange){
Object o = exchange.getIn().getBody(StreamCache.class);
return o.toString();
}
I wanted to get the whole file in this process (`ProcessStreamCaching`).
How can i correctly use the stream cache ? How can i get read the file again
from the beginning, that s why i want to use the stream cache.
The file before splitting contain :
XXXXX
YYYYY
ZZZZZ
AAAAAAAAAAAAA
BBBBBBBB
CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
SSSSSSSSSSSSSSSS
SSQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ
ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
Z
I thought that the stream cache was able to put all line of the file in a
cache, and i can retrieve these lines.
How can I see what is placed in the cache ?
I thought when i add `streamCache="true"` at this route `<route
id="SPLIT-FLOW" >`, each line of the splitter will be placed on the cache.
(or may be the whole file)
At the last process `ProcessStreamCaching`, if I wanted to reload the
`inputStream` from the beginning , but at this process i can't because i
have already read the `inputStream`.
I want to keep reading the file with streaming line per line, but at on
process I need to read for exemple the first line , that s why i wanted to
use the stream cache , who allows me to browse the file from the beginning.
May be I misunderstand the Stream Cache.
--
View this message in context: http://camel.465427.n5.nabble.com/Using-Stream-Cache-With-Apache-Camel-tp5774856.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Using Stream Cache With Apache Camel
Posted by kikou1984 <hi...@atos.net>.
When i m trying to get the cache it returns null
StreamCache cache = exchange.getIn().getBody(StreamCache.class);
Do you have an idea how i can retrieve the StreamCache ?
Thxs
--
View this message in context: http://camel.465427.n5.nabble.com/Using-Stream-Cache-With-Apache-Camel-tp5774856p5774868.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Using Stream Cache With Apache Camel
Posted by kikou1984 <hi...@atos.net>.
Hi Claus,
Well, the main purpose is to read a file block by block (for exemple, each
10 Mo) , for that i thought to use the splitter for split sub message, then
for other business process, i need to reload the file from the begining (not
really from the beginning but some date that has already be splitted) , for
exemple i need to retrieve the first line to make some operations ,
therefore i thought using streamcache.
I can't parsing many time the same File , because it takes a lot of memory
,mainly in the world of banking , we use very large files , so we have to
use splitter then find away to use the stream cache.
So , do you have an example about Stream cache, how can i put some data ( I
think we have to add on the route streamCache="true") and how can i retrieve
it.
Thxs.
--
View this message in context: http://camel.465427.n5.nabble.com/Using-Stream-Cache-With-Apache-Camel-tp5774856p5774923.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Using Stream Cache With Apache Camel
Posted by Claus Ibsen <cl...@gmail.com>.
The splitter will split the file and each of those splitted sub
messages do not have the entire content, but only that chunk of the
message. If you want to read the entire file again, then dont use the
splitter, but just do a from file -> to process, and you can access
the file using java.io.File / InputStream etc.
On Wed, Dec 9, 2015 at 1:41 PM, kikou1984 <hi...@atos.net> wrote:
> I m using stream caching with this example below.
>
> I have three routes.
>
> First, I used the split streaming with a token "\n", reading a file splitted
> by line.
>
> <route id="SPLIT-FLOW" streamCache="true">
> <from uri="file:src/data/forSplitCaching?noop=true"/>
> <split streaming="true">
> <tokenize token="\n"/>
> <to uri="direct:PROCESS-BUSINESS"/>
> </split>
> </route>
>
>
> Then I made a business process :
>
> <route id="PROCESS-BUSINESS" streamCache="true">
> <from uri="direct:PROCESS-BUSINESS"/>
> <bean ref="ProcessBusiness" method="dealRecord"/>
> <to uri="direct:STREAM-CACHING"/>
> </route>
>
> public class ProcessBusiness {
>
> public String dealRecord(@Body String body){
> System.out.println(body);
> return body;
> }
>
> And Finally,
>
> <route id="STREAM-CACHING">
> <from uri="direct:STREAM-CACHING"/>
> <bean ref="ProcessStreamCaching" method="usingStream"/>
> <setHeader headerName="CamelFileName">
> <simple>${header.CamelFileName}.${header.CamelSplitIndex}
> </simple>
> </setHeader>
> <to uri="file:src/out"/>
> </route>
>
> public class ProcessStreamCaching {
>
> public String usingStream(Exchange exchange){
> Object o = exchange.getIn().getBody(StreamCache.class);
> return o.toString();
> }
>
> I wanted to get the whole file in this process (`ProcessStreamCaching`).
>
> How can i correctly use the stream cache ? How can i get read the file again
> from the beginning, that s why i want to use the stream cache.
>
> The file before splitting contain :
>
> XXXXX
> YYYYY
> ZZZZZ
> AAAAAAAAAAAAA
> BBBBBBBB
> CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
> SSSSSSSSSSSSSSSS
> SSQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ
> ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
> Z
>
> I thought that the stream cache was able to put all line of the file in a
> cache, and i can retrieve these lines.
>
> How can I see what is placed in the cache ?
>
> I thought when i add `streamCache="true"` at this route `<route
> id="SPLIT-FLOW" >`, each line of the splitter will be placed on the cache.
> (or may be the whole file)
>
> At the last process `ProcessStreamCaching`, if I wanted to reload the
> `inputStream` from the beginning , but at this process i can't because i
> have already read the `inputStream`.
>
> I want to keep reading the file with streaming line per line, but at on
> process I need to read for exemple the first line , that s why i wanted to
> use the stream cache , who allows me to browse the file from the beginning.
>
> May be I misunderstand the Stream Cache.
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Using-Stream-Cache-With-Apache-Camel-tp5774856.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2