You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by kikou1984 <hi...@atos.net> on 2015/11/18 15:00:06 UTC

Is There a way to treat a large file?

I m using apache Camel to treat files.

However, i m asking if i can split a file depending on the number of byte.

For exemple , if i have a file with a size like 1 GO, is that possible to
read it by block of 10 MO. (with a parameter that defines the size of a
block)

Is there a component Camel EIP able to do this ?

I have tried with stream-caching but nothing happened.

    <camelContext xmlns="http://camel.apache.org/schema/spring"
trace="true">

    <properties>
        <property key="CamelCachedOutputStreamBufferSize" value="1024"/>
    </properties>


    <route id="READ-FILE-STREAM-CACHE" streamCache="true">
        <from uri="file:src/data?noop=true"/>
        <bean ref="readBean" method="readFileStream"/>
        <to uri="file:src/out"/>
    </route>

</camelContext>
Thanks



--
View this message in context: http://camel.465427.n5.nabble.com/Is-There-a-way-to-treat-a-large-file-tp5774055.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Is There a way to treat a large file?

Posted by Claus Ibsen <cl...@gmail.com>.
In your java method readFileStream

public void "readFileStream(FileInputStream is) {
 .. read the file in chunks as you wish
}

You can also look at the splitter eip in streaming mode, and provide
some expression that reads X chunks of bytes or split the big file in
X number of lines or something.

There is a tokenizer that can split by a token such as line terminator
and group N lines.


On Wed, Nov 18, 2015 at 3:26 PM, kikou1984 <hi...@atos.net> wrote:
> Thank you for the answer
>
> I didn't really understand,  how i can i add a parameter to a bean ?  ( Also
> should i keep the property CamelCachedOutputStreamBufferSize ? )
>
> Can you give me an example please.
>
> Thank you.
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Is-There-a-way-to-treat-a-large-file-tp5774055p5774059.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Is There a way to treat a large file?

Posted by kikou1984 <hi...@atos.net>.
Thank you for the answer

I didn't really understand,  how i can i add a parameter to a bean ?  ( Also
should i keep the property CamelCachedOutputStreamBufferSize ? ) 

Can you give me an example please.

Thank you.




--
View this message in context: http://camel.465427.n5.nabble.com/Is-There-a-way-to-treat-a-large-file-tp5774055p5774059.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Is There a way to treat a large file?

Posted by Claus Ibsen <cl...@gmail.com>.
You can just use FileInputStream or File etc as the parameter in your
readBean then the file is not loaded into memory.

Also there is the stream component but its not exactly the same as the
file component
http://camel.apache.org/stream



On Wed, Nov 18, 2015 at 3:00 PM, kikou1984 <hi...@atos.net> wrote:
> I m using apache Camel to treat files.
>
> However, i m asking if i can split a file depending on the number of byte.
>
> For exemple , if i have a file with a size like 1 GO, is that possible to
> read it by block of 10 MO. (with a parameter that defines the size of a
> block)
>
> Is there a component Camel EIP able to do this ?
>
> I have tried with stream-caching but nothing happened.
>
>     <camelContext xmlns="http://camel.apache.org/schema/spring"
> trace="true">
>
>     <properties>
>         <property key="CamelCachedOutputStreamBufferSize" value="1024"/>
>     </properties>
>
>
>     <route id="READ-FILE-STREAM-CACHE" streamCache="true">
>         <from uri="file:src/data?noop=true"/>
>         <bean ref="readBean" method="readFileStream"/>
>         <to uri="file:src/out"/>
>     </route>
>
> </camelContext>
> Thanks
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Is-There-a-way-to-treat-a-large-file-tp5774055.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2