File component reading files from beginning if the application gets restarted

[Akram <ak...@gmail.com>]

I'm attempting to consume messages from an file using an idempotent Key to
ensure that I consume the messages from the newly generated files by rolling
over the old one.

Here is my file component route

from("file:myFolder?noop=true&fileName=Myfile.json&idempotent=true&idempotentKey=${file:modified}-${file:size})

When I start my application for the first time, everything working fine and
also able to consume the messages from the newly generated files by rolling
over the old one.

When i restart the application it is reading the current file from the
beginning, instead of reading it from where it is left.

To resolve the above issue i tried using idempotent repo. But when the
current file(Myfile.json) gets modified, it is reading the file from the
beginning.

Here is my idempotent repo configuration
<bean id="fileStore"
class="org.apache.camel.processor.idempotent.FileIdempotentRepository">
    <property name="fileStore" value="target/.filestore.dat"/>
    <property name="maxFileStoreSize" value="512000"/>
    <property name="cacheSize" value="250"/>
</bean>

Here is my route
from("file:myFolder?noop=true&fileName=Myfile.json&idempotent=true&idempotentKey=${file:modified}-${file:size}&idempotentRepository=#fileStore")

Am i missing something here?



--
View this message in context: http://camel.465427.n5.nabble.com/File-component-reading-files-from-beginning-if-the-application-gets-restarted-tp5766163.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: File component reading files from beginning if the application gets restarted

[Akram <ak...@gmail.com>]

Then having an entry in idempotent repo is of no use. If i add a content to
the current file, it will read from the beginning only.



--
View this message in context: http://camel.465427.n5.nabble.com/File-component-reading-files-from-beginning-if-the-application-gets-restarted-tp5766163p5766166.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: File component reading files from beginning if the application gets restarted

[Claus Ibsen <cl...@gmail.com>]

Hi

That is how the file component works. If the file is "detected as
changed etc" then its routing the file again.

If you want to read the same file as a "stream" then look at the
stream component
http://camel.apache.org/stream

And btw that component will also read the file from the beginning if
you restart the JVM / app etc.
If you really only want to read a file chunk by chunk only once, you
need to make your own "house keeping" how far you have read and so on.

On Wed, Apr 22, 2015 at 4:24 PM, Akram <ak...@gmail.com> wrote:
> I'm attempting to consume messages from an file using an idempotent Key to
> ensure that I consume the messages from the newly generated files by rolling
> over the old one.
>
> Here is my file component route
>
> from("file:myFolder?noop=true&fileName=Myfile.json&idempotent=true&idempotentKey=${file:modified}-${file:size})
>
> When I start my application for the first time, everything working fine and
> also able to consume the messages from the newly generated files by rolling
> over the old one.
>
> When i restart the application it is reading the current file from the
> beginning, instead of reading it from where it is left.
>
> To resolve the above issue i tried using idempotent repo. But when the
> current file(Myfile.json) gets modified, it is reading the file from the
> beginning.
>
> Here is my idempotent repo configuration
> <bean id="fileStore"
> class="org.apache.camel.processor.idempotent.FileIdempotentRepository">
>     <property name="fileStore" value="target/.filestore.dat"/>
>     <property name="maxFileStoreSize" value="512000"/>
>     <property name="cacheSize" value="250"/>
> </bean>
>
> Here is my route
> from("file:myFolder?noop=true&fileName=Myfile.json&idempotent=true&idempotentKey=${file:modified}-${file:size}&idempotentRepository=#fileStore")
>
> Am i missing something here?
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-component-reading-files-from-beginning-if-the-application-gets-restarted-tp5766163.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/