You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by sakouhi <sa...@gmail.com> on 2017/05/15 11:00:16 UTC
Camel FTP: how to work with corrupt files.
Hi,
I'm newbie in apache camel. I configured a route from camel ftp to JMS
topic. I noticed that the processed files, they are either deleted, moved
either processed again and again. In my work, I have a lot of corrupt files
that are loaded into HDFS, so I need to reentered the same files into FTP, I
don't need to send again the full file, but this time extract only the new
lines (appended lines).
Thanks,
Regards
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-FTP-how-to-work-with-corrupt-files-tp5799531.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel FTP: how to work with corrupt files.
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
You can see the full list of what Camel components there are at this link:
https://github.com/apache/camel/tree/master/components#components
On Tue, May 16, 2017 at 11:00 AM, sakouhi <sa...@gmail.com> wrote:
> Thank you Sir,
> Another question please? is there a flume component in apache camel? I saw
> kafka, flink, ActiveMQ....... I have 2 months trying to integrate apache
> camel with flume. I hope to find a possible way do to that.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-FTP-how-to-work-with-corrupt-files-tp5799531p5799572.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Camel FTP: how to work with corrupt files.
Posted by sakouhi <sa...@gmail.com>.
Thank you Sir,
Another question please? is there a flume component in apache camel? I saw
kafka, flink, ActiveMQ....... I have 2 months trying to integrate apache
camel with flume. I hope to find a possible way do to that.
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-FTP-how-to-work-with-corrupt-files-tp5799531p5799572.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel FTP: how to work with corrupt files.
Posted by fkdiogo <fk...@gmail.com>.
Hey Claus,
I'm having an issue with corrupted files. I'm uploading the entire file to
an ftp through camel-ftp, I'm not getting
any error, but when I consume it, sometimes it's corrupted (missing some
lines in it).
My guess is that maybe the connection is dropping when the file is being
uploaded.
Do you have any guess to what it could be and how it could be solved?
Thanks alot.
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-FTP-how-to-work-with-corrupt-files-tp5799531p5799621.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel FTP: how to work with corrupt files.
Posted by Claus Ibsen <cl...@gmail.com>.
There is no such functionality in camel-ftp. It uploads the entire file.
On Tue, May 16, 2017 at 10:06 AM, sakouhi <sa...@gmail.com> wrote:
> Thanks for reply. I work on Big data project. Sometimes the FTP server
> receives an incomplete files. Then, I need to load them again. I no want to
> have a duplicate files into hdfs. I just need to load the new lines. I look
> for an option in FTP component that scan the new lines and send them to
> JMS/Flume ....
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-FTP-how-to-work-with-corrupt-files-tp5799531p5799560.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Camel FTP: how to work with corrupt files.
Posted by sakouhi <sa...@gmail.com>.
Thanks for reply. I work on Big data project. Sometimes the FTP server
receives an incomplete files. Then, I need to load them again. I no want to
have a duplicate files into hdfs. I just need to load the new lines. I look
for an option in FTP component that scan the new lines and send them to
JMS/Flume ....
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-FTP-how-to-work-with-corrupt-files-tp5799531p5799560.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel FTP: how to work with corrupt files.
Posted by Frank Wein <fr...@fau.de>.
Hi,
I'm not sure what your problem is, can you maybe try to re-phrase it? Do
you mean that you only want to partially transfer a file to FTP (and you
already know which file part you want to re-transfer)?
Frank
sakouhi wrote:
> Hi,
>
> I'm newbie in apache camel. I configured a route from camel ftp to JMS
> topic. I noticed that the processed files, they are either deleted, moved
> either processed again and again. In my work, I have a lot of corrupt files
> that are loaded into HDFS, so I need to reentered the same files into FTP, I
> don't need to send again the full file, but this time extract only the new
> lines (appended lines).
>
> Thanks,
> Regards
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-FTP-how-to-work-with-corrupt-files-tp5799531.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Frank Wein
Forschungsgruppe Netz
Friedrich-Alexander-Universitaet Erlangen-Nuernberg
Regionales Rechenzentrum Erlangen (RRZE)
Martensstrasse 1, 91058 Erlangen, Germany
Tel. +49 9131/85-29983, Fax +49 9131/302941
frank.wein@fau.de
www.rrze.fau.de