How to consume selected files from FTP endpoint?

[Jack Frosch <ja...@gmail.com>]

Our application consumes files from a folder on a supplier's FTP server.  Unfortunately, the supplier uses the same folder for documents intended to be consumed by other applications. We’d like to consume (i.e. read, then delete or move) our files which can be identified by file name pattern, leaving the other files we don’t care about in the folder.

I was hoping a filter would do this, but it seems the filter gets applied after the file is consumed.

I’m sure I’m overlooking something, but am Googled out. :-)

Is there a way to configure the FTP endpoint using Camel 2.15.1 to only consume a subset of files from a folder on an FTP server?

Thanks in advance!

—
Jack

Re: How to consume selected files from FTP endpoint?

[Jack Frosch <jf...@objistics.com>]

Thanks Claus.

I’ll take another run at this.

—
Jack Frosch







On 11/12/15, 10:08 AM, "Claus Ibsen" <cl...@gmail.com> wrote:

>The filter should work while scanning for files - not after. eg if you
>configure the filter on the ftp endpoint. There is a bunch of options
>for filtering with include/exclude/ant/filter etc.
>
>As ftp extends file you can find all the options in the file docs
>http://camel.apache.org/file2
>
>On Thu, Nov 12, 2015 at 5:04 PM, Jack Frosch <ja...@gmail.com> wrote:
>> Our application consumes files from a folder on a supplier's FTP server.  Unfortunately, the supplier uses the same folder for documents intended to be consumed by other applications. We’d like to consume (i.e. read, then delete or move) our files which can be identified by file name pattern, leaving the other files we don’t care about in the folder.
>>
>> I was hoping a filter would do this, but it seems the filter gets applied after the file is consumed.
>>
>> I’m sure I’m overlooking something, but am Googled out. :-)
>>
>> Is there a way to configure the FTP endpoint using Camel 2.15.1 to only consume a subset of files from a folder on an FTP server?
>>
>> Thanks in advance!
>>
>> —
>> Jack
>>
>
>
>
>-- 
>Claus Ibsen
>-----------------
>http://davsclaus.com @davsclaus
>Camel in Action 2: https://www.manning.com/ibsen2

Re: How to consume selected files from FTP endpoint?

[Jack Frosch <jf...@objistics.com>]

Claus,

I found I could filter the files using the “include" option on the FTP end points. Thank you for reminding this Camel noob that FTP extends File.

For anyone interested:

Using the Java fluent DSL, I was using something like this:

  from(“ftp:server/someFolder&logonOptions&move=.done")
    .filter(predicateMatchingPrefixAndSomeSuffix)
    .process(myProcessor)
    .to(destinationEp)

That was consuming every file in someFolder, but only processing those with matching names. That led to my earlier question about filter unexpectedly being applied after all files were consumed from the FTP folder.

After doing some research into File and File2, I tried the “include" option:

  from(“ftp:server/someFolder&[logonOptions]&include=(?i)SomePrefix_.*_SomeSuffix.xml&move=.done”)
    .process(myProcessor)
    .to(destinationEp)


Using the include option as above, Camel only reads and moves files that case insensitively match the filename pattern.


I suppose I was thinking the ftp fetch was being lazily applied after the filter() processed the file names. (I'll blame it on my recently doing some functional programming with Java 8! :-)

Thanks for the help.

—
Jack Frosch






On 11/12/15, 10:08 AM, "Claus Ibsen" <cl...@gmail.com> wrote:

>The filter should work while scanning for files - not after. eg if you
>configure the filter on the ftp endpoint. There is a bunch of options
>for filtering with include/exclude/ant/filter etc.
>
>As ftp extends file you can find all the options in the file docs
>http://camel.apache.org/file2
>
>On Thu, Nov 12, 2015 at 5:04 PM, Jack Frosch <ja...@gmail.com> wrote:
>> Our application consumes files from a folder on a supplier's FTP server.  Unfortunately, the supplier uses the same folder for documents intended to be consumed by other applications. We’d like to consume (i.e. read, then delete or move) our files which can be identified by file name pattern, leaving the other files we don’t care about in the folder.
>>
>> I was hoping a filter would do this, but it seems the filter gets applied after the file is consumed.
>>
>> I’m sure I’m overlooking something, but am Googled out. :-)
>>
>> Is there a way to configure the FTP endpoint using Camel 2.15.1 to only consume a subset of files from a folder on an FTP server?
>>
>> Thanks in advance!
>>
>> —
>> Jack
>>
>
>
>
>-- 
>Claus Ibsen
>-----------------
>http://davsclaus.com @davsclaus
>Camel in Action 2: https://www.manning.com/ibsen2

Re: How to consume selected files from FTP endpoint?

[Claus Ibsen <cl...@gmail.com>]

The filter should work while scanning for files - not after. eg if you
configure the filter on the ftp endpoint. There is a bunch of options
for filtering with include/exclude/ant/filter etc.

As ftp extends file you can find all the options in the file docs
http://camel.apache.org/file2

On Thu, Nov 12, 2015 at 5:04 PM, Jack Frosch <ja...@gmail.com> wrote:
> Our application consumes files from a folder on a supplier's FTP server.  Unfortunately, the supplier uses the same folder for documents intended to be consumed by other applications. We’d like to consume (i.e. read, then delete or move) our files which can be identified by file name pattern, leaving the other files we don’t care about in the folder.
>
> I was hoping a filter would do this, but it seems the filter gets applied after the file is consumed.
>
> I’m sure I’m overlooking something, but am Googled out. :-)
>
> Is there a way to configure the FTP endpoint using Camel 2.15.1 to only consume a subset of files from a folder on an FTP server?
>
> Thanks in advance!
>
> —
> Jack
>



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2