You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by ychawla <pr...@yahoo.com> on 2008/12/17 19:44:25 UTC
Poll a collection of subfolders with a single endpoint
Hello All,
I have a well defined file structure where I am going to be picking up files
from. It looks like this:
Mailboxes (STATIC)
Mailbox1 (dynamic)
dropOffFolder (static)
Mailbox2 (dynamic)
dropOffFolder (static)
Currently I have endpoints defined like this:
<endpoint id="mailbox1" uri="file:/mailboxes/Mailbox1/dropOffFolder"/>
<endpoint id="mailbox1" uri="file:/mailboxes/Mailbox2/dropOffFolder"/>
Would it be possible through File Language, simple or another mechanism to
have a single endpoint with * being a wilcard:
<endpoint id="mailboxes" uri="file:/mailboxes/*/dropOffFolder"/>
I checked the documentation and I couldn't find anything.
Thanks!
Yogesh
--
View this message in context: http://www.nabble.com/Poll-a-collection-of-subfolders-with-a-single-endpoint-tp21058872s22882p21058872.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Poll a collection of subfolders with a single endpoint
Posted by Ramon Buckland <ra...@gmail.com>.
>
>
> Feel free to add a ticket for it.
>
Free I feel - https://issues.apache.org/activemq/browse/CAMEL-1198
Re: Poll a collection of subfolders with a single endpoint
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Dec 19, 2008 at 10:50 AM, Ramon Buckland
<ra...@gmail.com> wrote:
>>
>> c)
>> I have been considering supporting the ant path style
>> (**/dropOffFolder), that you have in spring resource.
>> But I didn't want to write an ant path style parser and camel-core
>> should not require spring.
>> But that would be cool
>>
>>
> Depend on the ant jar :-)
> We use the DirScanner quite a lot in code, because it just works really
> well.
> The ability to have two "selectors" would be good.
>
> **/somedir/**/fileprefix*.csv,**/someother/dir/filepre*.txt
>
> A snippet of ANT code doing this is simply...
>
> DirectoryScanner dirScanner = new DirectoryScanner();
> dirScanner.setBasedir(myBaseDir);
> dirScanner.setIncludes(includes); // String[]
> dirScanner.setExcludes(excludes);
> ...
> dirScanner.scan();
> ..
> // work with files in dirScanner.getIncludedFiles()
>
> simple :-)
Pretty cool.
I guess we should support a pluggable strategy for scanning
files/dirs. Then we can have deps on ANT (in camel-???) or spring in
camel-spring.
Yeah that is cool.
Feel free to add a ticket for it.
>
> r.
>
--
/Claus Ibsen
Apache Camel Committer
Blog: http://davsclaus.blogspot.com/
Re: Poll a collection of subfolders with a single endpoint
Posted by Ramon Buckland <ra...@gmail.com>.
>
> c)
> I have been considering supporting the ant path style
> (**/dropOffFolder), that you have in spring resource.
> But I didn't want to write an ant path style parser and camel-core
> should not require spring.
> But that would be cool
>
>
Depend on the ant jar :-)
We use the DirScanner quite a lot in code, because it just works really
well.
The ability to have two "selectors" would be good.
**/somedir/**/fileprefix*.csv,**/someother/dir/filepre*.txt
A snippet of ANT code doing this is simply...
DirectoryScanner dirScanner = new DirectoryScanner();
dirScanner.setBasedir(myBaseDir);
dirScanner.setIncludes(includes); // String[]
dirScanner.setExcludes(excludes);
...
dirScanner.scan();
..
// work with files in dirScanner.getIncludedFiles()
simple :-)
r.
Re: Poll a collection of subfolders with a single endpoint
Posted by Claus Ibsen <cl...@gmail.com>.
And if you mailbox files has a special extension then using the
recursive option and the regexPattern to set the pattern for the file
extensions to include/match.
But we could also add option for
- includedNamePrefix
- includedNamePostfix
But yet again then there might be to many options and too confusing
how to use the file component.
Any feedback on it is highly appreciated. As we would like to get it
improved in Camel 2.0.
On Wed, Dec 17, 2008 at 8:49 PM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> a)
> You can use the recursive=true option and Camel will walk all sub
> folders. If that is sufficient
> <endpoint id="mailboxes" uri="file:/mailboxe/?consumer.recurisve=true"/>
>
> b)
> If you need more fine grained control of which folders / files to
> consume there are several options
> - excludedNamePrefix
> - excludedNamePostfix
> - consumer.regexPattern
>
> But they only operates on the java.io.File.getName() so you wont have
> the path to test from.
> I guess that is a limitation sometimes and we could add options for
> the File.getPath() so you have both combos
> - consumer.pathRegexPattern
>
> But yet again the file component has many options now and it can get a
> bit confusing.
>
> c)
> I have been considering supporting the ant path style
> (**/dropOffFolder), that you have in spring resource.
> But I didn't want to write an ant path style parser and camel-core
> should not require spring.
> But that would be cool
>
> d)
> In Camel 2.0 we have support for setting a java.io.FileFilter where
> you can do the test in java code, and thus it would be possible.
>
> e)
> File language is meant for the file producer to set the filename, so
> you can not currently use it for filtering which files/folders should
> be included in the file consumer. But I guess we could do filtering
> using expressions, but we dont have nice support for this in the DSL
> as from is generic.
>
>
> Any thoughts?
>
>
>
> On Wed, Dec 17, 2008 at 7:44 PM, ychawla <pr...@yahoo.com> wrote:
>>
>> Hello All,
>> I have a well defined file structure where I am going to be picking up files
>> from. It looks like this:
>>
>> Mailboxes (STATIC)
>> Mailbox1 (dynamic)
>> dropOffFolder (static)
>> Mailbox2 (dynamic)
>> dropOffFolder (static)
>>
>> Currently I have endpoints defined like this:
>> <endpoint id="mailbox1" uri="file:/mailboxes/Mailbox1/dropOffFolder"/>
>> <endpoint id="mailbox1" uri="file:/mailboxes/Mailbox2/dropOffFolder"/>
>>
>> Would it be possible through File Language, simple or another mechanism to
>> have a single endpoint with * being a wilcard:
>>
>> <endpoint id="mailboxes" uri="file:/mailboxes/*/dropOffFolder"/>
>>
>> I checked the documentation and I couldn't find anything.
>>
>> Thanks!
>>
>> Yogesh
>> --
>> View this message in context: http://www.nabble.com/Poll-a-collection-of-subfolders-with-a-single-endpoint-tp21058872s22882p21058872.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
>
> /Claus Ibsen
> Apache Camel Committer
> Blog: http://davsclaus.blogspot.com/
>
--
/Claus Ibsen
Apache Camel Committer
Blog: http://davsclaus.blogspot.com/
Re: Poll a collection of subfolders with a single endpoint
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
a)
You can use the recursive=true option and Camel will walk all sub
folders. If that is sufficient
<endpoint id="mailboxes" uri="file:/mailboxe/?consumer.recurisve=true"/>
b)
If you need more fine grained control of which folders / files to
consume there are several options
- excludedNamePrefix
- excludedNamePostfix
- consumer.regexPattern
But they only operates on the java.io.File.getName() so you wont have
the path to test from.
I guess that is a limitation sometimes and we could add options for
the File.getPath() so you have both combos
- consumer.pathRegexPattern
But yet again the file component has many options now and it can get a
bit confusing.
c)
I have been considering supporting the ant path style
(**/dropOffFolder), that you have in spring resource.
But I didn't want to write an ant path style parser and camel-core
should not require spring.
But that would be cool
d)
In Camel 2.0 we have support for setting a java.io.FileFilter where
you can do the test in java code, and thus it would be possible.
e)
File language is meant for the file producer to set the filename, so
you can not currently use it for filtering which files/folders should
be included in the file consumer. But I guess we could do filtering
using expressions, but we dont have nice support for this in the DSL
as from is generic.
Any thoughts?
On Wed, Dec 17, 2008 at 7:44 PM, ychawla <pr...@yahoo.com> wrote:
>
> Hello All,
> I have a well defined file structure where I am going to be picking up files
> from. It looks like this:
>
> Mailboxes (STATIC)
> Mailbox1 (dynamic)
> dropOffFolder (static)
> Mailbox2 (dynamic)
> dropOffFolder (static)
>
> Currently I have endpoints defined like this:
> <endpoint id="mailbox1" uri="file:/mailboxes/Mailbox1/dropOffFolder"/>
> <endpoint id="mailbox1" uri="file:/mailboxes/Mailbox2/dropOffFolder"/>
>
> Would it be possible through File Language, simple or another mechanism to
> have a single endpoint with * being a wilcard:
>
> <endpoint id="mailboxes" uri="file:/mailboxes/*/dropOffFolder"/>
>
> I checked the documentation and I couldn't find anything.
>
> Thanks!
>
> Yogesh
> --
> View this message in context: http://www.nabble.com/Poll-a-collection-of-subfolders-with-a-single-endpoint-tp21058872s22882p21058872.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
/Claus Ibsen
Apache Camel Committer
Blog: http://davsclaus.blogspot.com/