You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Ole Andreas Hegle <he...@businesscape.no> on 2007/06/12 15:45:01 UTC
FilePolling
Hi
I am using Camel 1.0 (java 6) and have a file-component for polling
files from a folder.
However files only get polled when CamelContext is started, and not
later when new files are added to the folder I monitor.
I have configured it like this:
from("file:"+orderFolder).process(new OrderFileProcessor(context,
applicationEndpoint, adapter.getConfigurationFolder()));
(which when inserted becomes)
from("file:C:/folder").process(new Processor());
Does anybody know why files are only polled at startup, and not later?
Thanks in advance
-Ole Andreas
Re: FilePolling
Posted by James Strachan <ja...@gmail.com>.
On 6/15/07, Ole Andreas Hegle <he...@businesscape.no> wrote:
> Hi There
>
> I have discovered some issues with the file-component, but I need to
> clear up one thing here.
>
> Are there any reason why the ScheduledPollEndpoint use the
> "consumer."-prefix on options which go to the ScheduledPollConsumer?
> (initialDelay, delay and useFixedDelay).
> As far as I can se, no other endpoints/consumers use a
> "consumer."-prefix for sorting-attributes. Shall we keep the
> "consumer."-prefix, or shall we move the attributes from
> ScheduledPollConsumer to ScheduledPollEndpoint? What design guidelines
> do you prefer? (If we keep it in the consumer, we must update the
> user-manual on the web)
The reason I separated them out like that was in case the endpoint or
producer had separate properties. Though as an end user, I guess it
kinda sucks :).
The main issue is you create & validate the endpoint; then later on
you create & validate the consumer; which is why I kinda split them up
a bit (so that at each step we can validate the properties used).
It'd be cleaner for end users if we just had a flat properties thing;
but that'd make the possible validation code a tad more complex
(having to check that any remaining properties on an endpoint
parameter map really are destined for a producer/consumer).
If someone can figure out a neat way to solve that so we can add
properties for the producer, consumer or endpoint in a simple way that
also can validate; am really happy to trash the prefix stuff (it was
just the easiest way to get started :)
--
James
-------
http://macstrac.blogspot.com/
Re: FilePolling
Posted by Ole Andreas Hegle <he...@businesscape.no>.
Hi There
I have discovered some issues with the file-component, but I need to
clear up one thing here.
Are there any reason why the ScheduledPollEndpoint use the
"consumer."-prefix on options which go to the ScheduledPollConsumer?
(initialDelay, delay and useFixedDelay).
As far as I can se, no other endpoints/consumers use a
"consumer."-prefix for sorting-attributes. Shall we keep the
"consumer."-prefix, or shall we move the attributes from
ScheduledPollConsumer to ScheduledPollEndpoint? What design guidelines
do you prefer? (If we keep it in the consumer, we must update the
user-manual on the web)
Regards. Ole Andreas
James Strachan wrote:
> On 6/12/07, Ole Andreas Hegle <he...@businesscape.no> wrote:
>> Hi
>>
>> I need at least functionality to disable the last-modified vs. polltime
>> test, så I can create a fix for this.
>
> Awesome! :)
>
>> How/where do I submit the patch when it is ready?
>
> Here's details
>
> http://activemq.apache.org/camel/contributing.html
>
> a patch file is ideal; if you're having trouble making one, just
> attach the code to a JIRA and we can take it from there. Many thanks
> Ole!
>
Re: FilePolling
Posted by James Strachan <ja...@gmail.com>.
On 6/12/07, Ole Andreas Hegle <he...@businesscape.no> wrote:
> Hi
>
> I need at least functionality to disable the last-modified vs. polltime
> test, så I can create a fix for this.
Awesome! :)
> How/where do I submit the patch when it is ready?
Here's details
http://activemq.apache.org/camel/contributing.html
a patch file is ideal; if you're having trouble making one, just
attach the code to a JIRA and we can take it from there. Many thanks
Ole!
--
James
-------
http://macstrac.blogspot.com/
Re: FilePolling
Posted by Ole Andreas Hegle <he...@businesscape.no>.
Hi
I need at least functionality to disable the last-modified vs. polltime
test, så I can create a fix for this.
How/where do I submit the patch when it is ready?
-Ole Andreas
Hiram Chirino wrote:
> On 6/12/07, Ole Andreas Hegle <he...@businesscape.no> wrote:
>> Hi
>>
>> Yes, to make it optional would be nice.
>>
>> Other alternatives is to create an options for "delete processed files",
>> "move processed files" or to keep track of all files in the polled
>> folder to prevent files beeing polled twice, unless they are modified.
>>
>
> Those 3 strategies make lots of sense.. want to take a stab at
> implementing them ;-)
>
>> -Ole Andreas
>>
>>
>> James Strachan wrote:
>> > On 6/12/07, Rob Davies <ra...@gmail.com> wrote:
>> >> ooh - school boy error - want to issue a patch? - if not I'll try and
>> >> fix
>> >
>> > Should we just trash the lastModified > lastPollTime test? Or make it
>> > optional?
>> >
>> > James
>> >
>> >> On Jun 12, 2007, at 5:02 PM, Ole Andreas Hegle wrote:
>> >>
>> >> > Hi James
>> >> >
>> >> > Thank you for your answer.
>> >> >
>> >> > I have just found the problem, the file-poller worked when i
>> >> > modified the file, and it seems to me as the problem is located in
>> >> > org.apache.camel.component.file.FileConsumer.java and the method
>> >> >
>> >> > *protected* *void* pollFile(*final* File file) {
>> >> > *if* (file.exists() && file.lastModified() > lastPollTime) {
>> >> > *if* (isValidFile(file)) {
>> >> > processFile(file);
>> >> > }
>> >> > }
>> >> > }
>> >> >
>> >> >
>> >> > My files are sent as attachment to a receiver by email, and then
>> >> > imported to the system by saving the email-attachment in the folder
>> >> > monitored by the FileConsumer. The modified-timestamp will be older
>> >> > than the lastPollTime-timestamp, and because of this files will not
>> >> > be polled. Are there any way to dissable the
>> file.lastModified-method?
>> >> > (I move the files to an archive folder when they are processed)
>> >> >
>> >> > Perhaps I can have another component running and modifying the
>> >> > files to change the last modified-timestamp on the files to make
>> >> > them pollable, but I would rather not...
>> >> >
>> >> > Anybody knows what I can do with this?
>> >> >
>> >> > Regards Ole Andreas
>> >> >
>> >>
>> >>
>> >
>> >
>>
>>
>
>
Re: FilePolling
Posted by Hiram Chirino <hi...@hiramchirino.com>.
On 6/12/07, Ole Andreas Hegle <he...@businesscape.no> wrote:
> Hi
>
> Yes, to make it optional would be nice.
>
> Other alternatives is to create an options for "delete processed files",
> "move processed files" or to keep track of all files in the polled
> folder to prevent files beeing polled twice, unless they are modified.
>
Those 3 strategies make lots of sense.. want to take a stab at
implementing them ;-)
> -Ole Andreas
>
>
> James Strachan wrote:
> > On 6/12/07, Rob Davies <ra...@gmail.com> wrote:
> >> ooh - school boy error - want to issue a patch? - if not I'll try and
> >> fix
> >
> > Should we just trash the lastModified > lastPollTime test? Or make it
> > optional?
> >
> > James
> >
> >> On Jun 12, 2007, at 5:02 PM, Ole Andreas Hegle wrote:
> >>
> >> > Hi James
> >> >
> >> > Thank you for your answer.
> >> >
> >> > I have just found the problem, the file-poller worked when i
> >> > modified the file, and it seems to me as the problem is located in
> >> > org.apache.camel.component.file.FileConsumer.java and the method
> >> >
> >> > *protected* *void* pollFile(*final* File file) {
> >> > *if* (file.exists() && file.lastModified() > lastPollTime) {
> >> > *if* (isValidFile(file)) {
> >> > processFile(file);
> >> > }
> >> > }
> >> > }
> >> >
> >> >
> >> > My files are sent as attachment to a receiver by email, and then
> >> > imported to the system by saving the email-attachment in the folder
> >> > monitored by the FileConsumer. The modified-timestamp will be older
> >> > than the lastPollTime-timestamp, and because of this files will not
> >> > be polled. Are there any way to dissable the file.lastModified-method?
> >> > (I move the files to an archive folder when they are processed)
> >> >
> >> > Perhaps I can have another component running and modifying the
> >> > files to change the last modified-timestamp on the files to make
> >> > them pollable, but I would rather not...
> >> >
> >> > Anybody knows what I can do with this?
> >> >
> >> > Regards Ole Andreas
> >> >
> >>
> >>
> >
> >
>
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
Re: FilePolling
Posted by Ole Andreas Hegle <he...@businesscape.no>.
Hi
Yes, to make it optional would be nice.
Other alternatives is to create an options for "delete processed files",
"move processed files" or to keep track of all files in the polled
folder to prevent files beeing polled twice, unless they are modified.
-Ole Andreas
James Strachan wrote:
> On 6/12/07, Rob Davies <ra...@gmail.com> wrote:
>> ooh - school boy error - want to issue a patch? - if not I'll try and
>> fix
>
> Should we just trash the lastModified > lastPollTime test? Or make it
> optional?
>
> James
>
>> On Jun 12, 2007, at 5:02 PM, Ole Andreas Hegle wrote:
>>
>> > Hi James
>> >
>> > Thank you for your answer.
>> >
>> > I have just found the problem, the file-poller worked when i
>> > modified the file, and it seems to me as the problem is located in
>> > org.apache.camel.component.file.FileConsumer.java and the method
>> >
>> > *protected* *void* pollFile(*final* File file) {
>> > *if* (file.exists() && file.lastModified() > lastPollTime) {
>> > *if* (isValidFile(file)) {
>> > processFile(file);
>> > }
>> > }
>> > }
>> >
>> >
>> > My files are sent as attachment to a receiver by email, and then
>> > imported to the system by saving the email-attachment in the folder
>> > monitored by the FileConsumer. The modified-timestamp will be older
>> > than the lastPollTime-timestamp, and because of this files will not
>> > be polled. Are there any way to dissable the file.lastModified-method?
>> > (I move the files to an archive folder when they are processed)
>> >
>> > Perhaps I can have another component running and modifying the
>> > files to change the last modified-timestamp on the files to make
>> > them pollable, but I would rather not...
>> >
>> > Anybody knows what I can do with this?
>> >
>> > Regards Ole Andreas
>> >
>>
>>
>
>
Re: FilePolling
Posted by James Strachan <ja...@gmail.com>.
On 6/12/07, Rob Davies <ra...@gmail.com> wrote:
> ooh - school boy error - want to issue a patch? - if not I'll try and
> fix
Should we just trash the lastModified > lastPollTime test? Or make it optional?
James
> On Jun 12, 2007, at 5:02 PM, Ole Andreas Hegle wrote:
>
> > Hi James
> >
> > Thank you for your answer.
> >
> > I have just found the problem, the file-poller worked when i
> > modified the file, and it seems to me as the problem is located in
> > org.apache.camel.component.file.FileConsumer.java and the method
> >
> > *protected* *void* pollFile(*final* File file) {
> > *if* (file.exists() && file.lastModified() > lastPollTime) {
> > *if* (isValidFile(file)) {
> > processFile(file);
> > }
> > }
> > }
> >
> >
> > My files are sent as attachment to a receiver by email, and then
> > imported to the system by saving the email-attachment in the folder
> > monitored by the FileConsumer. The modified-timestamp will be older
> > than the lastPollTime-timestamp, and because of this files will not
> > be polled. Are there any way to dissable the file.lastModified-method?
> > (I move the files to an archive folder when they are processed)
> >
> > Perhaps I can have another component running and modifying the
> > files to change the last modified-timestamp on the files to make
> > them pollable, but I would rather not...
> >
> > Anybody knows what I can do with this?
> >
> > Regards Ole Andreas
> >
>
>
--
James
-------
http://macstrac.blogspot.com/
Re: FilePolling
Posted by Rob Davies <ra...@gmail.com>.
ooh - school boy error - want to issue a patch? - if not I'll try and
fix
On Jun 12, 2007, at 5:02 PM, Ole Andreas Hegle wrote:
> Hi James
>
> Thank you for your answer.
>
> I have just found the problem, the file-poller worked when i
> modified the file, and it seems to me as the problem is located in
> org.apache.camel.component.file.FileConsumer.java and the method
>
> *protected* *void* pollFile(*final* File file) {
> *if* (file.exists() && file.lastModified() > lastPollTime) {
> *if* (isValidFile(file)) {
> processFile(file);
> }
> }
> }
>
>
> My files are sent as attachment to a receiver by email, and then
> imported to the system by saving the email-attachment in the folder
> monitored by the FileConsumer. The modified-timestamp will be older
> than the lastPollTime-timestamp, and because of this files will not
> be polled. Are there any way to dissable the file.lastModified-method?
> (I move the files to an archive folder when they are processed)
>
> Perhaps I can have another component running and modifying the
> files to change the last modified-timestamp on the files to make
> them pollable, but I would rather not...
>
> Anybody knows what I can do with this?
>
> Regards Ole Andreas
>
Re: FilePolling
Posted by Ole Andreas Hegle <he...@businesscape.no>.
Hi James
Thank you for your answer.
I have just found the problem, the file-poller worked when i modified
the file, and it seems to me as the problem is located in
org.apache.camel.component.file.FileConsumer.java and the method
*protected* *void* pollFile(*final* File file) {
*if* (file.exists() && file.lastModified() > lastPollTime) {
*if* (isValidFile(file)) {
processFile(file);
}
}
}
My files are sent as attachment to a receiver by email, and then
imported to the system by saving the email-attachment in the folder
monitored by the FileConsumer. The modified-timestamp will be older than
the lastPollTime-timestamp, and because of this files will not be
polled. Are there any way to dissable the file.lastModified-method?
(I move the files to an archive folder when they are processed)
Perhaps I can have another component running and modifying the files to
change the last modified-timestamp on the files to make them pollable,
but I would rather not...
Anybody knows what I can do with this?
Regards Ole Andreas
Re: FilePolling
Posted by James Strachan <ja...@gmail.com>.
On 6/12/07, Ole Andreas Hegle <he...@businesscape.no> wrote:
> Hi
Hi Ole!
> I am using Camel 1.0 (java 6) and have a file-component for polling
> files from a folder.
>
> However files only get polled when CamelContext is started, and not
> later when new files are added to the folder I monitor.
You start the context and don't stop it right?
> I have configured it like this:
> from("file:"+orderFolder).process(new OrderFileProcessor(context,
> applicationEndpoint, adapter.getConfigurationFolder()));
>
> (which when inserted becomes)
> from("file:C:/folder").process(new Processor());
>
> Does anybody know why files are only polled at startup, and not later?
The file consumer derived from the ScheduledPollConsumer which should
perform repeated timed polls (which are configurable on the endpoint
URI). Maybe if debug logging is enabled you'll see if the directory is
repeatedly pooled or not?
I guess a good test case could be for a route to be started, then a
new directory to be created after the first poll etc.
--
James
-------
http://macstrac.blogspot.com/