You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Marek2009 <Ma...@icw.de> on 2009/06/19 17:49:22 UTC

Re: "Can not delete file" when combining file poller and JMS

Hello Klaus, Adrian, Bruno,

although the workaround you suggest is trivial, it took us a while to
implement and test it. :-)

I would like to confirm that inserting a "convertBodyTo(String.class)"
worked for us (Camel 1.6.0). 
Since the modification we have not encountered the file access issue in any
of our tests, including load tests with larger numbers of files.

We hope to be able to remove the workaround code from the route, as soon as
we upgrade to Camel 1.6.1 or later. :-)

Many thanks for your help!
Kind Regards

Marek


Claus Ibsen-2 wrote:
> 
> Hi
> 
> Yeah I have tracked it down to a potential buffer stream not being
> closed, that is used by the JMS consumer.
> Converting the file content to String based beforehand should fix it
> as Ade suggested.
> 
> However the bugfix is committed to 1.6.1 and 2.0 so hopefully it
> should work out of the box in the future.
> 
> 
> On Wed, Apr 8, 2009 at 4:54 PM, Adrian Trenaman <tr...@progress.com>
> wrote:
>> Hi Bruno, Marek,
>>
>> Yeah, I encountered someting similar with Camel 1.5.4.0-fuse on Windows:
>> if
>> you convertBodyTo(String.class) after the from("file:...") then it should
>> be
>> ok.
>>
>> /Ade
>> On 8 Apr 2009, at 15:50, bruno.borges@gmail.com wrote:
>>
>>> Have you tried to put a direct:endpoint in the middle, between file:*
>>> and
>>> jms:* ?
>>>
>>> Before routing to direct:, convert body to String. Then you route
>>> from("direct:endpoint").to("jms:queue")
>>>
>>> Maybe this works.
>>>
>>> Cheers,
>>> Bruno Borges
>>>
>>> On Apr 8, 2009 11:39am, Marek2009 <Ma...@icw.de> wrote:
>>>
>>>
>>>> Hello,
>>>
>>>
>>>
>>>> I have encountered an issue when combining file poller and JMS queue in
>>>> one
>>>
>>>> route. If the poller is set to "noop=no" and tries to either delete or
>>>> move
>>>
>>>> the files, it fails every now and then (say, every 5th to 10th file).
>>>> For
>>>
>>>> instance, with "noop=no&delete=yes" on a Windows machine the exception
>>>
>>>> thrown is: "2009-03-27T17:13:59,781 [Thread: 1
>>>
>>>> org.apache.camel.component.file.FileComponent@1af5350] WARN -
>>>
>>>> component.file.FileConsumer | Error committing file strategy:
>>>
>>>>
>>>> org.apache.camel.component.file.strategy.DeleteFileProcessStrategy@10c81a6
>>>
>>>> java.io.IOException: Can not delete file:
>>>> .\data\pxsa\transfer\in\file.txt"
>>>
>>>
>>>
>>>> Apparently, the issue has been discussed in in various Camel forums:
>>>
>>>> - http://osdir.com/ml/users-camel-apache/2009-02/msg00119.html
>>>
>>>> - http://osdir.com/ml/users-camel-apache/2009-02/msg00123.html
>>>
>>>> -
>>>
>>>>
>>>> http://mail-archives.apache.org/mod_mbox/camel-users/200902.mbox/%3C5380c69c0902130209l679c3ab0j641036484ee14875@mail.gmail.com%3E
>>>
>>>> - http://cwiki.apache.org/SM/discussion-forums.html#nabble-p21504030
>>>
>>>
>>>
>>>> Unfortunately, none of the solutions suggested there (using
>>>> delete=true,
>>>
>>>> converting body to string) worked for us 100%. Adding a delay reduced
>>>> the
>>>
>>>> occurence of the exception, though. Setting
>>>
>>>> "consumer.exclusiveReadLock=false" is no option for us, as we want to
>>>
>>>> prevent incomplete files from being read.
>>>
>>>
>>>
>>>> Removing the JMS endpoint from the route in our application prevents
>>>> the
>>>
>>>> error, which indicates that the cause is connected to JMS. This is also
>>>> our
>>>
>>>> workaround at the moment.
>>>
>>>
>>>
>>>> I wonder whether
>>>
>>>> a) the problem is familiar to somebody?
>>>
>>>> b) there is a "trick", an endpoint setting or an addition to the route,
>>>> that
>>>
>>>> reliably eliminates the problem?
>>>
>>>
>>>
>>>> Any help is greatly appreciated.
>>>
>>>
>>>
>>>> Kind Regards
>>>
>>>> Marek
>>>
>>>
>>>
>>>> --
>>>
>>>> View this message in context:
>>>> http://www.nabble.com/%22Can-not-delete-file%22-when-combining-file-poller-and-JMS-tp22951942p22951942.html
>>>
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>>
>>
>> ---
>> Adrian Trenaman, Consultant Fellow, PS - Opensource Center of Competence
>> Progress Software Corp
>> Shelbourne Road, Dublin 4, Ireland
>> ---
>> +353-1-637-2659 (Office)
>> +353-1-637-2882 (Fax)
>> +353-86-6051026 (Mobile)
>>  adrian.trenaman (Skype)
>> ----
>> Blog: http://trenaman.blogspot.com
>>
>>
>>
>>
>>
>>
>>
>>
>>
> 
> 
> 
> -- 
> Claus Ibsen
> Apache Camel Committer
> 
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
> Apache Camel Reference Card:
> http://refcardz.dzone.com/refcardz/enterprise-integration
> 
> 

-- 
View this message in context: http://www.nabble.com/%22Can-not-delete-file%22-when-combining-file-poller-and-JMS-tp22951942p24114062.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: "Can not delete file" when combining file poller and JMS

Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jun 19, 2009 at 5:49 PM, Marek2009 <Ma...@icw.de> wrote:

>
> Hello Klaus, Adrian, Bruno,
>
> although the workaround you suggest is trivial, it took us a while to
> implement and test it. :-)
>
> I would like to confirm that inserting a "convertBodyTo(String.class)"
> worked for us (Camel 1.6.0).
> Since the modification we have not encountered the file access issue in any
> of our tests, including load tests with larger numbers of files.
>
> We hope to be able to remove the workaround code from the route, as soon as
> we upgrade to Camel 1.6.1 or later. :-)
>

Hi Marek

Thanks for sharing this. Nice to know the workaround worked.
Fingers crossed that its fixed in 1.6.1 with that missing close().




>
> Many thanks for your help!
> Kind Regards
>
> Marek
>
>
> Claus Ibsen-2 wrote:
> >
> > Hi
> >
> > Yeah I have tracked it down to a potential buffer stream not being
> > closed, that is used by the JMS consumer.
> > Converting the file content to String based beforehand should fix it
> > as Ade suggested.
> >
> > However the bugfix is committed to 1.6.1 and 2.0 so hopefully it
> > should work out of the box in the future.
> >
> >
> > On Wed, Apr 8, 2009 at 4:54 PM, Adrian Trenaman <tr...@progress.com>
> > wrote:
> >> Hi Bruno, Marek,
> >>
> >> Yeah, I encountered someting similar with Camel 1.5.4.0-fuse on Windows:
> >> if
> >> you convertBodyTo(String.class) after the from("file:...") then it
> should
> >> be
> >> ok.
> >>
> >> /Ade
> >> On 8 Apr 2009, at 15:50, bruno.borges@gmail.com wrote:
> >>
> >>> Have you tried to put a direct:endpoint in the middle, between file:*
> >>> and
> >>> jms:* ?
> >>>
> >>> Before routing to direct:, convert body to String. Then you route
> >>> from("direct:endpoint").to("jms:queue")
> >>>
> >>> Maybe this works.
> >>>
> >>> Cheers,
> >>> Bruno Borges
> >>>
> >>> On Apr 8, 2009 11:39am, Marek2009 <Ma...@icw.de> wrote:
> >>>
> >>>
> >>>> Hello,
> >>>
> >>>
> >>>
> >>>> I have encountered an issue when combining file poller and JMS queue
> in
> >>>> one
> >>>
> >>>> route. If the poller is set to "noop=no" and tries to either delete or
> >>>> move
> >>>
> >>>> the files, it fails every now and then (say, every 5th to 10th file).
> >>>> For
> >>>
> >>>> instance, with "noop=no&delete=yes" on a Windows machine the exception
> >>>
> >>>> thrown is: "2009-03-27T17:13:59,781 [Thread: 1
> >>>
> >>>> org.apache.camel.component.file.FileComponent@1af5350] WARN -
> >>>
> >>>> component.file.FileConsumer | Error committing file strategy:
> >>>
> >>>>
> >>>>
> org.apache.camel.component.file.strategy.DeleteFileProcessStrategy@10c81a6
> >>>
> >>>> java.io.IOException: Can not delete file:
> >>>> .\data\pxsa\transfer\in\file.txt"
> >>>
> >>>
> >>>
> >>>> Apparently, the issue has been discussed in in various Camel forums:
> >>>
> >>>> - http://osdir.com/ml/users-camel-apache/2009-02/msg00119.html
> >>>
> >>>> - http://osdir.com/ml/users-camel-apache/2009-02/msg00123.html
> >>>
> >>>> -
> >>>
> >>>>
> >>>>
> http://mail-archives.apache.org/mod_mbox/camel-users/200902.mbox/%3C5380c69c0902130209l679c3ab0j641036484ee14875@mail.gmail.com%3E
> >>>
> >>>> - http://cwiki.apache.org/SM/discussion-forums.html#nabble-p21504030
> >>>
> >>>
> >>>
> >>>> Unfortunately, none of the solutions suggested there (using
> >>>> delete=true,
> >>>
> >>>> converting body to string) worked for us 100%. Adding a delay reduced
> >>>> the
> >>>
> >>>> occurence of the exception, though. Setting
> >>>
> >>>> "consumer.exclusiveReadLock=false" is no option for us, as we want to
> >>>
> >>>> prevent incomplete files from being read.
> >>>
> >>>
> >>>
> >>>> Removing the JMS endpoint from the route in our application prevents
> >>>> the
> >>>
> >>>> error, which indicates that the cause is connected to JMS. This is
> also
> >>>> our
> >>>
> >>>> workaround at the moment.
> >>>
> >>>
> >>>
> >>>> I wonder whether
> >>>
> >>>> a) the problem is familiar to somebody?
> >>>
> >>>> b) there is a "trick", an endpoint setting or an addition to the
> route,
> >>>> that
> >>>
> >>>> reliably eliminates the problem?
> >>>
> >>>
> >>>
> >>>> Any help is greatly appreciated.
> >>>
> >>>
> >>>
> >>>> Kind Regards
> >>>
> >>>> Marek
> >>>
> >>>
> >>>
> >>>> --
> >>>
> >>>> View this message in context:
> >>>>
> http://www.nabble.com/%22Can-not-delete-file%22-when-combining-file-poller-and-JMS-tp22951942p22951942.html
> >>>
> >>>> Sent from the Camel - Users mailing list archive at Nabble.com.
> >>>
> >>>
> >>>
> >>
> >> ---
> >> Adrian Trenaman, Consultant Fellow, PS - Opensource Center of Competence
> >> Progress Software Corp
> >> Shelbourne Road, Dublin 4, Ireland
> >> ---
> >> +353-1-637-2659 (Office)
> >> +353-1-637-2882 (Fax)
> >> +353-86-6051026 (Mobile)
> >>  adrian.trenaman (Skype)
> >> ----
> >> Blog: http://trenaman.blogspot.com
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> >
> > --
> > Claus Ibsen
> > Apache Camel Committer
> >
> > Open Source Integration: http://fusesource.com
> > Blog: http://davsclaus.blogspot.com/
> > Twitter: http://twitter.com/davsclaus
> > Apache Camel Reference Card:
> > http://refcardz.dzone.com/refcardz/enterprise-integration
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/%22Can-not-delete-file%22-when-combining-file-poller-and-JMS-tp22951942p24114062.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>


-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus