You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Claus Ibsen <cl...@gmail.com> on 2009/01/30 07:44:09 UTC
[HEADS UP] - File component changes in Camel 2.0
Hi
As some of you know the file component have had a major refactor ...
actually you can nearly consider it as a rewrite in Camel 2.0.
This mail is about a few remaining issues I want to give a heads up
upon and feedback:
Force a filename to be provided when wring a file
====================================
I want to force file producer always requiring a header value with the
filename to write.
What we have in Camel 1.x is that if no filename header is provided it
will fallback to use the message id as the filename.
For me that has no use, as its kinda like telling a database here is
some data store it somewhere, without providing, schema, table, column
names.
So I want it to reject writing a file and report an exception that the
filename is missing.
The file language supports you if you want to use the message id as
the filename. Just set the header value as: ${id}
And also remove option: ignoreFileNameHeader
Thoughts?
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: [HEADS UP] - File component changes in Camel 2.0
Posted by Jon Anstey <ja...@gmail.com>.
Yeah, great work on all the improvements to the file component Claus. I
think it will warrant a blog post of its own when Camel 2.0 comes out :)
Not sure about this suggestion though... I kinda like having defaults in
place so you don't always have to provide configuration. Like, having a
default file name generated is great for testing purposes or for new users
just trying Camel out. Also maybe the file component may be used to drop
files off for processing by some other application which may not care about
the filename, just the contents.
Well, I don't have any big reasons to not go forward with this... these are
just my pre-coffee thoughts :)
Also I'm fine with removing ignoreFileNameHeader.
On Fri, Jan 30, 2009 at 3:14 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> As some of you know the file component have had a major refactor ...
> actually you can nearly consider it as a rewrite in Camel 2.0.
>
> This mail is about a few remaining issues I want to give a heads up
> upon and feedback:
>
>
> Force a filename to be provided when wring a file
> ====================================
>
> I want to force file producer always requiring a header value with the
> filename to write.
> What we have in Camel 1.x is that if no filename header is provided it
> will fallback to use the message id as the filename.
>
> For me that has no use, as its kinda like telling a database here is
> some data store it somewhere, without providing, schema, table, column
> names.
>
> So I want it to reject writing a file and report an exception that the
> filename is missing.
>
> The file language supports you if you want to use the message id as
> the filename. Just set the header value as: ${id}
>
> And also remove option: ignoreFileNameHeader
>
>
> Thoughts?
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
>
--
Cheers,
Jon
http://janstey.blogspot.com/
Re: [HEADS UP] - File component changes in Camel 2.0
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Yeah please take a look at the GenericFileExclusiveReadLockStrategy
and see if you can implement it as a implementation of this interface?
Then it can be added into camel 2.0 as an other option to choose.
But beware that the code in trunk is currently have 2 file components.
FileComponent is the old one that is currently active
NewFileComponent is the new one temporary there side by side until its
rock solid and can replace the FileComponent.
The trick is that you can modify the file component file
Index: src/main/resources/META-INF/services/org/apache/camel/component/file
===================================================================
--- src/main/resources/META-INF/services/org/apache/camel/component/file
(revision 739943)
+++ src/main/resources/META-INF/services/org/apache/camel/component/file
(working copy)
@@ -15,5 +15,5 @@
# limitations under the License.
#
-class=org.apache.camel.component.file.FileComponent
-strategy.factory.class=org.apache.camel.component.file.strategy.FileProcessStrategyFactory
\ No newline at end of file
+class=org.apache.camel.component.file.NewFileComponent
+strategy.factory.class=org.apache.camel.component.file.strategy.NewFileProcessStrategyFactory
\ No newline at end of file
Where you just use the NewFileComponent and the New factory.
On Mon, Feb 2, 2009 at 3:54 PM, Aaron Crickenberger
<aa...@intalgent.com> wrote:
> On a slightly related tangent, I'm partly responsible for the hackish code
> in FileConsumer's (old) isUnchanged() method (
> https://issues.apache.org/activemq/browse/CAMEL-250).
>
> Camel 1.5's exclusiveReadLock feature is unfortunately still picking up
> files too early, so I have to fall back to the deprecated behavior. This
> happens regardless of the file processing strategy (delete, rename, etc.)
> Is there any chance the hackish double-polling stuff can be put back in to
> 2.0 as a non-default file locking strategy? I can take a crack at it, I
> just wasn't sure if the code was still in flux.
>
> - aaron
>
> On Mon, Feb 2, 2009 at 4:44 AM, Claus Ibsen <cl...@gmail.com> wrote:
>
>> On Fri, Jan 30, 2009 at 5:40 PM, Aaron Crickenberger
>> <aa...@intalgent.com> wrote:
>> > I would hesitate if only because requiring a particular header seems off,
>> > are there other components that do the same?
>> >
>> > I haven't looked much at Camel 2.0's code, but it looks like camel-1.x's
>> > file component's "expression" property could support both scenarios. Use
>> a
>> > default ${id} expression, but allow user to configure w/ a ${in.header}
>> > expression that barks if the header's not present, no?
>> That is correct.
>>
>> Thanks for the feedback. We will leave it as is.
>>
>> >
>> > - aaron
>> >
>> > On Fri, Jan 30, 2009 at 9:49 AM, James Strachan <
>> james.strachan@gmail.com>wrote:
>> >
>> >> So long as files are written in some directory and they are unique and
>> >> preserve order - does it matter if they are generated (from the
>> >> message ID say) or the user explicitly gives some name? Like Jon I'd
>> >> be tempted ot leave the default behaviour?
>> >>
>> >> 2009/1/30 Claus Ibsen <cl...@gmail.com>:
>> >> > Hi
>> >> >
>> >> > As some of you know the file component have had a major refactor ...
>> >> > actually you can nearly consider it as a rewrite in Camel 2.0.
>> >> >
>> >> > This mail is about a few remaining issues I want to give a heads up
>> >> > upon and feedback:
>> >> >
>> >> >
>> >> > Force a filename to be provided when wring a file
>> >> > ====================================
>> >> >
>> >> > I want to force file producer always requiring a header value with the
>> >> > filename to write.
>> >> > What we have in Camel 1.x is that if no filename header is provided it
>> >> > will fallback to use the message id as the filename.
>> >> >
>> >> > For me that has no use, as its kinda like telling a database here is
>> >> > some data store it somewhere, without providing, schema, table, column
>> >> > names.
>> >> >
>> >> > So I want it to reject writing a file and report an exception that the
>> >> > filename is missing.
>> >> >
>> >> > The file language supports you if you want to use the message id as
>> >> > the filename. Just set the header value as: ${id}
>> >> >
>> >> > And also remove option: ignoreFileNameHeader
>> >> >
>> >> >
>> >> > Thoughts?
>> >> >
>> >> > --
>> >> > Claus Ibsen
>> >> > Apache Camel Committer
>> >> >
>> >> > Open Source Integration: http://fusesource.com
>> >> > Blog: http://davsclaus.blogspot.com/
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> James
>> >> -------
>> >> http://macstrac.blogspot.com/
>> >>
>> >> Open Source Integration
>> >> http://fusesource.com/
>> >>
>> >
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: [HEADS UP] - File component changes in Camel 2.0
Posted by Aaron Crickenberger <aa...@intalgent.com>.
On a slightly related tangent, I'm partly responsible for the hackish code
in FileConsumer's (old) isUnchanged() method (
https://issues.apache.org/activemq/browse/CAMEL-250).
Camel 1.5's exclusiveReadLock feature is unfortunately still picking up
files too early, so I have to fall back to the deprecated behavior. This
happens regardless of the file processing strategy (delete, rename, etc.)
Is there any chance the hackish double-polling stuff can be put back in to
2.0 as a non-default file locking strategy? I can take a crack at it, I
just wasn't sure if the code was still in flux.
- aaron
On Mon, Feb 2, 2009 at 4:44 AM, Claus Ibsen <cl...@gmail.com> wrote:
> On Fri, Jan 30, 2009 at 5:40 PM, Aaron Crickenberger
> <aa...@intalgent.com> wrote:
> > I would hesitate if only because requiring a particular header seems off,
> > are there other components that do the same?
> >
> > I haven't looked much at Camel 2.0's code, but it looks like camel-1.x's
> > file component's "expression" property could support both scenarios. Use
> a
> > default ${id} expression, but allow user to configure w/ a ${in.header}
> > expression that barks if the header's not present, no?
> That is correct.
>
> Thanks for the feedback. We will leave it as is.
>
> >
> > - aaron
> >
> > On Fri, Jan 30, 2009 at 9:49 AM, James Strachan <
> james.strachan@gmail.com>wrote:
> >
> >> So long as files are written in some directory and they are unique and
> >> preserve order - does it matter if they are generated (from the
> >> message ID say) or the user explicitly gives some name? Like Jon I'd
> >> be tempted ot leave the default behaviour?
> >>
> >> 2009/1/30 Claus Ibsen <cl...@gmail.com>:
> >> > Hi
> >> >
> >> > As some of you know the file component have had a major refactor ...
> >> > actually you can nearly consider it as a rewrite in Camel 2.0.
> >> >
> >> > This mail is about a few remaining issues I want to give a heads up
> >> > upon and feedback:
> >> >
> >> >
> >> > Force a filename to be provided when wring a file
> >> > ====================================
> >> >
> >> > I want to force file producer always requiring a header value with the
> >> > filename to write.
> >> > What we have in Camel 1.x is that if no filename header is provided it
> >> > will fallback to use the message id as the filename.
> >> >
> >> > For me that has no use, as its kinda like telling a database here is
> >> > some data store it somewhere, without providing, schema, table, column
> >> > names.
> >> >
> >> > So I want it to reject writing a file and report an exception that the
> >> > filename is missing.
> >> >
> >> > The file language supports you if you want to use the message id as
> >> > the filename. Just set the header value as: ${id}
> >> >
> >> > And also remove option: ignoreFileNameHeader
> >> >
> >> >
> >> > Thoughts?
> >> >
> >> > --
> >> > Claus Ibsen
> >> > Apache Camel Committer
> >> >
> >> > Open Source Integration: http://fusesource.com
> >> > Blog: http://davsclaus.blogspot.com/
> >> >
> >>
> >>
> >>
> >> --
> >> James
> >> -------
> >> http://macstrac.blogspot.com/
> >>
> >> Open Source Integration
> >> http://fusesource.com/
> >>
> >
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
>
Re: [HEADS UP] - File component changes in Camel 2.0
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 30, 2009 at 5:40 PM, Aaron Crickenberger
<aa...@intalgent.com> wrote:
> I would hesitate if only because requiring a particular header seems off,
> are there other components that do the same?
>
> I haven't looked much at Camel 2.0's code, but it looks like camel-1.x's
> file component's "expression" property could support both scenarios. Use a
> default ${id} expression, but allow user to configure w/ a ${in.header}
> expression that barks if the header's not present, no?
That is correct.
Thanks for the feedback. We will leave it as is.
>
> - aaron
>
> On Fri, Jan 30, 2009 at 9:49 AM, James Strachan <ja...@gmail.com>wrote:
>
>> So long as files are written in some directory and they are unique and
>> preserve order - does it matter if they are generated (from the
>> message ID say) or the user explicitly gives some name? Like Jon I'd
>> be tempted ot leave the default behaviour?
>>
>> 2009/1/30 Claus Ibsen <cl...@gmail.com>:
>> > Hi
>> >
>> > As some of you know the file component have had a major refactor ...
>> > actually you can nearly consider it as a rewrite in Camel 2.0.
>> >
>> > This mail is about a few remaining issues I want to give a heads up
>> > upon and feedback:
>> >
>> >
>> > Force a filename to be provided when wring a file
>> > ====================================
>> >
>> > I want to force file producer always requiring a header value with the
>> > filename to write.
>> > What we have in Camel 1.x is that if no filename header is provided it
>> > will fallback to use the message id as the filename.
>> >
>> > For me that has no use, as its kinda like telling a database here is
>> > some data store it somewhere, without providing, schema, table, column
>> > names.
>> >
>> > So I want it to reject writing a file and report an exception that the
>> > filename is missing.
>> >
>> > The file language supports you if you want to use the message id as
>> > the filename. Just set the header value as: ${id}
>> >
>> > And also remove option: ignoreFileNameHeader
>> >
>> >
>> > Thoughts?
>> >
>> > --
>> > Claus Ibsen
>> > Apache Camel Committer
>> >
>> > Open Source Integration: http://fusesource.com
>> > Blog: http://davsclaus.blogspot.com/
>> >
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://fusesource.com/
>>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Re: [HEADS UP] - File component changes in Camel 2.0
Posted by Aaron Crickenberger <aa...@intalgent.com>.
I would hesitate if only because requiring a particular header seems off,
are there other components that do the same?
I haven't looked much at Camel 2.0's code, but it looks like camel-1.x's
file component's "expression" property could support both scenarios. Use a
default ${id} expression, but allow user to configure w/ a ${in.header}
expression that barks if the header's not present, no?
- aaron
On Fri, Jan 30, 2009 at 9:49 AM, James Strachan <ja...@gmail.com>wrote:
> So long as files are written in some directory and they are unique and
> preserve order - does it matter if they are generated (from the
> message ID say) or the user explicitly gives some name? Like Jon I'd
> be tempted ot leave the default behaviour?
>
> 2009/1/30 Claus Ibsen <cl...@gmail.com>:
> > Hi
> >
> > As some of you know the file component have had a major refactor ...
> > actually you can nearly consider it as a rewrite in Camel 2.0.
> >
> > This mail is about a few remaining issues I want to give a heads up
> > upon and feedback:
> >
> >
> > Force a filename to be provided when wring a file
> > ====================================
> >
> > I want to force file producer always requiring a header value with the
> > filename to write.
> > What we have in Camel 1.x is that if no filename header is provided it
> > will fallback to use the message id as the filename.
> >
> > For me that has no use, as its kinda like telling a database here is
> > some data store it somewhere, without providing, schema, table, column
> > names.
> >
> > So I want it to reject writing a file and report an exception that the
> > filename is missing.
> >
> > The file language supports you if you want to use the message id as
> > the filename. Just set the header value as: ${id}
> >
> > And also remove option: ignoreFileNameHeader
> >
> >
> > Thoughts?
> >
> > --
> > Claus Ibsen
> > Apache Camel Committer
> >
> > Open Source Integration: http://fusesource.com
> > Blog: http://davsclaus.blogspot.com/
> >
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>
Re: [HEADS UP] - File component changes in Camel 2.0
Posted by James Strachan <ja...@gmail.com>.
So long as files are written in some directory and they are unique and
preserve order - does it matter if they are generated (from the
message ID say) or the user explicitly gives some name? Like Jon I'd
be tempted ot leave the default behaviour?
2009/1/30 Claus Ibsen <cl...@gmail.com>:
> Hi
>
> As some of you know the file component have had a major refactor ...
> actually you can nearly consider it as a rewrite in Camel 2.0.
>
> This mail is about a few remaining issues I want to give a heads up
> upon and feedback:
>
>
> Force a filename to be provided when wring a file
> ====================================
>
> I want to force file producer always requiring a header value with the
> filename to write.
> What we have in Camel 1.x is that if no filename header is provided it
> will fallback to use the message id as the filename.
>
> For me that has no use, as its kinda like telling a database here is
> some data store it somewhere, without providing, schema, table, column
> names.
>
> So I want it to reject writing a file and report an exception that the
> filename is missing.
>
> The file language supports you if you want to use the message id as
> the filename. Just set the header value as: ${id}
>
> And also remove option: ignoreFileNameHeader
>
>
> Thoughts?
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
>
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://fusesource.com/