You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Matt Benson <gu...@yahoo.com> on 2008/06/18 19:18:43 UTC
Incubation proposal for [flatfile] component
At long last, I have prepared the preliminary
proposal. I'd like to have the support of Commons as
the sponsoring PMC. I'd also like it if I could get a
couple of mentors from the group as well. The current
proposal can be viewed at:
http://people.apache.org/~mbenson/flatfile-proposal/proposal
If you have any interest in this being a Commons
component, or if you are a PMC member and feel you
would veto Commons' sponsorship of this podling,
please say so. Assuming the initial reaction is
positive, I will then call a vote to make the
sponsorship official and move the proposal to the
Incubator wiki where I hope volunteering mentors will
add themselves. :)
Thanks,
Matt
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Incubation proposal for [flatfile] component
Posted by Christian Grobmeier <gr...@gmail.com>.
On Wed, Jun 18, 2008 at 11:53 PM, Matt Benson <gu...@yahoo.com> wrote:
> --- Emmanuel Bourg <eb...@apache.org> wrote:
>
> > That looks interesting, this kind of structure can
> > still be found today
> > in ar and cpio archives used for deb and rpm
> > packages. I also used this
> > style of file in inter bank communications a few
> > years ago.
> >
> > That would be awesome if the component was able to
> > operate on binary
> > files too, you would define the structure in a DSL
> > like language and map
> > it to your beans. Just dreaming :)
>
> Actually, there's not much reason it couldn't support
> binary files. The method signatures are defined in
> terms of bytes rather than chars, and unicode is
> supported in the DSL's string literals (I'm aware that
> the DSL stuff badly needs i18n support). It's
> possible that "Flatfile" doesn't capture the potential
> as well as some other name might. :|
>
That would be great. I even had to write parsers for fiiles in which binary
data and textual data are mixed. This happened several times
i worked with MQSeries. I think the API should support fixed lenght data
too.
Maybe there could be some implementation for a lenght field definition.
With that I mean a field of 4 ints, telling you how often a structure of xy
chars
is repeating and so on.
Chris
Re: Incubation proposal for [flatfile] component
Posted by Matt Benson <gu...@yahoo.com>.
--- Emmanuel Bourg <eb...@apache.org> wrote:
> That looks interesting, this kind of structure can
> still be found today
> in ar and cpio archives used for deb and rpm
> packages. I also used this
> style of file in inter bank communications a few
> years ago.
>
> That would be awesome if the component was able to
> operate on binary
> files too, you would define the structure in a DSL
> like language and map
> it to your beans. Just dreaming :)
Actually, there's not much reason it couldn't support
binary files. The method signatures are defined in
terms of bytes rather than chars, and unicode is
supported in the DSL's string literals (I'm aware that
the DSL stuff badly needs i18n support). It's
possible that "Flatfile" doesn't capture the potential
as well as some other name might. :|
>
> Do you plan to implement push/pull parsing (maybe it
> already does).
My understanding of the push/pull terminology is
limited to the context of XML. I have a hard time
resolving those to what the Flatfile library does, but
basically: you have an Entity instance. You tell it
to read itself from an InputStream. When that
synchronous call returns you have a populated Entity.
Currently, then, the read operations only work for
fully-specified, non-dynamic field definitions. I
definitely want to explore strategies for improving
this functionality; the InputFilteringDynamicField
class is something I began to write for the purposes
of trying out a lookahead strategy. I didn't go too
far with that because once it was certain that I would
be allowed to attempt ASF incubation, i kind of wanted
to save some work for that process. ;)
>
> Are there any similar open source projects ?
I think Hen mentioned something he had that did a
small part of this. Spring-Batch came out fairly
recently and irritated me, because it does a little of
this in its bigger picture, but it's quite difficult
to compare the two IMHO. The Spring-Batch stuff looks
like a slightly more polished version of the same
stuff I've seen in different shops for years--with an
OSS license stuck on. I think the DSL and nested
structures are a big win for the Flatfile code, but I
would say that.
Thanks for looking,
Matt
>
> Emmanuel Bourg
>
>
> Matt Benson a écrit :
> > At long last, I have prepared the preliminary
> > proposal. I'd like to have the support of Commons
> as
> > the sponsoring PMC. I'd also like it if I could
> get a
> > couple of mentors from the group as well. The
> current
> > proposal can be viewed at:
> >
>
http://people.apache.org/~mbenson/flatfile-proposal/proposal
> >
> > If you have any interest in this being a Commons
> > component, or if you are a PMC member and feel you
> > would veto Commons' sponsorship of this podling,
> > please say so. Assuming the initial reaction is
> > positive, I will then call a vote to make the
> > sponsorship official and move the proposal to the
> > Incubator wiki where I hope volunteering mentors
> will
> > add themselves. :)
> >
> > Thanks,
> > Matt
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> dev-unsubscribe@commons.apache.org
> For additional commands, e-mail:
> dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Incubation proposal for [flatfile] component
Posted by Emmanuel Bourg <eb...@apache.org>.
That looks interesting, this kind of structure can still be found today
in ar and cpio archives used for deb and rpm packages. I also used this
style of file in inter bank communications a few years ago.
That would be awesome if the component was able to operate on binary
files too, you would define the structure in a DSL like language and map
it to your beans. Just dreaming :)
Do you plan to implement push/pull parsing (maybe it already does).
Are there any similar open source projects ?
Emmanuel Bourg
Matt Benson a écrit :
> At long last, I have prepared the preliminary
> proposal. I'd like to have the support of Commons as
> the sponsoring PMC. I'd also like it if I could get a
> couple of mentors from the group as well. The current
> proposal can be viewed at:
> http://people.apache.org/~mbenson/flatfile-proposal/proposal
>
> If you have any interest in this being a Commons
> component, or if you are a PMC member and feel you
> would veto Commons' sponsorship of this podling,
> please say so. Assuming the initial reaction is
> positive, I will then call a vote to make the
> sponsorship official and move the proposal to the
> Incubator wiki where I hope volunteering mentors will
> add themselves. :)
>
> Thanks,
> Matt
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Incubation proposal for [flatfile] component
Posted by Matt Benson <gu...@yahoo.com>.
--- sebb <se...@gmail.com> wrote:
> On 18/06/2008, Matt Benson <gu...@yahoo.com>
> wrote:
> > At long last, I have prepared the preliminary
> > proposal. I'd like to have the support of
> Commons as
> > the sponsoring PMC. I'd also like it if I could
> get a
> > couple of mentors from the group as well. The
> current
> > proposal can be viewed at:
> >
>
http://people.apache.org/~mbenson/flatfile-proposal/proposal
>
> If I understand the docs, the proposed project
> allows one to parse or
> generate flat files in different formats, the
> formats being defined by
> a data description language - is that correct?
> If so, it might help to say that both
> reading/writing of flat files is
> supported.
Noted.
>
> It would really help to have some simple examples -
> and maybe some
> not-so-simple examples: e.g. could it be used to
> parse some of the ASF
> data files?
You mean like the foundation files? Eventually, but
ATM it's fairly weak on reading dynamically-sized
input. I'll try to knock some examples together
soon-ish. :|
>
> > If you have any interest in this being a Commons
> > component, or if you are a PMC member and feel
> you
> > would veto Commons' sponsorship of this podling,
> > please say so. Assuming the initial reaction is
> > positive, I will then call a vote to make the
> > sponsorship official and move the proposal to the
> > Incubator wiki where I hope volunteering mentors
> will
> > add themselves. :)
>
> Seems to me that Commons is a suitable home for the
> code.
Thanks again,
Matt
>
> > Thanks,
> > Matt
> >
> >
> >
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail:
> dev-help@commons.apache.org
> >
> >
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> dev-unsubscribe@commons.apache.org
> For additional commands, e-mail:
> dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Incubation proposal for [flatfile] component
Posted by sebb <se...@gmail.com>.
On 18/06/2008, Matt Benson <gu...@yahoo.com> wrote:
> At long last, I have prepared the preliminary
> proposal. I'd like to have the support of Commons as
> the sponsoring PMC. I'd also like it if I could get a
> couple of mentors from the group as well. The current
> proposal can be viewed at:
> http://people.apache.org/~mbenson/flatfile-proposal/proposal
If I understand the docs, the proposed project allows one to parse or
generate flat files in different formats, the formats being defined by
a data description language - is that correct?
If so, it might help to say that both reading/writing of flat files is
supported.
It would really help to have some simple examples - and maybe some
not-so-simple examples: e.g. could it be used to parse some of the ASF
data files?
> If you have any interest in this being a Commons
> component, or if you are a PMC member and feel you
> would veto Commons' sponsorship of this podling,
> please say so. Assuming the initial reaction is
> positive, I will then call a vote to make the
> sponsorship official and move the proposal to the
> Incubator wiki where I hope volunteering mentors will
> add themselves. :)
Seems to me that Commons is a suitable home for the code.
> Thanks,
> Matt
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Incubation proposal for [flatfile] component
Posted by Emmanuel Bourg <eb...@apache.org>.
Martin Cooper a écrit :
> Pardon me if I missed some previous discussion, but I'm not finding anything
> about "flatfile" in the archives.
I believe this was discussed along the csv component some time ago.
> Why is this heading for incubation? Given that you are the sole developer
> and already a committer here, why not just drop it in the sandbox?
That might be a good test since we have some difficulties to bootstrap
communities around the sandbox components.
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Incubation proposal for [flatfile] component
Posted by Emmanuel Bourg <eb...@apache.org>.
Martin Cooper a écrit :
> Pardon me if I missed some previous discussion, but I'm not finding anything
> about "flatfile" in the archives.
That was in march in year:
http://markmail.org/message/gdrsr5gtekjxlsgo
Emmanuel Bourg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Incubation proposal for [flatfile] component
Posted by Matt Benson <gu...@yahoo.com>.
--- Henri Yandell <fl...@gmail.com> wrote:
> On Thu, Jun 19, 2008 at 5:36 AM, Niall Pemberton
> <ni...@gmail.com> wrote:
> > On Thu, Jun 19, 2008 at 3:15 AM, Martin Cooper
> <ma...@apache.org> wrote:
> >> On Wed, Jun 18, 2008 at 10:18 AM, Matt Benson
> <gu...@yahoo.com> wrote:
> >>
> >>> At long last, I have prepared the preliminary
> >>> proposal.
> >>
> >>
> >> Pardon me if I missed some previous discussion,
> but I'm not finding anything
> >> about "flatfile" in the archives.
> >>
> >> Why is this heading for incubation? Given that
> you are the sole developer
> >> and already a committer here, why not just drop
> it in the sandbox?
> >
> > I believe we would need to go through the
> *short-form* incubator
> > process (i.e. IP clearance) rather than the
> full-blown project
> > incubation:
> >
> >
> http://incubator.apache.org/ip-clearance/index.html
> >
> > ...but otherwise I agree with Martin - once the
> appropriate IP
> > clearance forms have been processed thru' the
> incubator, then it could
> > just come straight into the sandbox.
>
> +1.
>
> With CSV the community was external to our own. No
> overlap. In this
> case it's a subset, so the only real issue Matt
> faces for flatfile is
> that we don't allow releases from the sandbox until
> he can find a few
> interested developers and get it promoted.
>
The contrast with CSV was indeed my reason for
believing it was necessary to pursue "full" incubation
in this case. If we are agreed as a community that
this is unnecessary, I will proceed with the IP
clearance-to-sandbox approach, then. :)
-Matt
> Hen
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> dev-unsubscribe@commons.apache.org
> For additional commands, e-mail:
> dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Incubation proposal for [flatfile] component
Posted by Henri Yandell <fl...@gmail.com>.
On Thu, Jun 19, 2008 at 5:36 AM, Niall Pemberton
<ni...@gmail.com> wrote:
> On Thu, Jun 19, 2008 at 3:15 AM, Martin Cooper <ma...@apache.org> wrote:
>> On Wed, Jun 18, 2008 at 10:18 AM, Matt Benson <gu...@yahoo.com> wrote:
>>
>>> At long last, I have prepared the preliminary
>>> proposal.
>>
>>
>> Pardon me if I missed some previous discussion, but I'm not finding anything
>> about "flatfile" in the archives.
>>
>> Why is this heading for incubation? Given that you are the sole developer
>> and already a committer here, why not just drop it in the sandbox?
>
> I believe we would need to go through the *short-form* incubator
> process (i.e. IP clearance) rather than the full-blown project
> incubation:
>
> http://incubator.apache.org/ip-clearance/index.html
>
> ...but otherwise I agree with Martin - once the appropriate IP
> clearance forms have been processed thru' the incubator, then it could
> just come straight into the sandbox.
+1.
With CSV the community was external to our own. No overlap. In this
case it's a subset, so the only real issue Matt faces for flatfile is
that we don't allow releases from the sandbox until he can find a few
interested developers and get it promoted.
Hen
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Incubation proposal for [flatfile] component
Posted by Niall Pemberton <ni...@gmail.com>.
On Thu, Jun 19, 2008 at 3:15 AM, Martin Cooper <ma...@apache.org> wrote:
> On Wed, Jun 18, 2008 at 10:18 AM, Matt Benson <gu...@yahoo.com> wrote:
>
>> At long last, I have prepared the preliminary
>> proposal.
>
>
> Pardon me if I missed some previous discussion, but I'm not finding anything
> about "flatfile" in the archives.
>
> Why is this heading for incubation? Given that you are the sole developer
> and already a committer here, why not just drop it in the sandbox?
I believe we would need to go through the *short-form* incubator
process (i.e. IP clearance) rather than the full-blown project
incubation:
http://incubator.apache.org/ip-clearance/index.html
...but otherwise I agree with Martin - once the appropriate IP
clearance forms have been processed thru' the incubator, then it could
just come straight into the sandbox.
Niall
> --
> Martin Cooper
>
>
>
>> I'd like to have the support of Commons as
>> the sponsoring PMC. I'd also like it if I could get a
>> couple of mentors from the group as well. The current
>> proposal can be viewed at:
>> http://people.apache.org/~mbenson/flatfile-proposal/proposal<http://people.apache.org/%7Embenson/flatfile-proposal/proposal>
>>
>> If you have any interest in this being a Commons
>> component, or if you are a PMC member and feel you
>> would veto Commons' sponsorship of this podling,
>> please say so. Assuming the initial reaction is
>> positive, I will then call a vote to make the
>> sponsorship official and move the proposal to the
>> Incubator wiki where I hope volunteering mentors will
>> add themselves. :)
>>
>> Thanks,
>> Matt
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Incubation proposal for [flatfile] component
Posted by Martin Cooper <ma...@apache.org>.
On Wed, Jun 18, 2008 at 10:18 AM, Matt Benson <gu...@yahoo.com> wrote:
> At long last, I have prepared the preliminary
> proposal.
Pardon me if I missed some previous discussion, but I'm not finding anything
about "flatfile" in the archives.
Why is this heading for incubation? Given that you are the sole developer
and already a committer here, why not just drop it in the sandbox?
--
Martin Cooper
> I'd like to have the support of Commons as
> the sponsoring PMC. I'd also like it if I could get a
> couple of mentors from the group as well. The current
> proposal can be viewed at:
> http://people.apache.org/~mbenson/flatfile-proposal/proposal<http://people.apache.org/%7Embenson/flatfile-proposal/proposal>
>
> If you have any interest in this being a Commons
> component, or if you are a PMC member and feel you
> would veto Commons' sponsorship of this podling,
> please say so. Assuming the initial reaction is
> positive, I will then call a vote to make the
> sponsorship official and move the proposal to the
> Incubator wiki where I hope volunteering mentors will
> add themselves. :)
>
> Thanks,
> Matt
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>