You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by Matthias Friedrich <ma...@mafr.de> on 2012/10/08 20:28:49 UTC
Javadoc improvements
Hi,
one thing I'd like to get into our next release is a first step
towards better API documentation. For this it would be helpful to
agree on a reduced set of packages intended for client use (the
"published" API).
I used the javadoc tool's grouping and exclusion mechanism to only
display packages that I think should be part of the published API.
See [1] on how this could look like, compared to our current
documentation at [2]. Is this list correct? Did I miss something?
With just some exclusions I got from 243 classes/interfaces down to
158. We could reduce this even further by making implementation
classes package private where possible. I'll run an analysis as soon
as we have agreed on the set of published packages.
I'm not sure about the "Other Packages" section. o.a.c.tool should
probably be removed, with its content thrown into the util package.
Part of the o.a.c.types looks like it would be better off in the
base package (PType, PTypeFamily) while the rest looks like helper
functionality for o.a.c.types.* that shouldn't be published. What
do you think?
Regards,
Matthias
[1] http://tmp.mafr.de/crunch/apidocs/0.4.0-incubating-SNAPSHOT/
[2] http://incubator.apache.org/crunch/apidocs/0.3.0/
Re: Javadoc improvements
Posted by Gabriel Reid <ga...@gmail.com>.
On Tue, Oct 9, 2012 at 7:36 PM, Matthias Friedrich <ma...@mafr.de> wrote:
> On Tuesday, 2012-10-09, Gabriel Reid wrote:
>> On Mon, Oct 8, 2012 at 8:28 PM, Matthias Friedrich <ma...@mafr.de> wrote:
> [...]
>>> I used the javadoc tool's grouping and exclusion mechanism to only
>>> display packages that I think should be part of the published API.
>>> See [1] on how this could look like, compared to our current
>>> documentation at [2]. Is this list correct? Did I miss something?
>
>> In addition to the lib.join package that Josh mentioned, we might want
>> to include the format-specific packages (i.e. seq, avro, text) under
>> o.a.c.io. I'm not totally sure on this, as they are indirectly
>> accessible via o.a.c.At and o.a.c.To, but with the file naming stuff I
>> was looking at adding as part of CRUNCH-91, the file naming API was is
>> available on the format-specific implementation classes. On the other
>> hand, it could just be added to the At and To classes, and then this
>> becomes a non-issue.
>
> Since we have the static factories I think we don't need to show the
> implementation packages. Is there anything in there users would want
> to reference directly? I didn't find anything, but I'm not that
> familiar with the code.
Ok, good point -- this just means that we need to expand the static
factories with some extra methods, but I'm fine with that.
>
> Hmm, regarding the file naming, I'm not sure which pattern we should
> follow in general: a) have implementation packages that are excluded
> from the docs and provide access via static factories from a base
> package, or b) hide the base package, put factories next to the
> implementation and make the implementation package private.
>
> A lot of Crunch leans towards a) and it would be a major pain to
> refactor it. On the other hand that's where our dependency cycles come
> from, at least when the implementation packages depend on base :)
Yes, I definitely think we should stick with the static factories, but
there is quite a bit of stuff that we can untangle there as well.
Re: Javadoc improvements
Posted by Matthias Friedrich <ma...@mafr.de>.
On Tuesday, 2012-10-09, Gabriel Reid wrote:
> On Mon, Oct 8, 2012 at 8:28 PM, Matthias Friedrich <ma...@mafr.de> wrote:
[...]
>> I used the javadoc tool's grouping and exclusion mechanism to only
>> display packages that I think should be part of the published API.
>> See [1] on how this could look like, compared to our current
>> documentation at [2]. Is this list correct? Did I miss something?
> In addition to the lib.join package that Josh mentioned, we might want
> to include the format-specific packages (i.e. seq, avro, text) under
> o.a.c.io. I'm not totally sure on this, as they are indirectly
> accessible via o.a.c.At and o.a.c.To, but with the file naming stuff I
> was looking at adding as part of CRUNCH-91, the file naming API was is
> available on the format-specific implementation classes. On the other
> hand, it could just be added to the At and To classes, and then this
> becomes a non-issue.
Since we have the static factories I think we don't need to show the
implementation packages. Is there anything in there users would want
to reference directly? I didn't find anything, but I'm not that
familiar with the code.
Hmm, regarding the file naming, I'm not sure which pattern we should
follow in general: a) have implementation packages that are excluded
from the docs and provide access via static factories from a base
package, or b) hide the base package, put factories next to the
implementation and make the implementation package private.
A lot of Crunch leans towards a) and it would be a major pain to
refactor it. On the other hand that's where our dependency cycles come
from, at least when the implementation packages depend on base :)
Regards,
Matthias
Re: Javadoc improvements
Posted by Gabriel Reid <ga...@gmail.com>.
On Mon, Oct 8, 2012 at 8:28 PM, Matthias Friedrich <ma...@mafr.de> wrote:
> one thing I'd like to get into our next release is a first step
> towards better API documentation. For this it would be helpful to
> agree on a reduced set of packages intended for client use (the
> "published" API).
+1, this is definitely a place where we can get a big gain in usability
>
> I used the javadoc tool's grouping and exclusion mechanism to only
> display packages that I think should be part of the published API.
> See [1] on how this could look like, compared to our current
> documentation at [2]. Is this list correct? Did I miss something?
In addition to the lib.join package that Josh mentioned, we might want
to include the format-specific packages (i.e. seq, avro, text) under
o.a.c.io. I'm not totally sure on this, as they are indirectly
accessible via o.a.c.At and o.a.c.To, but with the file naming stuff I
was looking at adding as part of CRUNCH-91, the file naming API was is
available on the format-specific implementation classes. On the other
hand, it could just be added to the At and To classes, and then this
becomes a non-issue.
>
> With just some exclusions I got from 243 classes/interfaces down to
> 158. We could reduce this even further by making implementation
> classes package private where possible. I'll run an analysis as soon
> as we have agreed on the set of published packages.
>
> I'm not sure about the "Other Packages" section. o.a.c.tool should
> probably be removed, with its content thrown into the util package.
> Part of the o.a.c.types looks like it would be better off in the
> base package (PType, PTypeFamily) while the rest looks like helper
> functionality for o.a.c.types.* that shouldn't be published. What
> do you think?
>
> Regards,
> Matthias
>
> [1] http://tmp.mafr.de/crunch/apidocs/0.4.0-incubating-SNAPSHOT/
> [2] http://incubator.apache.org/crunch/apidocs/0.3.0/
Re: Javadoc improvements
Posted by Matthias Friedrich <ma...@mafr.de>.
On Monday, 2012-10-08, Josh Wills wrote:
> On Mon, Oct 8, 2012 at 11:28 AM, Matthias Friedrich <ma...@mafr.de> wrote:
[...]
>> I used the javadoc tool's grouping and exclusion mechanism to only
>> display packages that I think should be part of the published API.
>> See [1] on how this could look like, compared to our current
>> documentation at [2]. Is this list correct? Did I miss something?
> lib.join.* seems to be the biggest thing missing.
Hmm, right. It's the reference to JoinFn that needs .lib.join,
otherwise we could exclude .lib.join. Maybe it would be better to
move the Join class to .lib.join? That's where I would look for it.
>> With just some exclusions I got from 243 classes/interfaces down to
>> 158. We could reduce this even further by making implementation
>> classes package private where possible. I'll run an analysis as soon
>> as we have agreed on the set of published packages.
Update: It's not as many as I thought (~30), and most are excluded
anyway.
>> I'm not sure about the "Other Packages" section. o.a.c.tool should
>> probably be removed, with its content thrown into the util package.
>> Part of the o.a.c.types looks like it would be better off in the
>> base package (PType, PTypeFamily) while the rest looks like helper
>> functionality for o.a.c.types.* that shouldn't be published. What
>> do you think?
> I have a sentimental attachment to o.a.c.tool b/c it reminds me of the
> Scrunch PipelineApp stuff, but on the other hand I never end up using
> it for anything. Agree w/you w/respect to moving much of the rest of
> o.a.c.util into o.a.c.types.
OK, to sum it up:
* Protos, PTypes -> .types
* CrunchTool -> .util
* Remove Collects because it's unused?
Regards,
Matthias
Re: Javadoc improvements
Posted by Josh Wills <jo...@gmail.com>.
On Mon, Oct 8, 2012 at 11:28 AM, Matthias Friedrich <ma...@mafr.de> wrote:
> Hi,
>
> one thing I'd like to get into our next release is a first step
> towards better API documentation. For this it would be helpful to
> agree on a reduced set of packages intended for client use (the
> "published" API).
>
> I used the javadoc tool's grouping and exclusion mechanism to only
> display packages that I think should be part of the published API.
> See [1] on how this could look like, compared to our current
> documentation at [2]. Is this list correct? Did I miss something?
lib.join.* seems to be the biggest thing missing.
>
> With just some exclusions I got from 243 classes/interfaces down to
> 158. We could reduce this even further by making implementation
> classes package private where possible. I'll run an analysis as soon
> as we have agreed on the set of published packages.
>
> I'm not sure about the "Other Packages" section. o.a.c.tool should
> probably be removed, with its content thrown into the util package.
> Part of the o.a.c.types looks like it would be better off in the
> base package (PType, PTypeFamily) while the rest looks like helper
> functionality for o.a.c.types.* that shouldn't be published. What
> do you think?
I have a sentimental attachment to o.a.c.tool b/c it reminds me of the
Scrunch PipelineApp stuff, but on the other hand I never end up using
it for anything. Agree w/you w/respect to moving much of the rest of
o.a.c.util into o.a.c.types.
J
>
> Regards,
> Matthias
>
> [1] http://tmp.mafr.de/crunch/apidocs/0.4.0-incubating-SNAPSHOT/
> [2] http://incubator.apache.org/crunch/apidocs/0.3.0/