You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by "Josh Wills (JIRA)" <ji...@apache.org> on 2012/12/04 23:10:58 UTC
[jira] [Created] (CRUNCH-124) Javadoc the From, To, and At static
methods and add convenience methods for sequenceFiles
Josh Wills created CRUNCH-124:
---------------------------------
Summary: Javadoc the From, To, and At static methods and add convenience methods for sequenceFiles
Key: CRUNCH-124
URL: https://issues.apache.org/jira/browse/CRUNCH-124
Project: Crunch
Issue Type: Improvement
Components: IO
Reporter: Josh Wills
Aside from org.apache.crunch.*, the o.a.c.io.(From|To|At) methods are among the most user-facing classes we have. We should javadoc them thoroughly and add more convenience methods that make them easier to get started with for new Crunch users.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-124) Javadoc the From, To, and At static
methods and add convenience methods for sequenceFiles
Posted by "Josh Wills (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510740#comment-13510740 ]
Josh Wills commented on CRUNCH-124:
-----------------------------------
@Ashish will do in my next rev.
@Gabriel my thinking was that the common case for consuming an Avro file was one where some kind of external schema was already defined and the generated Java code implemented SpecificRecord. To cut down on the number of functions, we could opt to remove the type restrictions on the method and use Avros.records(clazz) to generate the PType, although we'd need the docs to be explicit about what was going on there. Is that your preference? I'm also curious what other Avro users think is the right behavior here.
> Javadoc the From, To, and At static methods and add convenience methods for sequenceFiles
> -----------------------------------------------------------------------------------------
>
> Key: CRUNCH-124
> URL: https://issues.apache.org/jira/browse/CRUNCH-124
> Project: Crunch
> Issue Type: Improvement
> Components: IO
> Reporter: Josh Wills
> Attachments: CRUNCH-124.patch
>
>
> Aside from org.apache.crunch.*, the o.a.c.io.(From|To|At) methods are among the most user-facing classes we have. We should javadoc them thoroughly and add more convenience methods that make them easier to get started with for new Crunch users.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CRUNCH-124) Javadoc the From, To, and At static
methods and add convenience methods for sequenceFiles
Posted by "Josh Wills (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Wills updated CRUNCH-124:
------------------------------
Attachment: CRUNCH-124.patch
I did this in two phases-- the first phase just adds javadoc and some additional convenience methods. The second phase adds a new top-level interface, TableSourceTarget, and uses it to abstract away the specific implementations of classes that are returned by the At factory methods. [~mafr] what is your take on this?
> Javadoc the From, To, and At static methods and add convenience methods for sequenceFiles
> -----------------------------------------------------------------------------------------
>
> Key: CRUNCH-124
> URL: https://issues.apache.org/jira/browse/CRUNCH-124
> Project: Crunch
> Issue Type: Improvement
> Components: IO
> Reporter: Josh Wills
> Attachments: CRUNCH-124.patch
>
>
> Aside from org.apache.crunch.*, the o.a.c.io.(From|To|At) methods are among the most user-facing classes we have. We should javadoc them thoroughly and add more convenience methods that make them easier to get started with for new Crunch users.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-124) Javadoc the From, To, and At static
methods and add convenience methods for sequenceFiles
Posted by "Gabriel Reid (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510724#comment-13510724 ]
Gabriel Reid commented on CRUNCH-124:
-------------------------------------
Nice update, especially having the docs to remind me that the value (and not the key) is used when reading a single field from sequence files :-)
I really like the new methods in From and At where you only have to provide the class instead of the PType. One small nitpick though -- I noticed that only the specific AvroType is supported here (and not generics or reflects). I think supporting generics and reflects here in the same way could be useful -- on the other hand, it would make an already-big list of static methods even bigger. Was that omission intentional?
> Javadoc the From, To, and At static methods and add convenience methods for sequenceFiles
> -----------------------------------------------------------------------------------------
>
> Key: CRUNCH-124
> URL: https://issues.apache.org/jira/browse/CRUNCH-124
> Project: Crunch
> Issue Type: Improvement
> Components: IO
> Reporter: Josh Wills
> Attachments: CRUNCH-124.patch
>
>
> Aside from org.apache.crunch.*, the o.a.c.io.(From|To|At) methods are among the most user-facing classes we have. We should javadoc them thoroughly and add more convenience methods that make them easier to get started with for new Crunch users.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-124) Javadoc the From, To, and At static
methods and add convenience methods for sequenceFiles
Posted by "Gabriel Reid (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510770#comment-13510770 ]
Gabriel Reid commented on CRUNCH-124:
-------------------------------------
[~jwills] I think the way it is now will be fine. You're probably right that the most common use-case is using SpecificRecord, and it's not that big of a bother to provide the PType for the other use cases when necessary.
In my Avro experience, I've never really relied on generic records, although in the past I did use the reflection-based Avro serialization (together with Lombok-annotated classes) quite a bit to do quick prototyping, which basically gives you the same as working with SpecificRecords without the hassle of creating a schema or generating classes. I think that this was mostly motivated by laziness, and I don't think it was that common of a use case.
> Javadoc the From, To, and At static methods and add convenience methods for sequenceFiles
> -----------------------------------------------------------------------------------------
>
> Key: CRUNCH-124
> URL: https://issues.apache.org/jira/browse/CRUNCH-124
> Project: Crunch
> Issue Type: Improvement
> Components: IO
> Reporter: Josh Wills
> Attachments: CRUNCH-124.patch
>
>
> Aside from org.apache.crunch.*, the o.a.c.io.(From|To|At) methods are among the most user-facing classes we have. We should javadoc them thoroughly and add more convenience methods that make them easier to get started with for new Crunch users.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-124) Javadoc the From, To, and At static
methods and add convenience methods for sequenceFiles
Posted by "Ashish Paliwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510285#comment-13510285 ]
Ashish Paliwal commented on CRUNCH-124:
---------------------------------------
@Josh - IMHO, would be great to add some examples in the class javadoc. This shall be of great help while using these user facing functions
> Javadoc the From, To, and At static methods and add convenience methods for sequenceFiles
> -----------------------------------------------------------------------------------------
>
> Key: CRUNCH-124
> URL: https://issues.apache.org/jira/browse/CRUNCH-124
> Project: Crunch
> Issue Type: Improvement
> Components: IO
> Reporter: Josh Wills
> Attachments: CRUNCH-124.patch
>
>
> Aside from org.apache.crunch.*, the o.a.c.io.(From|To|At) methods are among the most user-facing classes we have. We should javadoc them thoroughly and add more convenience methods that make them easier to get started with for new Crunch users.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira