You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Alan Gates (JIRA)" <ji...@apache.org> on 2010/06/18 21:45:24 UTC
[jira] Created: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
UDF manual and javadocs should make clear how to use RequiredFieldList
----------------------------------------------------------------------
Key: PIG-1460
URL: https://issues.apache.org/jira/browse/PIG-1460
Project: Pig
Issue Type: Bug
Components: documentation
Affects Versions: 0.7.0
Reporter: Alan Gates
Priority: Minor
The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1460:
----------------------------
Attachment: PIG-1460-1.patch
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Daniel Dai
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: PIG-1460-1.patch
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1460:
----------------------------
Attachment: (was: PIG-1460-1.patch)
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Daniel Dai
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: PIG-1460-1.patch
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-1460:
----------------------------
Fix Version/s: 0.8.0
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Priority: Minor
> Fix For: 0.8.0
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich reassigned PIG-1460:
-----------------------------------
Assignee: Daniel Dai (was: Pradeep Kamath)
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Daniel Dai
> Priority: Minor
> Fix For: 0.8.0
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich reassigned PIG-1460:
-----------------------------------
Assignee: Pradeep Kamath
Pradeep, could you provide the information needed and also update the javadoc. Then, please, re-assign to Corinne so that she can update the UDF manual, thanks.
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Pradeep Kamath
> Priority: Minor
> Fix For: 0.8.0
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Corinne Chandel (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Corinne Chandel resolved PIG-1460.
----------------------------------
Resolution: Fixed
UDF manual updated - patch submitted via pig-1600
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Corinne Chandel
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: PIG-1460-1.patch
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1460:
----------------------------
Attachment: PIG-1460-1.patch
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Daniel Dai
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: PIG-1460-1.patch
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai reassigned PIG-1460:
-------------------------------
Assignee: Corinne Chandel (was: Daniel Dai)
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Corinne Chandel
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: PIG-1460-1.patch
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888094#action_12888094 ]
Olga Natkovich commented on PIG-1460:
-------------------------------------
Pradeep, could you please review before Daniel commits the text and before Corinne updates the docs. Thanks.
Daniel, please, assign the ticket to Corinne once your patch is committed, thanks.
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Corinne Chandel
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: PIG-1460-1.patch
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1460:
----------------------------
Attachment: (was: PIG-1460-1.patch)
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Daniel Dai
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: PIG-1460-1.patch
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888090#action_12888090 ]
Daniel Dai commented on PIG-1460:
---------------------------------
Hi, Corinne,
Can you update Pig UDF Manual? After section "LoadPushDown", add a subsection (pushProjection() is a method of LoadPushDown (also help to polish my writing):
pushProjection(): This method tells LoadFunc which fields is used in Pig, so LoadFunc is able to optimize not to load unneeded fields. pushProjection takes a RequiredFieldList. Each item inside RequiredFieldList indicates a required field Pig needs. Pig will use column index RequiredField.index to communicate with the LoadFunc about the fields Pig needs. If the field is a map, Pig will optionally pass RequiredField.subFields which contains a list of keys Pig needs for that map. For example, if we need two keys "key1", "key2" for the map, we will have a subFields for that map which contains two RequiredField. The alias field for 1st RequiredField is "key1", alias field for 2nd RequiredField is "key2". RequiredField.type is reserved for future use. LoadFunc will use RequiredFieldResponse.requiredFieldRequestHonored to indicate whether the pushProjection request is honored.
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Daniel Dai
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: PIG-1460-1.patch
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880299#action_12880299 ]
Alan Gates commented on PIG-1460:
---------------------------------
>From email thread on the pig-user list:
{quote}
The documentation is also poor when it comes to describing what the
RequiredFieldList even is.
It has a name and an index field. The code itself seems to allow for
either of these to be filled. What do they mean?
Is it:
the schema returned by the loader is:
(id: int, name: chararray, department: chararray)
The RequiredFieldList is [ ("department", 1) , ("id", 0) ]
What does that mean?
* The name is the field name requested, and the index is the location it
should be in the result? so return (id: int, department: chararray)?
* The index is the index in the source schema, and the name is for
renaming, so return (department: chararray, id: int) (where the data in
department is actualy that from the original's name field)?
* The location in the RequiredFieldList array is the 'destination'
requested, the name is optional (if the schema had one) and the index is the
location in the original schema. so the above RequiredFieldList is actually
impossible, since "department" is always index 2.
{quote}
The last is the correct answer.
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Priority: Minor
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1460) UDF manual and javadocs should make
clear how to use RequiredFieldList
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1460:
----------------------------
Attachment: PIG-1460-1.patch
Patch for javadoc for RequiredFieldList
> UDF manual and javadocs should make clear how to use RequiredFieldList
> ----------------------------------------------------------------------
>
> Key: PIG-1460
> URL: https://issues.apache.org/jira/browse/PIG-1460
> Project: Pig
> Issue Type: Bug
> Components: documentation
> Affects Versions: 0.7.0
> Reporter: Alan Gates
> Assignee: Daniel Dai
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: PIG-1460-1.patch
>
>
> The UDF manual mentions that load function writers need to handle RequiredFieldList passed to LoadPushDown.pushProjection, but it does not specify how the writer should interpret the contents of that list. The javadoc is similarly vague.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.