You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Craig Macdonald (JIRA)" <ji...@apache.org> on 2008/02/16 10:55:11 UTC
[jira] Created: (PIG-112) Allow Java packages to be searched for
functions.
Allow Java packages to be searched for functions.
-------------------------------------------------
Key: PIG-112
URL: https://issues.apache.org/jira/browse/PIG-112
Project: Pig
Issue Type: Improvement
Reporter: Craig Macdonald
Priority: Minor
Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
packages:
* org.apache.pig.builtin.
* org.apache.pig.impl.builtin.
* com.yahoo.pig.yst.sds.ULT.
Users should be able to add to this packagelist.
Example in Grunt:
{noformat}
include com.pig.function.package.
{noformat}
Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-112) Allow Java packages to be searched for
functions.
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated PIG-112:
--------------------------------
Attachment: import.v3.patch
Oh, yes I see. Updated patch attached.
* Several related methods in PigContext are now non-static, eg instantiateFuncFromSpec, instantiateFunc, resolveClassName, addClassSearchPackage (new).
* packageImportList is now non-static, and initially populated during constructor, not static{} block.
> Allow Java packages to be searched for functions.
> -------------------------------------------------
>
> Key: PIG-112
> URL: https://issues.apache.org/jira/browse/PIG-112
> Project: Pig
> Issue Type: Improvement
> Reporter: Craig Macdonald
> Priority: Minor
> Attachments: import.v2.patch, import.v3.patch, patch.import
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
> Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
> packages:
> * org.apache.pig.builtin.
> * org.apache.pig.impl.builtin.
> * com.yahoo.pig.yst.sds.ULT.
> Users should be able to add to this packagelist.
> Example in Grunt:
> {noformat}
> include com.pig.function.package.
> {noformat}
> This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
> Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-112) Allow Java packages to be searched for
functions.
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated PIG-112:
--------------------------------
Description:
Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
packages:
* org.apache.pig.builtin.
* org.apache.pig.impl.builtin.
* com.yahoo.pig.yst.sds.ULT.
Users should be able to add to this packagelist.
Example in Grunt:
{noformat}
include com.pig.function.package.
{noformat}
This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
Should be trivial to implement, or could be a property.
was:
Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
packages:
* org.apache.pig.builtin.
* org.apache.pig.impl.builtin.
* com.yahoo.pig.yst.sds.ULT.
Users should be able to add to this packagelist.
Example in Grunt:
{noformat}
include com.pig.function.package.
{noformat}
Should be trivial to implement, or could be a property.
draw similarities with other envs.
> Allow Java packages to be searched for functions.
> -------------------------------------------------
>
> Key: PIG-112
> URL: https://issues.apache.org/jira/browse/PIG-112
> Project: Pig
> Issue Type: Improvement
> Reporter: Craig Macdonald
> Priority: Minor
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
> Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
> packages:
> * org.apache.pig.builtin.
> * org.apache.pig.impl.builtin.
> * com.yahoo.pig.yst.sds.ULT.
> Users should be able to add to this packagelist.
> Example in Grunt:
> {noformat}
> include com.pig.function.package.
> {noformat}
> This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
> Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-112) Allow Java packages to be searched for
functions.
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated PIG-112:
--------------------------------
Attachment: patch.import
initial patch for adding an import command to grunt.
This patch has a problem - I cant do any dump commands after applying the patch, so I have probably broken something in PigScriptParser.jj
{code}
grunt> dump urls
2008-02-18 11:47:27,907 [main] ERROR org.apache.pig.tools.grunt.GruntParser - Encountered "urls" at line 3, column 6.
Was expecting:
<IDENTIFIER> ...
org.apache.pig.tools.pigscript.parser.ParseException: Encountered "urls" at line 3, column 6.
Was expecting one of:
<EOF>
"cat" ...
"cd" ...
"cp" ...
"copyFromLocal" ...
"copyToLocal" ...
"define" ...
"dump" ...
"describe" ...
"explain" ...
"help" ...
"import" ...
"kill" ...
"ls" ...
"mv" ...
"mkdir" ...
"pwd" ...
"quit" ...
"register" ...
"rm" ...
"set" ...
"store" ...
"scriptDone" ...
"" ...
<EOL> ...
";" ...
{code}
Can someone give a look?
Thanks
Craig
> Allow Java packages to be searched for functions.
> -------------------------------------------------
>
> Key: PIG-112
> URL: https://issues.apache.org/jira/browse/PIG-112
> Project: Pig
> Issue Type: Improvement
> Reporter: Craig Macdonald
> Priority: Minor
> Attachments: patch.import
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
> Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
> packages:
> * org.apache.pig.builtin.
> * org.apache.pig.impl.builtin.
> * com.yahoo.pig.yst.sds.ULT.
> Users should be able to add to this packagelist.
> Example in Grunt:
> {noformat}
> include com.pig.function.package.
> {noformat}
> This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
> Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-112) Allow Java packages to be searched for
functions.
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated PIG-112:
--------------------------------
Attachment: import.v2.patch
Hi. Thanks for help Benjamin & Daniel. A working patch is attached.
Pi, there was no very easy way to make the packageImportList non-static, because GruntParser does not have access to PigContext.
C
> Allow Java packages to be searched for functions.
> -------------------------------------------------
>
> Key: PIG-112
> URL: https://issues.apache.org/jira/browse/PIG-112
> Project: Pig
> Issue Type: Improvement
> Reporter: Craig Macdonald
> Priority: Minor
> Attachments: import.v2.patch, patch.import
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
> Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
> packages:
> * org.apache.pig.builtin.
> * org.apache.pig.impl.builtin.
> * com.yahoo.pig.yst.sds.ULT.
> Users should be able to add to this packagelist.
> Example in Grunt:
> {noformat}
> include com.pig.function.package.
> {noformat}
> This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
> Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-112) Allow Java packages to be searched for
functions.
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584578#action_12584578 ]
Benjamin Reed commented on PIG-112:
-----------------------------------
I like import as well. (We actually had it at one time, I don't know what happened to it...)
I think the problem is that the tokenizer in the parser is pretty brain dead. It's probably detecting dump as a package rather than identifier. You may want to define package as a dotted sequence of identifiers. <IDENTIFIER> ("." <IDENTIFIER>)?
> Allow Java packages to be searched for functions.
> -------------------------------------------------
>
> Key: PIG-112
> URL: https://issues.apache.org/jira/browse/PIG-112
> Project: Pig
> Issue Type: Improvement
> Reporter: Craig Macdonald
> Priority: Minor
> Attachments: patch.import
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
> Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
> packages:
> * org.apache.pig.builtin.
> * org.apache.pig.impl.builtin.
> * com.yahoo.pig.yst.sds.ULT.
> Users should be able to add to this packagelist.
> Example in Grunt:
> {noformat}
> include com.pig.function.package.
> {noformat}
> This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
> Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-112) Allow Java packages to be searched for
functions.
Posted by "Pi Song (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584559#action_12584559 ]
Pi Song commented on PIG-112:
-----------------------------
Craig,
I like this concept. I have the same problem when I have to retype long package names everytime I refer to UDFs.
Comments :-
1. This "include" should be specific to each instance of PigContext so packageImportList should be modified to non-static
2. Your example makes use of the token "include" but your code uses "import". I prefer import but I also want to know other people's opinions.
3. I don't understand either why it affects only dump command. Can you regenerate the patch using the latest trunk so that I can help you on this?
> Allow Java packages to be searched for functions.
> -------------------------------------------------
>
> Key: PIG-112
> URL: https://issues.apache.org/jira/browse/PIG-112
> Project: Pig
> Issue Type: Improvement
> Reporter: Craig Macdonald
> Priority: Minor
> Attachments: patch.import
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
> Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
> packages:
> * org.apache.pig.builtin.
> * org.apache.pig.impl.builtin.
> * com.yahoo.pig.yst.sds.ULT.
> Users should be able to add to this packagelist.
> Example in Grunt:
> {noformat}
> include com.pig.function.package.
> {noformat}
> This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
> Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-112) Allow Java packages to be searched for
functions.
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615858#action_12615858 ]
Daniel Dai commented on PIG-112:
--------------------------------
Hi, Craig,
I checked the patch. The problem is that token definition PACKAGE is in front of token definition IDENTIFIER (line 48 in patch). So every token will be interpretted as PACKAGE whenever possible. Simply move it behind will solve the problem. Other code looks good. Can you regenerate the patch?
Thanks
> Allow Java packages to be searched for functions.
> -------------------------------------------------
>
> Key: PIG-112
> URL: https://issues.apache.org/jira/browse/PIG-112
> Project: Pig
> Issue Type: Improvement
> Reporter: Craig Macdonald
> Priority: Minor
> Attachments: patch.import
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
> Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
> packages:
> * org.apache.pig.builtin.
> * org.apache.pig.impl.builtin.
> * com.yahoo.pig.yst.sds.ULT.
> Users should be able to add to this packagelist.
> Example in Grunt:
> {noformat}
> include com.pig.function.package.
> {noformat}
> This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
> Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-112) Allow Java packages to be searched for
functions.
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated PIG-112:
--------------------------------
Patch Info: [Patch Available]
> Allow Java packages to be searched for functions.
> -------------------------------------------------
>
> Key: PIG-112
> URL: https://issues.apache.org/jira/browse/PIG-112
> Project: Pig
> Issue Type: Improvement
> Reporter: Craig Macdonald
> Priority: Minor
> Attachments: import.v2.patch, patch.import
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
> Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
> packages:
> * org.apache.pig.builtin.
> * org.apache.pig.impl.builtin.
> * com.yahoo.pig.yst.sds.ULT.
> Users should be able to add to this packagelist.
> Example in Grunt:
> {noformat}
> include com.pig.function.package.
> {noformat}
> This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
> Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-112) Allow Java packages to be searched for
functions.
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated PIG-112:
--------------------------------
Status: Patch Available (was: Open)
> Allow Java packages to be searched for functions.
> -------------------------------------------------
>
> Key: PIG-112
> URL: https://issues.apache.org/jira/browse/PIG-112
> Project: Pig
> Issue Type: Improvement
> Reporter: Craig Macdonald
> Priority: Minor
> Attachments: import.v2.patch, import.v3.patch, patch.import
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
> Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
> packages:
> * org.apache.pig.builtin.
> * org.apache.pig.impl.builtin.
> * com.yahoo.pig.yst.sds.ULT.
> Users should be able to add to this packagelist.
> Example in Grunt:
> {noformat}
> include com.pig.function.package.
> {noformat}
> This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
> Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-112) Allow Java packages to be searched for
functions.
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616313#action_12616313 ]
Olga Natkovich commented on PIG-112:
------------------------------------
GruntParser gets indiect access to PigContext via PigServer in setParam call. It is a bit ugly but should work.
> Allow Java packages to be searched for functions.
> -------------------------------------------------
>
> Key: PIG-112
> URL: https://issues.apache.org/jira/browse/PIG-112
> Project: Pig
> Issue Type: Improvement
> Reporter: Craig Macdonald
> Priority: Minor
> Attachments: import.v2.patch, patch.import
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Motivation: Why should a user have to supply the fully qualified name to his user defined function, if all the functions he ever uses are in a pre-defined list of packages?
> Currently PigContext has a list of Java packages that are searched for functions. Currently the list consists of the following
> packages:
> * org.apache.pig.builtin.
> * org.apache.pig.impl.builtin.
> * com.yahoo.pig.yst.sds.ULT.
> Users should be able to add to this packagelist.
> Example in Grunt:
> {noformat}
> include com.pig.function.package.
> {noformat}
> This has connections to imports in Java classes, namespaces in XQuery, and the PATH env var in shell scripting.
> Should be trivial to implement, or could be a property.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.