You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Supreeth (JIRA)" <ji...@apache.org> on 2011/07/15 00:22:00 UTC
[jira] [Created] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Need a way to deal with params and param_file in embedded pig in python
-----------------------------------------------------------------------
Key: PIG-2165
URL: https://issues.apache.org/jira/browse/PIG-2165
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Supreeth
I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
Thanks
Supreeth
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144962#comment-13144962 ]
Daniel Dai commented on PIG-2165:
---------------------------------
Unit test pass. test-patch:
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] -1 release audit. The applied patch generated 468 release audit warnings (more than the trunk's current 461 warnings).
No new files added, ignore release audit warning.
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch, PIG-2165-2.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2165:
----------------------------
Attachment: PIG-2165-1.patch
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2165:
----------------------------
Attachment: PIG-2165-2.patch
Add new test case as Ashutosh suggest.
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch, PIG-2165-2.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2165:
----------------------------
Fix Version/s: 0.10
Assignee: Daniel Dai
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2165:
----------------------------
Attachment: PIG-2165-2.patch
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch, PIG-2165-2.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Alan Gates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-2165:
----------------------------
Status: Patch Available (was: Open)
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2165:
----------------------------
Attachment: (was: PIG-2165-2.patch)
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch, PIG-2165-2.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2165:
----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Patch committed to both trunk and 0.10 branch.
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch, PIG-2165-2.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Julien Le Dem (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13065725#comment-13065725 ]
Julien Le Dem commented on PIG-2165:
------------------------------------
I see two options:
- One way would be to use the pig parameters and assign the values to global variables.
{code}pig -p foo=bar myscript.py{code}
and then in the script, the variable foo contains "bar"
- The other would be to populate sys.argv with the command line parameters
{code}sys.argv = [ "pig", "-p", "foo=bar", "myscript.py" ]{code}
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Fix For: 0.10
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Julien Le Dem (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126271#comment-13126271 ]
Julien Le Dem commented on PIG-2165:
------------------------------------
Hi Daniel,
This works but I would rather have the parameters set to global variables in the Interpreter.
Using the pre-processor is less intuitive as parameter values will need different escaping depending of the language or the location of the script where they are used. For example in a string vs not. Assigning to a global variable avoids this kind of issue.
Julien
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144463#comment-13144463 ]
Ashutosh Chauhan commented on PIG-2165:
---------------------------------------
+1 Looks good to me. If its easy enough, it will be good to add e2e test with param file as well, since patch touches upon that part of code.
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-2165:
--------------------------------
Fix Version/s: 0.10
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Fix For: 0.10
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106932#comment-13106932 ]
Daniel Dai commented on PIG-2165:
---------------------------------
Yes, for "$var" in your script, it can be either a bounded variable, or a command line parameter. I treat them the same in patch.
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066048#comment-13066048 ]
Alan Gates commented on PIG-2165:
---------------------------------
I like the idea of only passing the parameters from either the command line or the parameters file. I don't see why the Python script should care about other parameters that will be Pig specific. Whether those parameters are placed in global variables, in sys.argv, or in a separate dictionary (pig_argv?) I don't have an opinion on.
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Fix For: 0.10
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134533#comment-13134533 ]
Daniel Dai commented on PIG-2165:
---------------------------------
Hi, Julien,
PIG-2165-1.patch does use global variable PigContext.params to pass params. What it does is:
1. Put params into PigContext.params when processing command line
2. Adding PigContext.params to do variable binding:
{code}
// plist is the variable array to bind
if (getScriptContext().getPigContext().getParams()!=null) {
for (String param : getScriptContext().getPigContext().getParams()) {
plist.add(param);
}
}
// Use plist to do the binding
{code}
3. For param_file, instead of reading and put into params, I delegate it to ParameterSubstitutionPreprocessor, which already handles parameter_file:
{code}
psp.genSubstitutedFile(in, writer, plist.toArray(params), scriptContext.getPigContext().getParamFiles().toArray(type1));
{code}
4. I also did some refactory in main() to use PigContext.params
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Olga Natkovich (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-2165:
--------------------------------
Fix Version/s: (was: 0.10)
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Attachments: PIG-2165-1.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Julien Le Dem (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106928#comment-13106928 ]
Julien Le Dem commented on PIG-2165:
------------------------------------
Hi Daniel,
Are you suggesting to use the preprocessor to replace $variable with its value passed in "-p variable=value" ?
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126291#comment-13126291 ]
Daniel Dai commented on PIG-2165:
---------------------------------
Thanks Julien. I'm fine with global variable approach. I can submit another patch.
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2165-1.patch
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100777#comment-13100777 ]
Daniel Dai commented on PIG-2165:
---------------------------------
I think we shall stick with the conventional Pig command line options (-p, -m). I don't think we need to invent new ways specific for Pig embedding.
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Fix For: 0.10
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2165) Need a way to deal with params and
param_file in embedded pig in python
Posted by "Julien Le Dem (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066084#comment-13066084 ]
Julien Le Dem commented on PIG-2165:
------------------------------------
I agree, on second thoughts my second option is a bad idea as the script can be launched either through the command line or programmatically through the java API. In that case the only parameters that are available are the ones passed through -p.
There's no good way to reuse sys.argv here, it is not really a good fit as it is a list of strings.
Suggestions:
- a separate dictionary: Pig.getParameters() ?
- placed in global variables
> Need a way to deal with params and param_file in embedded pig in python
> -----------------------------------------------------------------------
>
> Key: PIG-2165
> URL: https://issues.apache.org/jira/browse/PIG-2165
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Supreeth
> Fix For: 0.10
>
>
> I am using embedded pig in python and cannot pass param key value pairs to the python script. The only way to pass params seem to be by passing it in the bind command.
> Is there a plan to have command line parameters to a pig embedded python script? Similar needs for param_file and using the environment variables.
> Thanks
> Supreeth
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira