[jira] Updated: (PIG-111) Configuration of Pig

["Pi Song (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pi Song updated PIG-111:
------------------------

    Attachment: PIG_111_v10.patch

This is such a tiring job. I have generated a new patch in-sync with the current trunk.

FIXED: PigContextTest failed because of the alias "input" which has become a reserved word in Pig after Streaming. I feel "input", "output", and "error" reserved words are too common alias names that people would use in data processing work

NOT FIXED: TestStreaming failed should be because of the PIG-181 issue. All the errors on my machine are from shipping and all the error messages look like exactly like what happens in *PIG-181 (null reference)* ==> Should be fixed as a part of PIG-181

*Alan*, I agree that we should have a translation layer from generic config to Hadoop specific config. As for the time being, we still have only Hadoop + local backend, missing it shouldn't cause too much trouble. I suggest trying to get this in first and we will get HConfiguration back soon. 
Reasons:-
1. This patch has been around for too long and took too much people's effort.
2. It's a main road block for other stuffs that require configuration
3. Currently so many basic configurations aren't working = Very bad for users especially first-timers.

PLEASE TEST WELL BEFORE COMMIT

> Configuration of Pig
> --------------------
>
>                 Key: PIG-111
>                 URL: https://issues.apache.org/jira/browse/PIG-111
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Craig Macdonald
>            Assignee: Stefan Groschupf
>         Attachments: after.png, before.png, config.patch.1502, PIG-111-v04.patch, PIG-111-v05.patch, PIG-111-v06.patch, PIG-111_v_3_sg.patch, PIG-111_v_7_r633244M.patch, PIG-111_v_8_r633244M.patch, PIG-111_v_9_r641081.patch, PIG-93-v01.patch, PIG-93-v02.patch, PIG_111_v10.patch, TEST-org.apache.pig.test.PigContextTest.txt, TEST-org.apache.pig.test.TestStreaming.txt
>
>
> This JIRA discusses issues relating to the configuration of Pig.
> Uses cases:
>  
> 1. I want to configure Pig programatically from Java
>  Motivation: pig can be embedded from another Java program, and configuration should be accessible to be set by the client code
> 2. I want to configure Pig from the command line
> 3. I want to configure Pig from the Pig shell (Grunt)
> 4. I want Pig to remember my configuration for every Pig session
>  Motivation: to save me typing in some configuration stuff every time.
> 5. I want Pig to remember my configuration for this script.
>  Motivation: I must use a common configuration for 50% of my Pig scripts - can I share this configuration between scripts.
> Current Status: 
>  * Pig uses System properties for some configuration
>  * A configuration properties object in PigContext is not used.
>  * pigrc can contain properties
>  * Configuration properties can not be set from Grunt
> Proposed solutions to use cases:
> 1. Configuration should be set in PigContext, and accessible from client code.
> 2. System properties are copied to PigContext, or can be specified on the command line (duplication with System properties)
> 3. Allow configuration properties to be set using the "set" command in Grunt
> 4. Pigrc can contain properties. Is this enough, or can other configuration stuff be set, eg aliases, imports, etc.
> 5. Add an include directive to pig, to allow a shared configuration/Pig script to be included.
> Connections to Shell scripting: 
>  * The source command in Bash allows another bash script file to be included - this allows shared variables to be set in one file shared between a set of scripts.
>  * Aliases can be set, according to user preferences, etc.
>  * All this can be done in your .bashrc file
> Issues: 
>  * What happens when you change a property after the property has been read?
>  * Can Grunt read a pigrc containing various statements etc before the PigServer is completely configured?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.