[jira] [Commented] (PIG-3175) Refactor PigServer and GruntServer

["Prashant Kommireddi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13575587#comment-13575587 ] 

Prashant Kommireddi commented on PIG-3175:
------------------------------------------

We could probably delegate the following functionality to a helper class PigExecutor

1. Parse
2. Build (lp, pp)
3. Launch/Execute

This would handle both PigServer and GruntServer requests, the advantage being GruntServer can get to it through PigServer and avoid circular dependencies.

{code}
   public PigServer(PigContext context, boolean connect) throws ExecException {
        this.pigContext = context;
        currDAG = new Graph(false);
         
        // other statements

        addJarsFromProperties();

        // create PigExecutor
        this.pigExecutor = new PigExecutor(this.pigContext, this.currDAG, this.jobName);
    }
{code}

PigExecutor would look something like this.

{code}
/**
 * This class handles requests from PigServer for parsing of scripts, building
 * the logical and physical plan. It also launches Pig jobs for execution on the
 * cluster (local or MR mode).
 */
public class PigExecutor {
    
    private Graph currDAG;
    private final PigContext pigContext;
    private final String jobName;
    
    private static final Log log = LogFactory.getLog(PigExecutor.class);

    public PigExecutor(PigContext pigContext, Graph currDAG, String jobName) {
        this.pigContext = pigContext;
        this.currDAG = currDAG;
        this.jobName = jobName;
    }

    /**
     * Compile and execute the current plan.
     * @return
     * @throws IOException
     */
    public PigStats execute(String jobPriority) throws IOException {
     
    }

    public PigStats executeCompiledLogicalPlan() throws ExecException, FrontendException {
 
   }
    
    /**
     * A common method for launching the jobs according to the physical plan
     * @param pp The physical plan
     * @param jobName A String containing the job name to be used
     * @return The PigStats object
     * @throws ExecException
     * @throws FrontendException
     */
    public PigStats launchPlan(PhysicalPlan pp, String jobName) throws ExecException, FrontendException {

    }

    /**
     * NOTE: For testing only. Don't use.
     * @throws IOException
     */
    @SuppressWarnings("unused")
    private LogicalPlan buildLp() throws IOException {

    }

    public PhysicalPlan compilePp() throws FrontendException {
        // translate lp to physical plan
    }

    public void setCurrentDAG(Graph currDAG) {
        this.currDAG = currDAG;
    }

    /**
     * This method parses the scripts and builds the LogicalPlan. This method
     * should be followed by {@link PigServer#executeBatch(boolean)} with
     * argument as false. Do Not use {@link PigServer#executeBatch()} after
     * calling this method as that will re-parse and build the script.
     * 
     * @throws IOException
     */

    public void parseAndBuild(boolean isBatchOn) throws IOException {

    }
    
    /**
     * Submits a batch of Pig commands for execution. Parse and build of script
     * should be skipped if user called {@link PigServer#parseAndBuild()}
     * before. Pass false as an argument in which case.
     * 
     * @param parseAndBuild
     * @return
     * @throws IOException
     */
    public PigStats executeBatch(boolean parseAndBuild, String jobPriority) throws IOException {

    }

    /** 
     * Parses Pig commands in either interactive mode or batch mode. 
     * In interactive mode, executes the plan right away whenever a 
     * STORE command is encountered.
     *
     * @throws IOException, ParseException
     */
    
    public int[] parseStopOnError(GruntParser gruntParser, boolean sameBatch) throws IOException, ParseException {

    }

    private int[] executeBatch(GruntParser gruntParser) throws IOException {

    }

    private void discardBatch(PigServer pigServer) throws IOException {

    }

}
{code}

Feedback appreciated. I will have a patch ready soon.
                
> Refactor PigServer and GruntServer
> ----------------------------------
>
>                 Key: PIG-3175
>                 URL: https://issues.apache.org/jira/browse/PIG-3175
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.10.0
>            Reporter: Prashant Kommireddi
>            Assignee: Prashant Kommireddi
>             Fix For: 0.12
>
>
> At present there is a lot of calls taking place to-from between PigServer and GruntServer. For example, PigServer.registerScript(..) calls GruntServer.parseStopOnError for parsing which in turn calls PigServer.executeBatch(). The code seems more complex than it should be and should be refactored. 
> Also, several methods on PigServer are not clear in what they do and should be well documented (probably in another JIRA).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira