[jira] [Resolved] (HIVE-8220) Refactor multi-insert code such that plan splitting and task generation are modular and reusable [Spark Branch]

["Xuefu Zhang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xuefu Zhang resolved HIVE-8220.
-------------------------------
    Resolution: Won't Fix

No needed with HIVE-8118.

> Refactor multi-insert code such that plan splitting and task generation are modular and reusable [Spark Branch]
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-8220
>                 URL: https://issues.apache.org/jira/browse/HIVE-8220
>             Project: Hive
>          Issue Type: Improvement
>          Components: Spark
>            Reporter: Xuefu Zhang
>              Labels: Spark-M1
>
> This is a followup for HIVE-7053. Currently the code to split the operator tree and to generate tasks is mingled and thus hard to understand and maintain. Logically the two seems independent. This can be improved by modulizing both. The following might be helpful:
> {code}
> @Override
> protected void generateTaskTree(List<Task<? extends Serializable>> rootTasks, ParseContext pCtx,
>       List<Task<MoveWork>> mvTask, Set<ReadEntity> inputs, Set<WriteEntity> outputs)
>       throws SemanticException {
> // 1. Identify if the plan is for multi-insert and split the plan if necessary
> List<Set<Operator>> operatorSets = multiInsertSplit(...);
> // 2. For each operator set, generate a task.
> for (Set<Operator> topOps : operatorSets) {
>   SparkTask task = generateTask(topOps);
>   ...
> }
> // 3. wire up the tasks
> ...
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)