You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Lefty Leverenz (JIRA)" <ji...@apache.org> on 2017/09/25 05:44:01 UTC
[jira] [Comment Edited] (HIVE-16867) Extend shared scan optimizer
to reuse computation from other operators
[ https://issues.apache.org/jira/browse/HIVE-16867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178542#comment-16178542 ]
Lefty Leverenz edited comment on HIVE-16867 at 9/25/17 5:43 AM:
----------------------------------------------------------------
Doc note: This adds *hive.optimize.shared.work* to HiveConf.java, so it needs to be documented in the wiki for release 3.0.0.
HIVE-17542 appends "Tez only" to the parameter description, so this belongs in the Tez section of Configuration Properties:
* [Configuration Properties -- Tez | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-Tez]
Added a TODOC3.0 label.
Edit (25/Sep/17): This also removes *hive.optimize.shared.scan* which was added by HIVE-16602 for release 3.0.0.
was (Author: lefty@hortonworks.com):
Doc note: This adds *hive.optimize.shared.work* to HiveConf.java, so it needs to be documented in the wiki for release 3.0.0.
HIVE-17542 appends "Tez only" to the parameter description, so this belongs in the Tez section of Configuration Properties:
* [Configuration Properties -- Tez | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-Tez]
Added a TODOC3.0 label.
> Extend shared scan optimizer to reuse computation from other operators
> ----------------------------------------------------------------------
>
> Key: HIVE-16867
> URL: https://issues.apache.org/jira/browse/HIVE-16867
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
> Affects Versions: 3.0.0
> Reporter: Jesus Camacho Rodriguez
> Assignee: Jesus Camacho Rodriguez
> Labels: TODOC3.0
> Fix For: 3.0.0
>
> Attachments: HIVE-16867.01.patch, HIVE-16867.02.patch, HIVE-16867.03.patch, HIVE-16867.04.patch, HIVE-16867.patch
>
>
> Follow-up of the work in HIVE-16602.
> HIVE-16602 introduced an optimization that identifies scans on input tables that can be merged so the data is read only once.
> This extension to that rule allows to reuse the computation that is done in the work containing those scans. In particular, we traverse both parts of the plan upstream and reuse the operators if possible.
> Currently, the optimizer will not go beyond the output edge(s) of that work. Follow-up extensions might remove this limitation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)