You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Jesus Camacho Rodriguez (JIRA)" <ji...@apache.org> on 2016/06/15 10:29:09 UTC
[jira] [Comment Edited] (HIVE-14002) Extend limit propagation to
subsequent RS operators
[ https://issues.apache.org/jira/browse/HIVE-14002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331503#comment-15331503 ]
Jesus Camacho Rodriguez edited comment on HIVE-14002 at 6/15/16 10:29 AM:
--------------------------------------------------------------------------
Thanks for reviewing [~ashutoshc]. In fact, _operator.acceptLimitPushdown()_ is used to bail out if the operator does not accept the limit pushdown. This is the same logic that was used for pushing it from the Limit operator, thus I would expect that the operators are properly classified and it is sound.
Further, if we cannot backtrack a column from the top RS to the bottom RS, we bail out too (in fact, the new _checkKeys_ method checks that).
was (Author: jcamachorodriguez):
Thanks for reviewing [~ashutoshc]. In fact, _operator.acceptLimitPushdown()_ is used to bail out if the operator does not accept the limit pushdown. This is the same logic that was used for pushing it from the Limit operator, thus I would expect that it is sound.
Further, if we cannot backtrack a column from the top RS to the bottom RS, we bail out too (in fact, the new _checkKeys_ method does that).
> Extend limit propagation to subsequent RS operators
> ---------------------------------------------------
>
> Key: HIVE-14002
> URL: https://issues.apache.org/jira/browse/HIVE-14002
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
> Affects Versions: 2.2.0
> Reporter: Nita Dembla
> Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14002.01.patch, HIVE-14002.patch
>
>
> In some occasions, for instance when RS dedup does not kick in, it is useful to propagate the limit to subsequent RS operators, as this will reduce intermediary results and impact performance. This issue covers that extension.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)