[jira] [Commented] (QUICKSTEP-76) LIP Filter Does not Work in the distributed version

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/QUICKSTEP-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16013609#comment-16013609 ] 

ASF GitHub Bot commented on QUICKSTEP-76:
-----------------------------------------

GitHub user zuyu opened a pull request:

    https://github.com/apache/incubator-quickstep/pull/249

    QUICKSTEP-76: Enabled LIP in the distributed version.

    Assigned to @jianqiao.
    
    This PR enables LIP feature in the distributed version. By computing the LIP filter group (aka equivalence class) based on the LIP filter usages in a LIP deployment, any work orders that use a LIP filter from the same group will be scheduled to the same node.
    
    In all SSB queries, since there is only one LIP filter group, when enabling LIP feature, all work orders that use any LIP will be scheduled to one node.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zuyu/incubator-quickstep lip-dist

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-quickstep/pull/249.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #249
    
----
commit 74c1b9daf8400a20b76701638e6ed6c99111264e
Author: Zuyu Zhang <zu...@apache.org>
Date:   2017-05-06T00:53:09Z

    QUICKSTEP-76: Enabled LIP in the distributed version.

----


> LIP Filter Does not Work in the distributed version
> ---------------------------------------------------
>
>                 Key: QUICKSTEP-76
>                 URL: https://issues.apache.org/jira/browse/QUICKSTEP-76
>             Project: Apache Quickstep
>          Issue Type: Bug
>          Components: Distributed Query Execution
>            Reporter: Zuyu Zhang
>         Attachments: test.sql
>
>
> In the distributed version where Ensemble are consist of multiple Shiftboss, a query w/ non-partitioned hash join whose probe relation is non-stored may get incorrect result, because a SelectWorkOrder on a block from the probe relation may be scheduled on a Shiftboss other than the one that has executed BuildHashJoin. Thus the LIP filter in the SelectWorkOrder has not initialized by BuildHashJoin, and would incorrectly produce no outputs.
> There is a quick fix to schedule SelectWorkOrder on the probe relation only on the Shiftboss that has executed BuildHashJoin.
> As for a partitioned hash join, there are two cases. If the stored probe relation does not need repartition, no changes needed (as no SelectWorkOrder needed). Otherwise, each SelectWorkOrder should be scheduled on the Shiftboss that has executed BuildHashWorkOrder with the same partition id.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)