You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@quickstep.apache.org by jianqiao <gi...@git.apache.org> on 2016/10/13 01:31:35 UTC
[GitHub] incubator-quickstep pull request #113: Optimizer changes for the LIPFilter f...
GitHub user jianqiao opened a pull request:
https://github.com/apache/incubator-quickstep/pull/113
Optimizer changes for the LIPFilter feature.
There will be three PRs for adding the _LIPFilter_ feature: optimizer changes / execution generator changes / relational operator changes. This is the first PR that does optimizer changes.
LIP (lookahead information passing) filter is an improved semi-join optimization technique that pushes "future selectivities" down a left-deep query plan tree. For linear / bush trees, we can also search for the left-deep pattern in the tree and apply the LIP filter optimization.
With LIP, the overall SSB benchmark (scale factor 100) performance will be improved from ~19 seconds to ~12 seconds on the cloudlab machine. The TPC-H benchmark performance will also be improved greatly (e.g. `Q02 6.7s -> 0.7s, Q17 166s -> 1.7s, Q20 55s -> 1.7s`).
Main changes in this PR:
(1) Rewrote `StarSchemaHashJoinOrderOptimization`. This changed some of the TPC-H query plans. One or two SSB query plans were also slightly changed due to an adjustment in `StarSchemaSimpleCostModel`. Nevertheless we will achieve an overall performance of ~12 seconds for the SSB benchmark when the full feature is there.
(2) Added a `LIPFilterConfiguration` data structure to store the LIP filter information during query optimization.
(3) Implemented optimizer rule `AttachLIPFilters` to decide and attach LIP filters to the physical query plan.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-quickstep lip-refactor-optimizer
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-quickstep/pull/113.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #113
----
commit 8740f50a710d0390155dc3a9a68dbbc2abe2cb7f
Author: Jianqiao Zhu <ji...@cs.wisc.edu>
Date: 2016-09-07T18:20:43Z
Optimizer changes for the LIPFilter feature.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-quickstep issue #113: Optimizer changes for the LIPFilter feature.
Posted by jianqiao <gi...@git.apache.org>.
Github user jianqiao commented on the issue:
https://github.com/apache/incubator-quickstep/pull/113
@pateljm Thanks! Will sit together with Harshad tomorrow to go through the code once more. Perhaps it's safer to merge the PR after that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-quickstep pull request #113: Optimizer changes for the LIPFilter f...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-quickstep/pull/113
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-quickstep issue #113: Optimizer changes for the LIPFilter feature.
Posted by pateljm <gi...@git.apache.org>.
Github user pateljm commented on the issue:
https://github.com/apache/incubator-quickstep/pull/113
@jianqiao Nice! LGTM. If you squash the commits to one and rebase, I can merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---