You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Dan Burkert (Code Review)" <ge...@cloudera.org> on 2017/12/21 05:58:53 UTC
[kudu-CR](branch-1.6.x) KUDU-2231: sparse column predicate can cause excessive data-block reads
Dan Burkert has uploaded this change for review. ( http://gerrit.cloudera.org:8080/8902
Change subject: KUDU-2231: sparse column predicate can cause excessive data-block reads
......................................................................
KUDU-2231: sparse column predicate can cause excessive data-block reads
When scanning with a sparsely-matching predicate, the CFileIterator can
repeatedly materialize non-predicate column blocks multiple times. The
result is huge amounts of CPU wasted in block decoding and poor
performance.
The root cause is that CFileIterator::SeekToOrdinal does not check
whether the currently materialized data block contains the ordinal index
being seeked to. Instead, it throws away the currently prepared blocks
(in CFileIterator::PrepareForNewSeek), and re-materializes the blocks
again.
This commit is a very targeted fix. Since I've had some time to get
familiar with this codepath in the past few days, I've found some things
that I think we could improve and simplify in follow-up commits, which
I've filed as KUDU-2243.
Change-Id: I8eb3be4a809f882ccd80c48612099b2071306ff7
Reviewed-on: http://gerrit.cloudera.org:8080/8869
Tested-by: Kudu Jenkins
Reviewed-by: Dan Burkert <da...@apache.org>
(cherry picked from commit 36ecb300901daf4006fae0c9be5657f2f6127233)
---
M src/kudu/cfile/cfile_reader.cc
M src/kudu/tablet/tablet-pushdown-test.cc
2 files changed, 107 insertions(+), 24 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/02/8902/1
--
To view, visit http://gerrit.cloudera.org:8080/8902
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: branch-1.6.x
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8eb3be4a809f882ccd80c48612099b2071306ff7
Gerrit-Change-Number: 8902
Gerrit-PatchSet: 1
Gerrit-Owner: Dan Burkert <da...@apache.org>
[kudu-CR](branch-1.6.x) KUDU-2231: sparse column predicate can cause excessive data-block reads
Posted by "Jean-Daniel Cryans (Code Review)" <ge...@cloudera.org>.
Jean-Daniel Cryans has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8902 )
Change subject: KUDU-2231: sparse column predicate can cause excessive data-block reads
......................................................................
KUDU-2231: sparse column predicate can cause excessive data-block reads
When scanning with a sparsely-matching predicate, the CFileIterator can
repeatedly materialize non-predicate column blocks multiple times. The
result is huge amounts of CPU wasted in block decoding and poor
performance.
The root cause is that CFileIterator::SeekToOrdinal does not check
whether the currently materialized data block contains the ordinal index
being seeked to. Instead, it throws away the currently prepared blocks
(in CFileIterator::PrepareForNewSeek), and re-materializes the blocks
again.
This commit is a very targeted fix. Since I've had some time to get
familiar with this codepath in the past few days, I've found some things
that I think we could improve and simplify in follow-up commits, which
I've filed as KUDU-2243.
Change-Id: I8eb3be4a809f882ccd80c48612099b2071306ff7
Reviewed-on: http://gerrit.cloudera.org:8080/8869
Tested-by: Kudu Jenkins
Reviewed-by: Dan Burkert <da...@apache.org>
(cherry picked from commit 36ecb300901daf4006fae0c9be5657f2f6127233)
Reviewed-on: http://gerrit.cloudera.org:8080/8902
Reviewed-by: Jean-Daniel Cryans <jd...@apache.org>
---
M src/kudu/cfile/cfile_reader.cc
M src/kudu/tablet/tablet-pushdown-test.cc
2 files changed, 107 insertions(+), 24 deletions(-)
Approvals:
Kudu Jenkins: Verified
Jean-Daniel Cryans: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/8902
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: branch-1.6.x
Gerrit-MessageType: merged
Gerrit-Change-Id: I8eb3be4a809f882ccd80c48612099b2071306ff7
Gerrit-Change-Number: 8902
Gerrit-PatchSet: 2
Gerrit-Owner: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR](branch-1.6.x) KUDU-2231: sparse column predicate can cause excessive data-block reads
Posted by "Jean-Daniel Cryans (Code Review)" <ge...@cloudera.org>.
Jean-Daniel Cryans has posted comments on this change. ( http://gerrit.cloudera.org:8080/8902 )
Change subject: KUDU-2231: sparse column predicate can cause excessive data-block reads
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/8902
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: branch-1.6.x
Gerrit-MessageType: comment
Gerrit-Change-Id: I8eb3be4a809f882ccd80c48612099b2071306ff7
Gerrit-Change-Number: 8902
Gerrit-PatchSet: 1
Gerrit-Owner: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 22 Dec 2017 20:08:20 +0000
Gerrit-HasComments: No