You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@kudu.apache.org by "Yuqi Du (Code Review)" <ge...@cloudera.org> on 2023/03/06 09:31:02 UTC

[kudu-CR] [java] KUDU-3455 Improve space complexity about prune hash partitions for in-list predicate

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19568

to look at the new patch set (#3).

Change subject: [java] KUDU-3455 Improve space complexity about prune hash partitions for in-list predicate
......................................................................

[java] KUDU-3455 Improve space complexity about prune hash partitions for in-list predicate

Pruning hash partitions for in-list predicate at java-client, the logic
codes has a high space complexity, and it may cause java-client out
of memory. At the same time, there are many deep copys for 'PartialRow',
those make the method a little slow.

This patch fixes the problem and provide a recursive algorithm, that
uses a method like 'deep first search' to pick all combinations and
try to release PartialRow objects ASAP. And improve speed by reducing
lots of copys.

Change-Id: Icd6f213cb705e1b2a001562cc7cebe4164281723
---
M java/kudu-client/src/main/java/org/apache/kudu/client/PartitionPruner.java
M java/kudu-client/src/test/java/org/apache/kudu/client/TestPartitionPruner.java
2 files changed, 359 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/68/19568/3
-- 
To view, visit http://gerrit.cloudera.org:8080/19568
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Icd6f213cb705e1b2a001562cc7cebe4164281723
Gerrit-Change-Number: 19568
Gerrit-PatchSet: 3
Gerrit-Owner: Yuqi Du <sh...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
Gerrit-Reviewer: Yuqi Du <sh...@gmail.com>