[kudu-CR] [compaction] Major Delta Compaction batching to avoid OOM

["Ashwani Raina (Code Review)" <ge...@cloudera.org>]

Hello Zoltan Martonka, Kurt Deschler, Alexey Serbin, Kudu Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/22406

to look at the new patch set (#9).

Change subject: [compaction] Major Delta Compaction batching to avoid OOM
......................................................................

[compaction] Major Delta Compaction batching to avoid OOM

The patch adds logic to do major delta compaction in batches
instead of processing all the delta blocks in one go. The batch
is defined by one or more row updates depending on how much space
those deltas consume in terms of blocks, mutations, maintaining
UNDO lists. For each batch, all the corresponding REDO deltas are
read and corresponding UNDO mutations are created to applied.
After all the delta blocks and their corresponding deltas are applied,
base data is committed.

Main highlights of this change are listed as:
1. Maintains last block pointer to the delta block that was read
   in the last batch.
2. Maintains the next row id to start from when next batch processing
   starts.
3. Keeps major delta compaction object's mode i.e. whether it is running
   in batch mode or normal mode (where no batching is done).
4. Maintains information whether all the delta blocks have been already
   read and processed. This denotes exit condition when running in batch
   mode.

Change-Id: I111d613ec717534df9ba09e5660420b5d6c9fed0
---
M src/kudu/tablet/compaction-highmem-test.cc
M src/kudu/tablet/delta_compaction.cc
M src/kudu/tablet/delta_compaction.h
M src/kudu/tablet/delta_iterator_merger.cc
M src/kudu/tablet/delta_iterator_merger.h
M src/kudu/tablet/delta_store.h
M src/kudu/tablet/deltafile.cc
M src/kudu/tablet/deltafile.h
M src/kudu/tablet/deltamemstore.h
M src/kudu/tablet/diskrowset.cc
10 files changed, 550 insertions(+), 17 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/06/22406/9
-- 
To view, visit http://gerrit.cloudera.org:8080/22406
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I111d613ec717534df9ba09e5660420b5d6c9fed0
Gerrit-Change-Number: 22406
Gerrit-PatchSet: 9
Gerrit-Owner: Ashwani Raina <ar...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>