[Impala-CR](cdh5-trunk) IMPALA-1346/1590: fix sorter buffer mgmt when spilling

["Tim Armstrong (Code Review)" <ge...@cloudera.org>]

Tim Armstrong has uploaded a new patch set (#2).

Change subject: IMPALA-1346/1590: fix sorter buffer mgmt when spilling
......................................................................

IMPALA-1346/1590: fix sorter buffer mgmt when spilling

The Sorter's memory management logic failed to correctly manage buffers
when spilling. It would try to make use of all buffers in the system,
neglecting to account for other operators' buffer usage.

This patch adjusts the logic so that it handles contention for buffers
so long as it can get enough buffers to make progress. Instead of
precalculating the number of buffers it thinks it should be able to
pin, it just makes a best-effort attempt to pin the initial buffers
as many runs as possible, up to a limit. As long as it can pin three
runs, it can make progress.

Testing:
Added an additional test that failed before the patch without OOM.
An analytic function test that was meant to fail also started succeeding
so I had to adjust the limit there too.

Change-Id: Idfe55cc13c7f2b54cba1d05ade44cbcf6bb573c0
---
M be/src/runtime/buffered-block-mgr.h
M be/src/runtime/sorter.cc
M be/src/runtime/sorter.h
M testdata/workloads/functional-query/queries/QueryTest/spilling.test
4 files changed, 174 insertions(+), 109 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/08/2908/2
-- 
To view, visit http://gerrit.cloudera.org:8080/2908
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idfe55cc13c7f2b54cba1d05ade44cbcf6bb573c0
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>