You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2017/06/06 17:34:37 UTC
[Impala-ASF-CR] PREVIEW: IMPALA-4703: reservation denial debug action
Tim Armstrong has uploaded a new patch set (#2).
Change subject: PREVIEW: IMPALA-4703: reservation denial debug action
......................................................................
PREVIEW: IMPALA-4703: reservation denial debug action
This is a preview because it depends on the buffer pool execution patch
and I want to get input on the design and scope.
Add debug action to deny reservation increases with some probability.
This allows us to test various scenarios, particularly:
* The case when the node only gets its initial reservation and must
run to completion without increasing its reservation.
* The case when there is some memory pressure and the node sometimes
gets a reservation increase and sometimes doesn't.
E.g. to deny all reservation requests after an ExecNode has opened:
set debug_action=-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0
This was applied to test_spilling. It caught a bug in the PAGG
with spilling string aggregations.
This required some minor extensions to the debug actions.
* Allow debug actions that apply to all ExecNodes if node_id is -1.
* Allow passing parameters to debug actions. The current grammar of the
actions is not well-oriented towards extension, so I resorted to using
@ as a new delimiter.
I also optimised ExecDebugAction() so that it is much faster in the
common case and extended --disable_mem_pools to prevent the buffer pool
from holding onto unused buffers.
Change-Id: Ied39bb091b12156e5dc61b528c6c0cd8de3fe657
---
M be/src/exec/exec-node.cc
M be/src/exec/exec-node.h
M be/src/exec/partitioned-aggregation-node.cc
M be/src/runtime/bufferpool/buffer-allocator.cc
M be/src/runtime/bufferpool/buffer-pool.cc
M be/src/runtime/bufferpool/buffer-pool.h
M be/src/runtime/bufferpool/reservation-tracker.cc
M be/src/runtime/bufferpool/reservation-tracker.h
M be/src/runtime/coordinator-backend-state.cc
M be/src/runtime/debug-options.cc
M be/src/runtime/debug-options.h
M common/thrift/ImpalaInternalService.thrift
M common/thrift/PlanNodes.thrift
M tests/query_test/test_spilling.py
14 files changed, 149 insertions(+), 38 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/7022/2
--
To view, visit http://gerrit.cloudera.org:8080/7022
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ied39bb091b12156e5dc61b528c6c0cd8de3fe657
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>