[Impala-ASF-CR] IMPALA-4858: add more info to MemLimitExceeded errors

["Tim Armstrong (Code Review)" <ge...@cloudera.org>]

Tim Armstrong has uploaded a new patch set (#2).

Change subject: IMPALA-4858: add more info to MemLimitExceeded errors
......................................................................

IMPALA-4858: add more info to MemLimitExceeded errors

This improves the usefulness of MemLimitExceeded errors:
- The host name is included.
- The memory left in the query and process limits is included.
- The approach for deciding whether to print the query or process
  MemTracker hierarchy is more robust: we show the query hierarchy
  only when we're closer to the query limit than the process limit.
  Previously if we were near but not over the process limit it
  printed the query output, which was often useless.
- The detailed output is included in the main error message, rather
  than in the "details" or merged errors. This reduces the impact
  of not logging those details - IMPALA-4697.

Note that the output still does not exactly reflect the state of
the world when the memory limit was exceeded because other threads
will concurrently allocate and release memory.

This also refactors the various related methods slightly to remove
RuntimeState::LogMemLimitExceeded() and reduce the number of
different methods that can log memory limit errors.

Sample output:
--------------
  Memory limit exceeded: Cannot perform aggregation at node with id 1. Failed to allocate 252 bytes for intermediate tuple.
  Fragment ab4765ecacca4d01:23edbb0800000002 could not allocate 252.00 B without exceeding limit.
  Error occurred on backend tarmstrong-box:22001 by fragment ab4765ecacca4d01:23edbb0800000002
  Memory left in process limit: 7.94 GB
  Memory left in query limit: 1.02 MB
  Query(ab4765ecacca4d01:23edbb0800000000): Limit=300.00 MB Total=298.98 MB Peak=299.98 MB
    Fragment ab4765ecacca4d01:23edbb0800000002: Total=105.32 MB Peak=114.14 MB
      AGGREGATION_NODE (id=1): Total=11.46 MB Peak=11.84 MB
      HDFS_SCAN_NODE (id=0): Total=93.73 MB Peak=102.09 MB
      DataStreamSender (dst_id=2): Total=70.02 KB Peak=86.02 KB
      CodeGen: Total=24.83 KB Peak=1.22 MB
    Block Manager: Limit=200.00 MB Total=185.00 MB Peak=186.00 MB
    Fragment ab4765ecacca4d01:23edbb0800000005: Total=185.66 MB Peak=194.66 MB
      AGGREGATION_NODE (id=3): Total=185.48 MB Peak=194.48 MB
      EXCHANGE_NODE (id=2): Total=0 Peak=0
      DataStreamRecvr: Total=160.24 KB Peak=435.45 KB
      DataStreamSender (dst_id=4): Total=688.00 B Peak=688.00 B
      CodeGen: Total=24.28 KB Peak=963.50 KB

Change-Id: Id4ab9d162cf7cd4508bce1efbfccfb4ba97e7355
---
M be/src/common/status.cc
M be/src/common/status.h
M be/src/runtime/collection-value-builder-test.cc
M be/src/runtime/mem-tracker.cc
M be/src/runtime/runtime-state.cc
M be/src/runtime/runtime-state.h
M be/src/util/debug-util.cc
M be/src/util/debug-util.h
8 files changed, 58 insertions(+), 41 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/6537/2
-- 
To view, visit http://gerrit.cloudera.org:8080/6537
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id4ab9d162cf7cd4508bce1efbfccfb4ba97e7355
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>