[Impala-CR](cdh5-trunk) IMPALA-3501: ee tests: detect build type and support different timeouts based on the same

["Michael Brown (Code Review)" <ge...@cloudera.org>]

Michael Brown has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/3156

Change subject: IMPALA-3501: ee tests: detect build type and support different timeouts based on the same
......................................................................

IMPALA-3501: ee tests: detect build type and support different timeouts based on the same

Impala compiled with the address sanitizer, or compiled with code
coverage, runs through code paths much slower. This can cause end-to-end
tests that pass on a non-ASAN or non-code coverage build to fail. Some
examples include IMPALA-2721, IMPALA-2973, and IMPALA-3501. These
classes of failures tend always to involve some time-sensitive condition
that fails to succed under such "slow builds".

The works-around in the past have been to simply increase the timeout.
The problem with this approach is that it relaxes conditions for tests
on builds that see the field--i.e., release builds--for builds that
never will--i.e., ASAN and code coverage.

This patch fixes that problem by allowing test authors to set timeout
values based on build type. The author may choose timeouts with a
default value, and different timeouts for either or both so-called "slow
builds": ASAN and code coverage.

We detect the so-called "build type" by inspecting the binary expected
to be at the path under test. This removes the need to make alterations
to Impala itself. The inspection done is to read the DWARF information
in the binary, specially the first compile unit's DW_AT_producer and
DW_AT_name DIE attributes. We employ a heuristic based on these
attributes' values to guess the build type. If we can't determine the
build type, we will assume it's a debug build. More information on this
is in IMPALA-3501.

A quick summary of the changes follows:

1. Move some of the logic in tests.common.skip to tests.common.environ .

2. Add Pyelftools for convenient deserialization of DWARF

3. Our Pyelftools usage requires ordereddict.OrderedDict, which isn't in
   python2.6; also add Monkeypatch to handle this.

4. Add ImpalaBuild and build_type_timeout, the core of the new
   functionality

5. Fix the statestore tests that only fail under code coverage (the
   basis for IMPALA-3501)

Testing:

The tests that were previously, reliably failing under code coverage now
pass. I also ran perfunctory tests of debug, release, and ASAN builds to
ensure our detection of build type is working. This patch will *not*
turn the code coverage builds green; there are other tests that fail,
and fixing all of them here is out of the scope of this patch.

Change-Id: I2b675c04c54e36d404fd9e5a6cf085fb8d6d0e47
---
A infra/python/deps/monkeypatch-0.1rc3.zip
A infra/python/deps/pyelftools-0.23.tar.gz
M infra/python/deps/requirements.txt
A tests/common/environ.py
M tests/common/skip.py
M tests/statestore/test_statestore.py
6 files changed, 234 insertions(+), 29 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/56/3156/1
-- 
To view, visit http://gerrit.cloudera.org:8080/3156
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2b675c04c54e36d404fd9e5a6cf085fb8d6d0e47
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Michael Brown <mi...@cloudera.com>