[Impala-ASF-CR] IMPALA-9629: Add CentOS 8.1 support to bootstrap system.sh

["Laszlo Gaal (Code Review)" <ge...@cloudera.org>]

Hello Zoltan Borok-Nagy, David Knupp, Tim Armstrong, Joe McDonnell, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15623

to look at the new patch set (#11).

Change subject: IMPALA-9629: Add CentOS 8.1 support to bootstrap_system.sh
......................................................................

IMPALA-9629: Add CentOS 8.1 support to bootstrap_system.sh

CentOS 8.1 is a new major version of the CentOS family.
It is now stable and popular enough to start supporting it for Impala
development.

Prepare a raw CentOS 8.1 system to support Impala development and testing.
This should work on a standalone computer, on a virtual machine,
or inside a Docker container.

Details:
- snappy-devel moved to the PowerTools repo, so it needs to be installed
  from there
- CentOS 8 has no default Python version. The bootstrap script installs
  (or configures) Python2 with pip2, then makes them the default via the
  "alternatives" mechanism. The installer is adaptive, it performs only
  the necessary steps, so it works in various environments.
  The installer logic is also shared between bin/bootstrap_system.sh and
  docker/entrypoint.sh
- The toolchain package tag "ec2-centos-8" is added to
  bootstrap_toolchain.py
- For some unknown reason, when the downloaded Maven tarball is extracted
  in a Docker-based test, the "bin" and "boot" directories are created
  with owner-only permissions. The 'impdev' users has no access to the
  maven executable, which then breaks the build.
  This patch forcibly restores the correct permissions on these
  directories; this is a no-op when the extraction happens correctly.
- TOOLCHAIN_ID is bumped to a build that already has CentOS 8 binaries.
- Centos8-specific bootstrap code was added to the Docker-based tests.

Tested:
- ran the Docker-based tests with --base-image=centos:8 to verify the following build
  phases are successful:
  * system prep
  * build
  * dataload
  and that test can start. Passing all tests is was not a requirement for this step,
  although plausible test results (i.e. not all of the tests fail) were.

- ran the Docker-based tests to verify nonregression with --base-image set to the
  following: centos:7, ubuntu:16.04, ubuntu:18.04.
  On centos:7 and ubuntu:16.04 the only failure was IMPALA-9097 (BE tests fail without
  the minicluster running); ubuntu:18.04 showed the same failures as the current upstream
  code.

- passed a core-mode test run on private infrastructure on Centos 7.4

- ran buildall.sh in core mode manually inside a Docker container, simulating a developer
  workflow (prep-build-dataload-test). There were several observed test failures, but
  the workflow itself was run to completion with no problems.

Change-Id: I3df5d48eca7a10219264e3604a4f05f072188e6e
---
M bin/bootstrap_system.sh
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M docker/entrypoint.sh
4 files changed, 116 insertions(+), 14 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/15623/11
-- 
To view, visit http://gerrit.cloudera.org:8080/15623
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3df5d48eca7a10219264e3604a4f05f072188e6e
Gerrit-Change-Number: 15623
Gerrit-PatchSet: 11
Gerrit-Owner: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>