[Impala-ASF-CR] IMPALA-8648: Add stress tests for ACID INSERTs/SELECTs

["Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>]

Hello Norbert Luksa, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14444

to look at the new patch set (#5).

Change subject: IMPALA-8648: Add stress tests for ACID INSERTs/SELECTs
......................................................................

IMPALA-8648: Add stress tests for ACID INSERTs/SELECTs

Added different stress tests for ACID operations. We already had one
stress test where an Impala client issued SELECTs while another process
inserted data through Hive. The test checked wether some invariants are
held.

I added the following tests:
* one Impala client inserts data, one Impala client reads and checks
  invariants
* multiple Impala clients issue INSERTs and TRUNCATEs, multiple Impala
  clients issue SELECTs and check invariants
* multiple Impala clients insert data, sometimes failures are injected
  during inserts to create aborted transactions. In the meantime
  multiple Impala clients read the data and check that if they see
  data from aborted transactions

During the tests I found and fixed a temporal deadlock.
CatalogOpExecutor.truncateTable() locked catalog_.versionLock_ and the
HMS ACID lock in a different order than the INSERT statements, it could
lead to a deadlock for 30 seconds, after that TRUNCATE failed.

Also, in the Frontend I switched the order of write id allocation and
locking. From now on at first we lock the table then allocate a write
id.

Change-Id: I066652bfa7d924742af01aef8df4512e00620c7d
---
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/Transaction.java
M fe/src/main/java/org/apache/impala/common/TransactionKeepalive.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M tests/common/impala_test_suite.py
M tests/stress/test_acid_stress.py
7 files changed, 376 insertions(+), 104 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/14444/5
-- 
To view, visit http://gerrit.cloudera.org:8080/14444
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I066652bfa7d924742af01aef8df4512e00620c7d
Gerrit-Change-Number: 14444
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Norbert Luksa <no...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>