[Impala-ASF-CR] IMPALA-8572: Log query events before unregister.

["Bharath Vissapragada (Code Review)" <ge...@cloudera.org>]

Hello radford nguyen, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14143

to look at the new patch set (#5).

Change subject: IMPALA-8572: Log query events before unregister.
......................................................................

IMPALA-8572: Log query events before unregister.

Currently, the query events (audits and lineages) are logged as a
part of query unregistration. This delays the event logging in cases
where the Unregister() is delayed by client for some reason (ex: Hue
does not call Unregister until the browser tab is closed) or the client
goes away without calling Unregister and the query timeout kicks in.

This patch moves this event logging to an earlier stage in the query
lifecycle. Moved the event logging related code into ClientRequestState
for easier code refactoring.

The conditions under which the events are logged are slightly
modified by this patch. Without the patch, events are logged for
unsuccessful queries if atleast a single fetch is perfomed. This patch
relaxes this guarantee to log events for any query that reaches
the FINISHED state (rows are available to fetch by the client) and does
not wait for a fetch to be performed. This simplifies the coordinator
state machine by avoid unnecessary synchronization.

Added some test coverage for coordinator side code paths for writing
lineages. fe specific lineage tests only verified the correctness of
lineage created but did not test whether it was being flushed correctly
to the disk.

Change-Id: I639b9c1acb9806b29292cd85be2863688453ca2e
---
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
A testdata/workloads/functional-query/queries/QueryTest/lineage.test
M tests/common/impala_test_suite.py
M tests/common/test_result_verifier.py
M tests/custom_cluster/test_lineage.py
M tests/unittests/test_file_parser.py
M tests/util/test_file_parser.py
10 files changed, 6,059 insertions(+), 239 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/14143/5
-- 
To view, visit http://gerrit.cloudera.org:8080/14143
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I639b9c1acb9806b29292cd85be2863688453ca2e
Gerrit-Change-Number: 14143
Gerrit-PatchSet: 5
Gerrit-Owner: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>