[jira] [Commented] (IMPALA-539) Impala should gather final runtime profile from fragments for aborted/cancelled query

["Michael Ho (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692475#comment-16692475 ] 

Michael Ho commented on IMPALA-539:
-----------------------------------

Ideally, we should always wait for all backend completion even in the case of cancellation below. Of course, this may be problematic if a backend is unresponsive for whatever reason so we can wait with a timeout once the cancellation RPCs have been issued.
{noformat}
void Coordinator::HandleExecStateTransition(
    const ExecState old_state, const ExecState new_state) {
  static const unordered_map<ExecState, const char *> exec_state_to_event{
    {ExecState::EXECUTING,        "Executing"},
    {ExecState::RETURNED_RESULTS, "Last row fetched"},
    {ExecState::CANCELLED,        "Execution cancelled"},
    {ExecState::ERROR,            "Execution error"}};
  if (old_state == new_state) return;
  // Once we enter a terminal state, we stay there, guaranteeing this code runs only once.
  DCHECK_EQ(old_state, ExecState::EXECUTING);
  // Should never transition to the initial state.
  DCHECK_NE(new_state, ExecState::EXECUTING);
  // Can't transition until the exec RPCs are no longer in progress. Otherwise, a
  // cancel RPC could be missed, and resources freed before a backend has had a chance
  // to take a resource reference.
  DCHECK(exec_rpcs_complete_barrier_ != nullptr &&
      exec_rpcs_complete_barrier_->pending() <= 0) << "exec rpcs not completed";

  query_events_->MarkEvent(exec_state_to_event.at(new_state));
  // This thread won the race to transitioning into a terminal state - terminate
  // execution and release resources.
  ReleaseExecResources();
  if (new_state == ExecState::RETURNED_RESULTS) {
    // TODO: IMPALA-6984: cancel all backends in this case too.
    WaitForBackends(); <<-----
  } else {
    CancelBackends();  <<----- should wait for backend completion too in this case.
  }
  ReleaseAdmissionControlResources();
  // Can compute summary only after we stop accepting reports from the backends. Both
  // WaitForBackends() and CancelBackends() ensures that.
  // TODO: should move this off of the query execution path?
  ComputeQuerySummary();
{noformat}

> Impala should gather final runtime profile from fragments for aborted/cancelled query
> -------------------------------------------------------------------------------------
>
>                 Key: IMPALA-539
>                 URL: https://issues.apache.org/jira/browse/IMPALA-539
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>    Affects Versions: Impala 1.1
>            Reporter: Alan Choi
>            Priority: Minor
>              Labels: query-lifecycle
>
> For cancelled/aborted queries, the final runtime profile from plan fragments are not recorded and reported by the coordinator. For short running queries, we won't see any profile at all because no intermediate runtime profile has been sent.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org