[jira] [Created] (IMPALA-14445) Improve error message for RPC backpressure and other KRPC error messages

["Joe McDonnell (Jira)" <ji...@apache.org>]

Joe McDonnell created IMPALA-14445:
--------------------------------------

             Summary: Improve error message for RPC backpressure and other KRPC error messages
                 Key: IMPALA-14445
                 URL: https://issues.apache.org/jira/browse/IMPALA-14445
             Project: IMPALA
          Issue Type: Task
          Components: Distributed Exec
    Affects Versions: Impala 5.0.0
            Reporter: Joe McDonnell


On a cluster, we saw queries failing with errors like this:
{noformat}
Exec() rpc failed: Remote error: Service unavailable: ExecQueryFInstances request on impala.ControlService from 1.2.3.4:12345 dropped due to backpressure. The service queue contains 0 items out of a maximum of 2147483647; memory consumption is 0.{noformat}
In this case, the executor is rejecting a message from the coordinator (1.2.3.4:12345 is the IP:port of a coordinator) and the query is failing due to that. The error message doesn't make it clear that the executor is the one rejecting the message, so this can lead to focusing on the coordinator rather than tracking down the executor. Also, the message should include the executor hostname to make it easy to know where to look.

The executor black-listing / query retry logic constructs a better message for the retryable status, but we don't use that for backpressure.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)