[jira] [Created] (IMPALA-13294) Add option to use long polling for get_state/GetOperationStatus

["Joe McDonnell (Jira)" <ji...@apache.org>]

Joe McDonnell created IMPALA-13294:
--------------------------------------

             Summary: Add option to use long polling for get_state/GetOperationStatus
                 Key: IMPALA-13294
                 URL: https://issues.apache.org/jira/browse/IMPALA-13294
             Project: IMPALA
          Issue Type: Task
          Components: Backend, Clients
    Affects Versions: Impala 4.5.0
            Reporter: Joe McDonnell
            Assignee: Joe McDonnell


Currently, Impala does an execute call, then the client polls waiting for the operation to finish (or error out). The client sleeps between polls, and this sleep time can be a substantial percentage of a short query's execution time.

Long polling allows status RPCs like HS2's GetOperationStatus and Beeswax's get_state to wait for query completion on the server side for a configurable amount of time. If the query completes during that time, the client can be notified immediately. If the query does not complete, then the wait times out and the client does a new status RPC.

Clients can adjust to this functionality by keeping track of how much time was spent in the status RPC, then only doing a client side sleep if the RPC was shorter than the desired sleep time. This allows a client to maintain its old behavior with old Impalas that don't have long polling while avoiding unnecessary sleeps when using long polling.

Hive has used long polling for a long time (see HIVE-5217).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org