[jira] [Created] (IMPALA-11979) Support option for consistent scan range scheduling across executor invocations with changing IP address

["David Rorke (Jira)" <ji...@apache.org>]

David Rorke created IMPALA-11979:
------------------------------------

             Summary: Support option for consistent scan range scheduling across executor invocations with changing IP address
                 Key: IMPALA-11979
                 URL: https://issues.apache.org/jira/browse/IMPALA-11979
             Project: IMPALA
          Issue Type: Improvement
          Components: be
    Affects Versions: Impala 4.2.0
            Reporter: David Rorke


The scheduler currently assigns scan ranges to executor hosts using a hash ring with the executor IP used as the hash key.  This results in varying scan range assignments across multiple instantiations of an executor if the executor IP changes.  This makes it more difficult to get repeatable scheduling behavior when debugging or performance testing in an environment where executor IPs can be assigned dynamically.

We should support a mode where the scheduler's executor hashing uses a fixed executor identifier that we specify (unique within an executor group).  This could also be useful for potential future optimizations like cache prewarming

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org