[jira] [Created] (IMPALA-11290) Add a mechanism to allow Impala to maintain 2 clusters during a rolling restart.

["Andrew Sherman (Jira)" <ji...@apache.org>]

Andrew Sherman created IMPALA-11290:
---------------------------------------

             Summary: Add a mechanism to allow Impala to maintain 2 clusters during a rolling restart.
                 Key: IMPALA-11290
                 URL: https://issues.apache.org/jira/browse/IMPALA-11290
             Project: IMPALA
          Issue Type: Bug
            Reporter: Andrew Sherman


A rolling restart is where we restart impala daemons one by one. This can be used to restart an Impala cluster while continuing to run queries. While this is happening we want to prevent different versions of mpala daemons from communicating.

One way to do this would be to publish each impala daemon's version in a statestore topic, so a coordinator could filter to only use executors with its own version. This would also mean that all daemons would have a global picture about the rolling upgrade, and do smart things.

So practically two sub clusters would live at once for some time - we could detect when the new version reaches a practical size (e.g. configurable number of coordinators and executors), and at this point the old impalads could blacklist themselves to make killing them faster.

Maybe we would have options to use the Impala version from version.cc, or to allow the version to be specified as a command line flag.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)