You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Andrew Sherman (Jira)" <ji...@apache.org> on 2022/05/13 20:16:00 UTC
[jira] [Updated] (IMPALA-11290) Add a mechanism to allow Impala to maintain 2 clusters during a rolling restart.
[ https://issues.apache.org/jira/browse/IMPALA-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Sherman updated IMPALA-11290:
------------------------------------
Description:
A rolling restart is where we restart impala daemons one by one. This can be used to restart an Impala cluster while continuing to run queries. While this is happening we want to prevent different versions of mpala daemons from communicating.
One way to do this would be to publish each impala daemon's version in a statestore topic, so a coordinator could filter to only use executors with its own version. This would also mean that all daemons would have a global picture about the rolling upgrade, and do smart things.
So practically two sub clusters would live at once for some time - we could detect when the new version reaches a practical size (e.g. configurable number of coordinators and executors), and at this point the old impalads could blacklist themselves to make killing them faster.
Maybe we would have options to use the Impala version from version.cc, or to allow the version to be specified as a command line flag.
Care would be needed when enabling this feature to avoid unintended consequences.
was:
A rolling restart is where we restart impala daemons one by one. This can be used to restart an Impala cluster while continuing to run queries. While this is happening we want to prevent different versions of mpala daemons from communicating.
One way to do this would be to publish each impala daemon's version in a statestore topic, so a coordinator could filter to only use executors with its own version. This would also mean that all daemons would have a global picture about the rolling upgrade, and do smart things.
So practically two sub clusters would live at once for some time - we could detect when the new version reaches a practical size (e.g. configurable number of coordinators and executors), and at this point the old impalads could blacklist themselves to make killing them faster.
Maybe we would have options to use the Impala version from version.cc, or to allow the version to be specified as a command line flag.
> Add a mechanism to allow Impala to maintain 2 clusters during a rolling restart.
> --------------------------------------------------------------------------------
>
> Key: IMPALA-11290
> URL: https://issues.apache.org/jira/browse/IMPALA-11290
> Project: IMPALA
> Issue Type: Bug
> Reporter: Andrew Sherman
> Priority: Major
>
> A rolling restart is where we restart impala daemons one by one. This can be used to restart an Impala cluster while continuing to run queries. While this is happening we want to prevent different versions of mpala daemons from communicating.
> One way to do this would be to publish each impala daemon's version in a statestore topic, so a coordinator could filter to only use executors with its own version. This would also mean that all daemons would have a global picture about the rolling upgrade, and do smart things.
> So practically two sub clusters would live at once for some time - we could detect when the new version reaches a practical size (e.g. configurable number of coordinators and executors), and at this point the old impalads could blacklist themselves to make killing them faster.
> Maybe we would have options to use the Impala version from version.cc, or to allow the version to be specified as a command line flag.
> Care would be needed when enabling this feature to avoid unintended consequences.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org