You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Dennis-Mircea Ciupitu (Jira)" <ji...@apache.org> on 2026/04/01 09:30:00 UTC
[jira] [Updated] (FLINK-39370) In-place scaling check only inspects K8s resource spec for adaptive scheduler, ignoring JM running configuration
[ https://issues.apache.org/jira/browse/FLINK-39370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis-Mircea Ciupitu updated FLINK-39370:
------------------------------------------
Description:
h1. Overview
{{NativeFlinkService.supportsInPlaceScaling()}} determines whether a running Flink job supports in-place rescaling by checking whether {{jobmanager.scheduler}} is set to {{Adaptive}} in the {_}observe configuration{_}, which is derived from the Kubernetes {{FlinkDeployment}} resource spec.
However, the adaptive scheduler can be configured through several mechanisms that are *not* reflected in the K8s resource spec:
* Flink's native {{flink-conf.yaml}} (baked into the Docker image or mounted via ConfigMap)
* Environment variables
* Dynamic properties passed via command-line arguments
When the adaptive scheduler is configured through any of these alternative mechanisms, {{observeConfig.get(SCHEDULER)}} returns the default value ({{default}}), and {{supportsInPlaceScaling()}} incorrectly returns {{false}}, even though the JobManager is actually running with the adaptive scheduler and fully supports in-place rescaling.
This forces an unnecessary full restart/redeploy cycle instead of a lightweight in-place scaling operation.
h1. Expected Behavior
The operator should detect that the running JobManager is using the adaptive scheduler (by querying its REST API) and proceed with in-place scaling.
h1. Actual Behavior
The operator only checks the {{FlinkDeployment}} spec configuration, finds no {{jobmanager.scheduler: Adaptive}} entry, and falls back to a full restart.
h1. Proposed Fix
The fix is:
1. Change {{supportsInPlaceScaling()}} from a static check to an instance method that
2. When the K8s resource spec does not indicate the adaptive scheduler, falls back to querying the JobManager's running configuration via the {{JobManagerJobConfigurationHeaders}} REST endpoint. If the JM's actual running config confirms {{{}jobmanager.scheduler: Adaptive{}}}, in-place scaling proceeds.
was:
h1. Overview
{{NativeFlinkService.supportsInPlaceScaling()}} determines whether a running Flink job supports in-place rescaling by checking whether {{jobmanager.scheduler}} is set to {{Adaptive}} in the {_}observe configuration{_}, which is derived from the Kubernetes {{FlinkDeployment}} resource spec.
However, the adaptive scheduler can be configured through several mechanisms that are *not* reflected in the K8s resource spec:
* Flink's native {{flink-conf.yaml}} (baked into the Docker image or mounted via ConfigMap)
* Environment variables
* Dynamic properties passed via command-line arguments
When the adaptive scheduler is configured through any of these alternative mechanisms, {{observeConfig.get(SCHEDULER)}} returns the default value ({{{}Default{}}}), and {{supportsInPlaceScaling()}} incorrectly returns {{{}false{}}}, even though the JobManager is actually running with the adaptive scheduler and fully supports in-place rescaling.
This forces an unnecessary full restart/redeploy cycle instead of a lightweight in-place scaling operation.
h1. Expected Behavior
The operator should detect that the running JobManager is using the adaptive scheduler (by querying its REST API) and proceed with in-place scaling.
h1. Actual Behavior
The operator only checks the {{FlinkDeployment}} spec configuration, finds no {{jobmanager.scheduler: Adaptive}} entry, and falls back to a full restart.
h1. Proposed Fix
Change {{supportsInPlaceScaling()}} from a static check to an instance method that, when the K8s resource spec does not indicate the adaptive scheduler, falls back to querying the JobManager's running configuration via the {{JobManagerJobConfigurationHeaders}} REST endpoint. If the JM's actual running config confirms {{{}jobmanager.scheduler: Adaptive{}}}, in-place scaling proceeds.
> In-place scaling check only inspects K8s resource spec for adaptive scheduler, ignoring JM running configuration
> ----------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-39370
> URL: https://issues.apache.org/jira/browse/FLINK-39370
> Project: Flink
> Issue Type: Bug
> Components: Autoscaler, Kubernetes Operator
> Affects Versions: 1.15.4
> Reporter: Dennis-Mircea Ciupitu
> Priority: Major
> Labels: autoscaling, operator, pull-request-available
> Fix For: 1.15.5
>
>
> h1. Overview
> {{NativeFlinkService.supportsInPlaceScaling()}} determines whether a running Flink job supports in-place rescaling by checking whether {{jobmanager.scheduler}} is set to {{Adaptive}} in the {_}observe configuration{_}, which is derived from the Kubernetes {{FlinkDeployment}} resource spec.
> However, the adaptive scheduler can be configured through several mechanisms that are *not* reflected in the K8s resource spec:
> * Flink's native {{flink-conf.yaml}} (baked into the Docker image or mounted via ConfigMap)
> * Environment variables
> * Dynamic properties passed via command-line arguments
> When the adaptive scheduler is configured through any of these alternative mechanisms, {{observeConfig.get(SCHEDULER)}} returns the default value ({{default}}), and {{supportsInPlaceScaling()}} incorrectly returns {{false}}, even though the JobManager is actually running with the adaptive scheduler and fully supports in-place rescaling.
> This forces an unnecessary full restart/redeploy cycle instead of a lightweight in-place scaling operation.
> h1. Expected Behavior
> The operator should detect that the running JobManager is using the adaptive scheduler (by querying its REST API) and proceed with in-place scaling.
> h1. Actual Behavior
> The operator only checks the {{FlinkDeployment}} spec configuration, finds no {{jobmanager.scheduler: Adaptive}} entry, and falls back to a full restart.
> h1. Proposed Fix
> The fix is:
> 1. Change {{supportsInPlaceScaling()}} from a static check to an instance method that
> 2. When the K8s resource spec does not indicate the adaptive scheduler, falls back to querying the JobManager's running configuration via the {{JobManagerJobConfigurationHeaders}} REST endpoint. If the JM's actual running config confirms {{{}jobmanager.scheduler: Adaptive{}}}, in-place scaling proceeds.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)