[jira] [Created] (FLINK-39390) Add Extensibility to Autoscaler Components via the Plugin Mechanism

["Dennis-Mircea Ciupitu (Jira)" <ji...@apache.org>]

Dennis-Mircea Ciupitu created FLINK-39390:
---------------------------------------------

             Summary: Add Extensibility to Autoscaler Components via the Plugin Mechanism
                 Key: FLINK-39390
                 URL: https://issues.apache.org/jira/browse/FLINK-39390
             Project: Flink
          Issue Type: Improvement
          Components: Autoscaler, Kubernetes Operator
    Affects Versions: 1.15.4
            Reporter: Dennis-Mircea Ciupitu
             Fix For: 1.15.5


h1. Summary

Enable plugin-based extensibility for the Flink Autoscaler’s core components ({{{}ScalingRealizer{}}}, {{{}AutoScalerStateStore{}}}, {{{}AutoScalerEventHandler{}}}, {{{}KubernetesJobAutoScalerContext{}}}) using the existing Flink plugin mechanism, consistent with how {{{}FlinkResourceListener{}}}, {{{}FlinkResourceMutator{}}}, and {{FlinkResourceValidator}} are already extensible today.
h1. Motivation

The Flink Autoscaler documentation already describes these interfaces as extensible, and the Autoscaler Standalone module demonstrates that alternative implementations can be wired in. However, for the built-in Kubernetes Operator autoscaler, there is currently no supported way to provide custom implementations of these components without forking the operator itself.

This is cumbersome and not straightforward for users who need to customize autoscaler behavior like persisting autoscaler state to an external database, routing scaling events to a custom monitoring system, or adjusting how scaling decisions are applied to Flink resources. Today, such customizations require maintaining a full fork of the operator, which creates a significant maintenance burden and makes it difficult to stay up to date with upstream releases.

The operator already has a well-established plugin mechanism for other extension points (listeners, mutators, validators). Extending this same mechanism to the autoscaler components would provide a consistent, low-friction extensibility model and align the actual capabilities with what the documentation already suggests is possible.

h1. Objective

Add plugin-based extensibility to the built-in Kubernetes Operator autoscaler so that:
 * Custom implementations of {{ScalingRealizer}}, {{AutoScalerStateStore}}, {{AutoScalerEventHandler}}, and {{KubernetesJobAutoScalerContext}} can be provided by placing plugin JARs in the operator’s plugin directory, with no forking required.
 * The discovery follows the same plugin mechanism already used by other operator extension points (listeners, mutators, validators), ensuring a consistent extensibility model.
 * When no custom plugin is found, the existing default Kubernetes implementations are used, preserving full backward compatibility.
 * Documentation is updated to guide users through creating and deploying custom autoscaler plugins.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)