You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@sdap.apache.org by "Kevin Marlis (Jira)" <ji...@apache.org> on 2025/06/06 17:23:00 UTC

[jira] [Created] (SDAP-537) Dynamic ingester scaling based on RMQ queue size

Kevin Marlis created SDAP-537:
---------------------------------

             Summary: Dynamic ingester scaling based on RMQ queue size
                 Key: SDAP-537
                 URL: https://issues.apache.org/jira/browse/SDAP-537
             Project: Apache Science Data Analytics Platform
          Issue Type: New Feature
            Reporter: Kevin Marlis


Dynamically scaling ingester pods on an as needed basis (within the confines of available node resources) is possible, but not with Kubernetes HPA. [KEDA|[https://keda.sh/]] looks to be an open source autoscaler that works with external metrics, like RMQ queue size.

We would need to explore the maturity of KEDA (they claim to be production grade) and see if it's a good fit for our needs or if there's better alternatives.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)