You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/19 08:30:45 UTC
[GitHub] [airflow] dstandish opened a new issue #13181: keda scaledobject not enabled even though keda enabled in helm config
dstandish opened a new issue #13181:
URL: https://github.com/apache/airflow/issues/13181
In brand new cluster using k3d locally, I first installed keda:
```bash
helm install keda \
--namespace keda kedacore/keda \
--version "v1.5.0"
```
Next, I installed airflow using this config:
```yaml
executor: CeleryExecutor
defaultAirflowTag: 2.0.0-python3.7
airflowVersion: 2.0.0
workers:
keda:
enabled: true
persistence:
enabled: false
pgbouncer:
enabled: true
```
I think this should create a scaled object `airflow-worker`.
But it does not.
As a result of course no autoscaling occurs.
But, when I applied this scaled object definition, the scaledobject was created and immediately autascaling started working --- the worker pod terminated.
```yaml
apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
name: airflow-worker
labels:
tier: airflow
component: worker-horizontalpodautoscaler
release: airflow
chart: "airflow-master"
heritage: Helm
deploymentName: airflow-worker
spec:
scaleTargetRef:
deploymentName: airflow-worker
pollingInterval: 30 # Optional. Default: 30 seconds
cooldownPeriod: 300 # Optional. Default: 300 seconds
maxReplicaCount: 100 # Optional. Default: 100
triggers:
- type: postgresql
metadata:
targetQueryValue: "1"
connection: AIRFLOW_CONN_AIRFLOW_DB
query: "SELECT ceil(COUNT(*)::decimal / 16) FROM task_instance WHERE state='running' OR state='queued'"
```
@turbaszek and @dimberman you may have insight ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on issue #13181: keda scaledobject not created even though keda enabled in helm config
Posted by GitBox <gi...@apache.org>.
dstandish commented on issue #13181:
URL: https://github.com/apache/airflow/issues/13181#issuecomment-748449759
figured it out, created pr
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj closed issue #13181: keda scaledobject not created even though keda enabled in helm config
Posted by GitBox <gi...@apache.org>.
mik-laj closed issue #13181:
URL: https://github.com/apache/airflow/issues/13181
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org