[GitHub] [airflow] hussein-awala commented on issue #29654: Not able to specify redis as a result backend

["hussein-awala (via GitHub)" <gi...@apache.org>]

hussein-awala commented on issue #29654:
URL: https://github.com/apache/airflow/issues/29654#issuecomment-1442614738

   Hello Arthur,
   
   > Error: Failed to render chart: exit status 1: Error: values don't meet the specifications of the schema(s) in the following chart(s):
     airflow:
       data.resultBackendConnection: pass is required
       data.resultBackendConnection: sslmode is required
       data.resultBackendConnection: Additional property password is not allowed
     Error: plugin "diff" exited with error
   
   We build the Url from the [values](https://github.com/apache/airflow/blob/main/chart/templates/secrets/result-backend-connection-secret.yaml#L46-L46), so you should provide it as the following to create the secret:
   ```yaml
   data:
     resultBackendConnection:
       protocol: redis
       host: my.host.io
       port: 6379
       db: airflow
       user: airflow_user
       pass: airflow_pass
       sslmode: disable
   ```
   This creates a new secret:
   ```yaml
   kind: Secret
   apiVersion: v1
   metadata:
     name: release-name-airflow-result-backend
     labels:
       tier: airflow
       release: release-name
       chart: airflow
       heritage: Helm
   type: Opaque
   data:
     connection: "ZGIrcmVkaXM6Ly9haXJmbG93X3VzZXI6YWlyZmxvd19wYXNzQG15Lmhvc3QuaW86NjM3OS9haXJmbG93"
   ```
   And here is the raw connection value
   ```bash
   $ echo ZGIrcmVkaXM6Ly9haXJmbG93X3VzZXI6YWlyZmxvd19wYXNzQG15Lmhvc3QuaW86NjM3OS9haXJmbG93 | base64 -d
   db+redis://airflow_user:airflow_pass@my.host.io:6379/airflow
   ```
   And if you prefer to avoid providing the password in the values or creating the secret by helm, you can create the secret with the same structure and provide its name to `.Values.data.resultBackendSecretName` which is the recommended way.
   
   > Although when I am doing so, I have error in my DagProcessor:
   Liveness probe failed: You have configured a result_backend of redis://my.host.io:6379/1, it is highly recommended to use an alternative result_backend (i.e. a database). 
   No alive jobs found.
   
   I think it's a problem in the provided url where the schema should be `db+redis` instead of `redis`.
   
   PS: if you create the secret and provide its name to `.Values.data.resultBackendSecretName` or if you fill all the fields of `.Values.data.resultBackendConnection` as I did in my example, you don't need to provide the config `config.celery.config` where the connection will [be added](https://github.com/apache/airflow/blob/main/chart/templates/_helpers.yaml#L86-L92) to Airflow by the chart.
   
   Can you please test the url format I suggested and check if you still have the same problem with the liveness probe


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org