You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Noël BARDELOT (Jira)" <ji...@apache.org> on 2020/03/04 17:20:00 UTC
[jira] [Updated] (AIRFLOW-6985) Airflow should handle the rediss://
protocol for TLS-enable Redis
[ https://issues.apache.org/jira/browse/AIRFLOW-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Noël BARDELOT updated AIRFLOW-6985:
-----------------------------------
Description:
Please see the following issue for Flower:
https://github.com/mher/flower/issues/639
Flower and Airflow do not handle TLS-enable connections to Redis the same way. Thus, when providing the same broker URL to Flower that the one provided to Airflow, Flower cannot start/work.
There are several issues at hand here:
* Airflow by itself does handle its configuration correctly using `ssl_active`, but does not handle `rediss://`
* Flower by itself handles `rediss://` but does not handle an additionnal `ssl_active` option
* in the Helm chart for Airflow, there is no easy way to provide a configuration to Flower (Flower gets its configuration via the same configuration as Airflow, and thus cannot be correctly configured because Airflow would throw an exception)
See airflow/config_templates/default_celery.py where an exception is raised if `rediss://` is used.
A nice and quick fix would be to handle `rediss://` and drive the TLS-enabled mode is that protocol is used while `ssl_active` is also set to true. If `rediss://` is used but `ssl_active` is set to false, an exception could be raised to warn the user that the configuration is inconsistent.
was:
Please see the following issue for Flower:
https://github.com/mher/flower/issues/639
Flower and Airflow do not handle TLS-enable connections to Redis the same way. Thus, when providing the same broker URL to Flower that the one provided to Airflow, it does not work and Flower cannot start/work.
There are several issues at hand here:
* Airflow by itself does handle its configuration correctly, but does not handle `rediss://`
* Flower by itself handles `rediss://` but does not handle an additionnal `enable_ssl` option
* in the Helm chart for Airflow, there is no easy way to provide a configuration to Flower (Flower gets its configuration via the same configuration as Airflow, and thus cannot be correctly configured because Airflow would throw an exception)
See airflow/config_templates/default_celery.py where an exception is raised if `rediss://` is used.
A nice and quick fix would be to handle `rediss://` and drive the TLS-enabled mode is that protocol is used while ssl_enable is also set to true. If `rediss://` is used but `ssl_enable` is set to false, an exception could be raised to warn the user that the configuration is inconsistent.
> Airflow should handle the rediss:// protocol for TLS-enable Redis
> -----------------------------------------------------------------
>
> Key: AIRFLOW-6985
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6985
> Project: Apache Airflow
> Issue Type: Improvement
> Components: configuration
> Affects Versions: 1.10.9
> Reporter: Noël BARDELOT
> Priority: Major
>
> Please see the following issue for Flower:
> https://github.com/mher/flower/issues/639
> Flower and Airflow do not handle TLS-enable connections to Redis the same way. Thus, when providing the same broker URL to Flower that the one provided to Airflow, Flower cannot start/work.
> There are several issues at hand here:
> * Airflow by itself does handle its configuration correctly using `ssl_active`, but does not handle `rediss://`
> * Flower by itself handles `rediss://` but does not handle an additionnal `ssl_active` option
> * in the Helm chart for Airflow, there is no easy way to provide a configuration to Flower (Flower gets its configuration via the same configuration as Airflow, and thus cannot be correctly configured because Airflow would throw an exception)
> See airflow/config_templates/default_celery.py where an exception is raised if `rediss://` is used.
> A nice and quick fix would be to handle `rediss://` and drive the TLS-enabled mode is that protocol is used while `ssl_active` is also set to true. If `rediss://` is used but `ssl_active` is set to false, an exception could be raised to warn the user that the configuration is inconsistent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)