You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/04/02 09:53:36 UTC
[GitHub] [airflow] potiuk commented on a change in pull request #22588: Print configuration on scheduler startup.
potiuk commented on a change in pull request #22588:
URL: https://github.com/apache/airflow/pull/22588#discussion_r841056341
##########
File path: tests/cli/commands/test_scheduler_command.py
##########
@@ -87,3 +91,18 @@ def test_graceful_shutdown(self, executor, mock_process, mock_scheduler_job):
scheduler_command.scheduler(args)
finally:
mock_process().terminate.assert_called()
+
+ @mock.patch.dict(os.environ, {'AIRFLOW__CORE__PARALLELISM': '35'}, clear=True)
+ @mock.patch("airflow.cli.commands.scheduler_command.SchedulerJob")
+ @mock.patch("airflow.cli.commands.scheduler_command.Process")
+ def test_scheduler_print_config(self, mock_process, mock_scheduler_job):
+ args = self.parser.parse_args(['scheduler'])
+ with conf_vars({("core", "sql_alchemy_conn_cmd"): 'echo hello'}):
+ with contextlib.redirect_stdout(io.StringIO()) as temp_stdout:
+ scheduler_command.scheduler(args)
+ output = temp_stdout.getvalue()
+
+ assert "sql_alchemy_conn = < hidden > [cmd]" in output
+ assert "max_active_tasks_per_dag = 16 [airflow.cfg]" in output
+ assert "parallelism = < hidden > [env var]" in output
Review comment:
Ough. I did not know `display_sensitive` is "so sensitive" :) .
I thin what we could do though is to implement a new flag in the dict `guess_sensitive_data` (?) and do few things there:
* hide anything coming from secret (I think there is flag that just shows path rather than value for that already and it is on by default)
* We already have the list of sensitive configs:
```
sensitive_config_values = {
('core', 'sql_alchemy_conn'),
('core', 'fernet_key'),
('celery', 'broker_url'),
('celery', 'flower_basic_auth'),
('celery', 'result_backend'),
('atlas', 'password'),
('smtp', 'smtp_password'),
('webserver', 'secret_key'),
}
```
* we can pass config through "redact()" - and we could extend the semantics of `sensitive_var_conn_names` to also cover key names of sensitive data (or we could even add another config `sensitive_configuration_names` where user could specify which of the config they want to hide).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org