[jira] [Commented] (AIRFLOW-276) List of dags does not refresh in UI for a while

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15412214#comment-15412214 ] 

ASF subversion and git services commented on AIRFLOW-276:
---------------------------------------------------------

Commit 9d254a317dd54f555270ca568aff1cd0500e1e53 in incubator-airflow's branch refs/heads/master from [~xuanji]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=9d254a3 ]

[AIRFLOW-276] Gunicorn rolling restart

- Tell gunicorn to prepend `[ready]` to worker process name once worker is ready (to serve requests) - in particular this happens after DAGs folder is parsed
- Airflow cli runs gunicorn as a child process instead of `excecvp`-ing over itself
- Airflow cli monitors gunicorn worker processes and restarts them by sending TTIN/TTOU signals to the gunicorn master process
- Fix bug where `conf.get('webserver', 'workers')` and `conf.get('webserver', 'webserver_worker_timeout')` were ignored

- Alternatively, https://github.com/apache/incubator-airflow/pull/1684/files does the same thing but the worker-restart script is provided separately for the user to run

- Start airflow, observe that workers are restarted
- Add new dags to dags folder and check that they show up
- Run `siege` against airflow while server is restarting and confirm that all requests succeed
- Run with configuration set to `batch_size = 0`, `batch_size = 1` and `batch_size = 4`

Closes #1685 from zodiac/xuanji_gunicorn_rolling_restart_2


> List of dags does not refresh in UI for a while
> -----------------------------------------------
>
>                 Key: AIRFLOW-276
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-276
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Li Xuanji
>            Assignee: Li Xuanji
>            Priority: Minor
>
> After creating a new dag (eg by adding a file to `~/airflow/dags`), the web UI does not show the new for a while. It only shows it when either
> 1. gunicorn decides to restart the worker process, or
> 2. a scheduler picks up the new dag, adds it to the airflow db, and the web UI notices it in the db



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)