You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Sam Dove (JIRA)" <ji...@apache.org> on 2019/03/19 01:18:00 UTC
[jira] [Updated] (AIRFLOW-4121) can trigger multiple dag_runs for a
dag_id when max_active_runs_per_dag=1
[ https://issues.apache.org/jira/browse/AIRFLOW-4121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sam Dove updated AIRFLOW-4121:
------------------------------
Description:
I can trigger multiple dag_runs for a dag_id when max_active_runs_per_dag=1, and I expect Airflow to prevent this with this environment setting. It doesn't fit our use case to be able to trigger/schedule the same dag id when it is currently in the middle of messing with things. It is not really a viable workaround for us to just not trigger the dag if it's running-- this needs to be built in or I need to implement it.
Steps I took to reproduce:
1. Download docker image for puckel/docker-airflow:1.10.2
2. Execute the below command
{code:java}
docker run -p 8080:8080 -e AIRFLOW__CORE__MAX_ACTIVE_RUNS_PER_DAG=1 -v <absolute path to local dag volumes folder>:/usr/local/airflow/dags -v /$HOME/.aws:/usr/local/airflow/.aws -d <name of local docker image for puckel/docker-airflow:1.10.2>
{code}
3. Open Airflow UI in a browser (localhost:8080) and go to Admin --> Configurations and verify max_active_runs_per_dag really is 1.
4. Execute the below command twice but at least 2 seconds apart (work around for https://issues.apache.org/jira/browse/AIRFLOW-764)
{code:java}
curl -X POST \
http://localhost:8080/api/experimental/dags/<replace with dag id>/dag_runs \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json'
{code}
5. In Airflow UI, go to Browse --> Dag Runs
6. Observe multiple dag runs for the same dag id
was:
I can trigger multiple dag_runs for a dag_id when max_active_runs_per_dag=1, and I expect Airflow to prevent this with this environment setting. It doesn't fit our use case to trigger/schedule the same dag id when it is currently in the middle of messing with things. It is not really a viable workaround for us to just not trigger the dag if it's running-- this needs to be built in or I need to implement it.
Steps I took to reproduce:
1. Download docker image for puckel/docker-airflow:1.10.2
2. Execute the below command
{code:java}
docker run -p 8080:8080 -e AIRFLOW__CORE__MAX_ACTIVE_RUNS_PER_DAG=1 -v <absolute path to local dag volumes folder>:/usr/local/airflow/dags -v /$HOME/.aws:/usr/local/airflow/.aws -d <name of local docker image for puckel/docker-airflow:1.10.2>
{code}
3. Open Airflow UI in a browser (localhost:8080) and go to Admin --> Configurations and verify max_active_runs_per_dag really is 1.
4. Execute the below command twice but at least 2 seconds apart (work around for https://issues.apache.org/jira/browse/AIRFLOW-764)
{code:java}
curl -X POST \
http://localhost:8080/api/experimental/dags/<replace with dag id>/dag_runs \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json'
{code}
5. In Airflow UI, go to Browse --> Dag Runs
6. Observe multiple dag runs for the same dag id
> can trigger multiple dag_runs for a dag_id when max_active_runs_per_dag=1
> -------------------------------------------------------------------------
>
> Key: AIRFLOW-4121
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4121
> Project: Apache Airflow
> Issue Type: Bug
> Components: api, DagRun
> Affects Versions: 1.10.2
> Reporter: Sam Dove
> Priority: Critical
>
> I can trigger multiple dag_runs for a dag_id when max_active_runs_per_dag=1, and I expect Airflow to prevent this with this environment setting. It doesn't fit our use case to be able to trigger/schedule the same dag id when it is currently in the middle of messing with things. It is not really a viable workaround for us to just not trigger the dag if it's running-- this needs to be built in or I need to implement it.
>
> Steps I took to reproduce:
> 1. Download docker image for puckel/docker-airflow:1.10.2
> 2. Execute the below command
>
> {code:java}
> docker run -p 8080:8080 -e AIRFLOW__CORE__MAX_ACTIVE_RUNS_PER_DAG=1 -v <absolute path to local dag volumes folder>:/usr/local/airflow/dags -v /$HOME/.aws:/usr/local/airflow/.aws -d <name of local docker image for puckel/docker-airflow:1.10.2>
> {code}
> 3. Open Airflow UI in a browser (localhost:8080) and go to Admin --> Configurations and verify max_active_runs_per_dag really is 1.
> 4. Execute the below command twice but at least 2 seconds apart (work around for https://issues.apache.org/jira/browse/AIRFLOW-764)
> {code:java}
> curl -X POST \
> http://localhost:8080/api/experimental/dags/<replace with dag id>/dag_runs \
> -H 'Cache-Control: no-cache' \
> -H 'Content-Type: application/json'
> {code}
> 5. In Airflow UI, go to Browse --> Dag Runs
> 6. Observe multiple dag runs for the same dag id
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)