You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/21 17:39:10 UTC

[GitHub] [airflow] avnerw74 opened a new issue #13225: Run Task via the stable REST API

avnerw74 opened a new issue #13225:
URL: https://github.com/apache/airflow/issues/13225


   **Description**
   
   I have noticed that the stable REST API doesn't have the ability to run a task (which is possible from the airflow web interface.
   I think it would be nice to have either:
   - Run task 
   - Run all failing tasks (rerun from point of failure)
   this ability for integrations.
   
   **Use case / motivation**
   
   I would like the ability to identify the failing tasks on a specific DAG Run and rerun only them.
   I would like to do it remotely (non-interactive) using the REST API.
   I could write a script that run only the failing tasks, but I couldn't find a way to "Run" a task, when I have the task instance ID.
   
   **Are you willing to submit a PR?**
   
   Not at this stage
   
   **Related Issues**
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] vemikhaylov commented on issue #13225: Run Task via the stable REST API

Posted by GitBox <gi...@apache.org>.
vemikhaylov commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-766983481


   Would like to implement it


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal commented on issue #13225: Run Task via the stable REST API

Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-771569614


   @vemikhaylov assigned to you


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ngmiduc commented on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
ngmiduc commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-1054587310


   Okay but when I use a task_id, how does the API know from which DAG run it should clear? It doesn't seem to me that the task_id is a unique id. Or does it just clear out all tasks which that task_id?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] Kaash commented on issue #13225: Run Task via the stable REST API

Posted by GitBox <gi...@apache.org>.
Kaash commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-783752899


   Am also looking at this feature as we want to "clear" tasks in DAGS based on task ids using the REST API. 
   
   Hey @vemikhaylov are you close to a solution? 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-1053555204


   You can get task instances via first: https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/get_task_instances


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal edited a comment on issue #13225: Run Task via the stable REST API

Posted by GitBox <gi...@apache.org>.
eladkal edited a comment on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-771569614


   @vemikhaylov assigned to you
   
   For the 3rd point in the issue I think the actual action needed is clear. We have it for dags not not for specific tasks in the dag https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/post_clear_task_instances
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal commented on issue #13225: Run Task via the stable REST API

Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-771569614


   @vemikhaylov assigned to you


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] vemikhaylov commented on issue #13225: Run Task via the stable REST API

Posted by GitBox <gi...@apache.org>.
vemikhaylov commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-766983481


   Would like to implement it


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal edited a comment on issue #13225: Run Task via the stable REST API

Posted by GitBox <gi...@apache.org>.
eladkal edited a comment on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-771569614


   @vemikhaylov assigned to you
   
   For the 3rd point in the issue I think the actual action needed is clear. We have it for dags not not for specific tasks in the dag https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/post_clear_task_instances
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] vemikhaylov commented on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
vemikhaylov commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-783931306


   > Am also looking at this feature as we want to "clear" tasks in DAGS based on task ids using the REST API.
   > 
   > Hey @vemikhaylov are you close to a solution?
   
   Yes, I have implemented the most of the business logic and tests, but have gone on vacation. I'm going to create the PR in the next couple of days.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk edited a comment on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-1053555204


   You can get task instances first via: https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/get_task_instances
   
   Then you can select those which you want to clear and clear them via the clear API.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #13225: Run Task via the stable REST API

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-749103594


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil edited a comment on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
kaxil edited a comment on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-784151905


   @vemikhaylov Yes, we should implement the `clearTaskInstances` as it works for all executors coz it let's Scheduler pick the TaskInstances instead of Webserver creating an executor and running it (which happens in https://github.com/apache/airflow/blob/master/airflow/www/views.py#L1311-L1331)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ephraimbuddy closed issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
ephraimbuddy closed issue #13225:
URL: https://github.com/apache/airflow/issues/13225


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] vemikhaylov commented on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
vemikhaylov commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-783948767


   > Just to be clear we already have
   > 
   > https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/post_clear_task_instances
   > 
   > `https://airflow.apache.org/api/v1/dags/{dag_id}/clearTaskInstances` but the ask (at least from @Kaash ) is to have an optional task_id filter for this endpoint
   
   Hey @kaxil 
   
   I have locally implemented the following interface:
   
   ```yaml
     /dags/{dag_id}/runTaskInstance:
       parameters:
         - $ref: '#/components/parameters/DAGID'
   
       post:
         summary: Run a task instance
         x-openapi-router-controller: airflow.api_connexion.endpoints.task_instance_endpoint
         operationId: run_task_instance
         tags: [DAG]
         requestBody:
           description: Parameters of action
           required: true
           content:
             application/json:
               schema:
                 $ref: '#/components/schemas/RunTaskInstance'
   
         responses:
           '202':
             description: Accepted.
           '400':
             $ref: '#/components/responses/BadRequest'
           '401':
             $ref: '#/components/responses/Unauthenticated'
           '403':
             $ref: '#/components/responses/PermissionDenied'
           '404':
             $ref: '#/components/responses/NotFound'
       
      RunTaskInstance:
         type: object
         properties:
           task_id:
             description: The task ID.
             type: string
   
           execution_date:
             description: The execution date.
             type: string
             format: datetime
   
           ignore_all_dependencies:
             description: Whether to ignore all ignoreable dependencies
             type: boolean
   
           ignore_task_dependencies:
             description: Whether to ignore task-specific dependencies such as depends_on_past and
               trigger rule
             type: boolean
   
           ignore_task_instance_state:
             description: Whether to ignore the task instance's previous failure/success
             type: boolean
   ```
   
   That's similar to what we have there:
   https://github.com/apache/airflow/blob/master/airflow/www/views.py#L1311-L1331
   
   That's is currently used to run a task instance from the UI, but works for the Celery and Kubernetes executors only.
   
   Is it actually relevant or completely the same problems can be solved using the `/dags/{dag_id}/clearTaskInstances` endpoint and the separate endpoint isn't needed in the stable REST API?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ngmiduc edited a comment on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
ngmiduc edited a comment on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-1048847125


   Is it also possible to clear a set of tasks with an associated dagRunId ? Like `/dags/{dag_id}/dagRuns/{dag_run_id}/clearTaskInstances`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ngmiduc commented on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
ngmiduc commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-1048847125


   Is it also possible to clear a set of tasks with an associated dagRunId ? Like `/dags/{dag_id}/dagRuns/{dag_run_id}clearTaskInstances`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-783768367


   Just to be clear we already have 
   
   https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/post_clear_task_instances
   
   `https://airflow.apache.org/api/v1/dags/{dag_id}/clearTaskInstances` but the ask (at least from @Kaash ) is to have an optional task_id filter for this endpoint


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-784151905


   @vemikhaylov Yes, we should implement the `clearTaskInstances` as it works for all executors coz it let's Scheduler pick the TaskInstances instead of Webserver creating an executor and running it


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #13225: Clear Tasks via the stable REST API with task_id filter

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13225:
URL: https://github.com/apache/airflow/issues/13225#issuecomment-1060016410


   You wo't use task_id. Look what "get_task_instances" return.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org