[jira] [Commented] (AIRFLOW-4547) Negative priority_weight should not be permitted

["Alex Abraham (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969606#comment-16969606 ] 

Alex Abraham commented on AIRFLOW-4547:
---------------------------------------

I actually have a use case where this is the desired behavior:

I have 50 tasks which send commands to a spark server to run a job on some data. Before I send the command to the server though, I want to make sure that the data exists. Therefore I created tasks before each spark task to check if the files exist and skip the spark task if they don't. The problem I was running into is that airflow made sure every single exists operator finished before submitting the first spark job. If you set the exists operator to -1 priority_weight, the spark jobs will execute since it will be a higher priority :)

> Negative priority_weight should not be permitted
> ------------------------------------------------
>
>                 Key: AIRFLOW-4547
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4547
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: operators
>    Affects Versions: 1.10.3
>            Reporter: Teresa Martyny
>            Priority: Major
>             Fix For: 2.0.0
>
>
> Airflow allows a dev to assign a negative priority_weight to a task. However, the Airflow code does math to determine the priority_weight on its own in models.py#priority_weight_total on line 2796
> This makes the final priority_weight wrong in the end. Airflow should raise an error if an operator has priority_weight assigned to a negative number at any point. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)