You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/01/14 22:54:47 UTC

[GitHub] [airflow] tooptoop4 opened a new pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet

tooptoop4 opened a new pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet
URL: https://github.com/apache/airflow/pull/7170
 
 
   ---
   Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN = JIRA ID<sup>*</sup>
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   <sup>*</sup> For document-only changes commit message can start with `[AIRFLOW-XXXX]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] xinbinhuang commented on a change in pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet

Posted by GitBox <gi...@apache.org>.
xinbinhuang commented on a change in pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet
URL: https://github.com/apache/airflow/pull/7170#discussion_r369346353
 
 

 ##########
 File path: airflow/api/common/experimental/mark_tasks.py
 ##########
 @@ -102,6 +102,10 @@ def set_state(
     if dag is None:
         raise ValueError("Received tasks with no DAG")
 
+    latest_execution_date = dag.latest_execution_date
+    if latest_execution_date is None:
+        return []
+
 
 Review comment:
   Double-checked this. I agree that it should not throw an (internal?) error. I am wondering what's the reason for this error handling @jmcarp @ashb @kaxil @potiuk as this is only used by the frontend `airflow/www/views.py`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] tooptoop4 commented on issue #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet

Posted by GitBox <gi...@apache.org>.
tooptoop4 commented on issue #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet
URL: https://github.com/apache/airflow/pull/7170#issuecomment-598869923
 
 
   closing as will be covered by https://github.com/apache/airflow/pull/7709

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet

Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet
URL: https://github.com/apache/airflow/pull/7170#issuecomment-574447220
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=h1) Report
   > :exclamation: No coverage uploaded for pull request base (`master@00c6758`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
   > The diff coverage is `95.42%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7170/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff            @@
   ##             master    #7170   +/-   ##
   =========================================
     Coverage          ?   84.66%           
   =========================================
     Files             ?      710           
     Lines             ?    39482           
     Branches          ?        0           
   =========================================
     Hits              ?    33426           
     Misses            ?     6056           
     Partials          ?        0
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [airflow/models/baseoperator.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvYmFzZW9wZXJhdG9yLnB5) | `96.25% <ø> (ø)` | |
   | [airflow/models/pool.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcG9vbC5weQ==) | `97.36% <ø> (ø)` | |
   | [airflow/gcp/hooks/bigquery.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvYmlncXVlcnkucHk=) | `91.81% <ø> (ø)` | |
   | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `94.92% <ø> (ø)` | |
   | [...flow/contrib/example\_dags/example\_gcs\_to\_gdrive.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX2djc190b19nZHJpdmUucHk=) | `100% <100%> (ø)` | |
   | [airflow/operators/gcs\_to\_bq.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2NzX3RvX2JxLnB5) | `70.58% <100%> (ø)` | |
   | [airflow/api/common/experimental/mark\_tasks.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9tYXJrX3Rhc2tzLnB5) | `94.83% <100%> (ø)` | |
   | [...irflow/contrib/operators/gcp\_container\_operator.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfY29udGFpbmVyX29wZXJhdG9yLnB5) | `100% <100%> (ø)` | |
   | [airflow/contrib/hooks/gcp\_container\_hook.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9jb250YWluZXJfaG9vay5weQ==) | `100% <100%> (ø)` | |
   | [airflow/ti\_deps/deps/pool\_slots\_available\_dep.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy90aV9kZXBzL2RlcHMvcG9vbF9zbG90c19hdmFpbGFibGVfZGVwLnB5) | `100% <100%> (ø)` | |
   | ... and [18 more](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=footer). Last update [00c6758...2eff0e7](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet
URL: https://github.com/apache/airflow/pull/7170#issuecomment-574447220
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=h1) Report
   > :exclamation: No coverage uploaded for pull request base (`master@00c6758`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
   > The diff coverage is `95.42%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7170/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff            @@
   ##             master    #7170   +/-   ##
   =========================================
     Coverage          ?   85.12%           
   =========================================
     Files             ?      710           
     Lines             ?    39482           
     Branches          ?        0           
   =========================================
     Hits              ?    33608           
     Misses            ?     5874           
     Partials          ?        0
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [airflow/models/baseoperator.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvYmFzZW9wZXJhdG9yLnB5) | `96.25% <ø> (ø)` | |
   | [airflow/models/pool.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcG9vbC5weQ==) | `97.36% <ø> (ø)` | |
   | [airflow/gcp/hooks/bigquery.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvYmlncXVlcnkucHk=) | `91.81% <ø> (ø)` | |
   | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `94.92% <ø> (ø)` | |
   | [...flow/contrib/example\_dags/example\_gcs\_to\_gdrive.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX2djc190b19nZHJpdmUucHk=) | `100% <100%> (ø)` | |
   | [airflow/operators/gcs\_to\_bq.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2NzX3RvX2JxLnB5) | `70.58% <100%> (ø)` | |
   | [airflow/api/common/experimental/mark\_tasks.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9tYXJrX3Rhc2tzLnB5) | `94.83% <100%> (ø)` | |
   | [...irflow/contrib/operators/gcp\_container\_operator.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfY29udGFpbmVyX29wZXJhdG9yLnB5) | `100% <100%> (ø)` | |
   | [airflow/contrib/hooks/gcp\_container\_hook.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9jb250YWluZXJfaG9vay5weQ==) | `100% <100%> (ø)` | |
   | [airflow/ti\_deps/deps/pool\_slots\_available\_dep.py](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree#diff-YWlyZmxvdy90aV9kZXBzL2RlcHMvcG9vbF9zbG90c19hdmFpbGFibGVfZGVwLnB5) | `100% <100%> (ø)` | |
   | ... and [18 more](https://codecov.io/gh/apache/airflow/pull/7170/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=footer). Last update [00c6758...2eff0e7](https://codecov.io/gh/apache/airflow/pull/7170?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] tooptoop4 closed pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet

Posted by GitBox <gi...@apache.org>.
tooptoop4 closed pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet
URL: https://github.com/apache/airflow/pull/7170
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] xinbinhuang commented on a change in pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet

Posted by GitBox <gi...@apache.org>.
xinbinhuang commented on a change in pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet
URL: https://github.com/apache/airflow/pull/7170#discussion_r368770447
 
 

 ##########
 File path: airflow/api/common/experimental/mark_tasks.py
 ##########
 @@ -102,6 +102,10 @@ def set_state(
     if dag is None:
         raise ValueError("Received tasks with no DAG")
 
+    latest_execution_date = dag.latest_execution_date
+    if latest_execution_date is None:
+        return []
+
 
 Review comment:
   This whole snippet should be modified inside `get_execution_dates`. 
   https://github.com/apache/airflow/blob/2eff0e750f5a0cfde6bb68af215631f781e8fd90/airflow/api/common/experimental/mark_tasks.py#L246-L250
   
   Also, isn't the original behavior (throwing an error) intended?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] tooptoop4 commented on a change in pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet

Posted by GitBox <gi...@apache.org>.
tooptoop4 commented on a change in pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet
URL: https://github.com/apache/airflow/pull/7170#discussion_r368826713
 
 

 ##########
 File path: airflow/api/common/experimental/mark_tasks.py
 ##########
 @@ -102,6 +102,10 @@ def set_state(
     if dag is None:
         raise ValueError("Received tasks with no DAG")
 
+    latest_execution_date = dag.latest_execution_date
+    if latest_execution_date is None:
+        return []
+
 
 Review comment:
   user is doing simple click from UI, nothing selected so why should they see nasty error?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] xinbinhuang commented on a change in pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet

Posted by GitBox <gi...@apache.org>.
xinbinhuang commented on a change in pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet
URL: https://github.com/apache/airflow/pull/7170#discussion_r368770447
 
 

 ##########
 File path: airflow/api/common/experimental/mark_tasks.py
 ##########
 @@ -102,6 +102,10 @@ def set_state(
     if dag is None:
         raise ValueError("Received tasks with no DAG")
 
+    latest_execution_date = dag.latest_execution_date
+    if latest_execution_date is None:
+        return []
+
 
 Review comment:
   This whole snippet should be modified inside `get_execution_dates`. 
   https://github.com/apache/airflow/blob/2eff0e750f5a0cfde6bb68af215631f781e8fd90/airflow/api/common/experimental/mark_tasks.py#L248-L250
   
   Also, isn't the original behavior (throwing an error) intended?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] ashb commented on a change in pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet

Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7170: [AIRFLOW-6562] mushroom cloud error when clicking 'mark failed/success' from graph view of dag that has never been run yet
URL: https://github.com/apache/airflow/pull/7170#discussion_r372655071
 
 

 ##########
 File path: airflow/api/common/experimental/mark_tasks.py
 ##########
 @@ -102,6 +102,10 @@ def set_state(
     if dag is None:
         raise ValueError("Received tasks with no DAG")
 
+    latest_execution_date = dag.latest_execution_date
+    if latest_execution_date is None:
+        return []
+
 
 Review comment:
   That error message is a bit of a misnomer. It used to look like this:
   
   ```diff
   -    assert latest_execution_date is not None
   +    if latest_execution_date is None:
   +        raise ValueError("Received non-localized date {}".format(execution_date))
   ```
   
   I agree with @xinbinhuang - this change should go in `get_execution_dates` (and check the value of `dates` after the call to short circuit the rest of the function.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services