You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/05/04 02:59:48 UTC
[GitHub] [airflow] humit0 opened a new pull request, #23468: Rename cluster_policy to task_policy
humit0 opened a new pull request, #23468:
URL: https://github.com/apache/airflow/pull/23468
From [documentation](https://airflow.apache.org/docs/apache-airflow/stable/concepts/overview.html), there are only three policies (`dag_policy`, `task_policy`, `task_instance_mutation_hook`)
But example code uses `cluster_policy`, so I rename `cluster_policy` to `task_policy`.
And to pass flake8, move python code to docs.
<img width="1791" alt="image" src="https://user-images.githubusercontent.com/8676247/166617744-2c27f4b6-31c6-4f6c-b1d9-8824b7bacdbc.png">
Left part is generated docs, and right part is stable docs.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragement file, named `{pr_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] humit0 commented on a diff in pull request #23468: Rename cluster_policy to task_policy
Posted by GitBox <gi...@apache.org>.
humit0 commented on code in PR #23468:
URL: https://github.com/apache/airflow/pull/23468#discussion_r868833606
##########
docs/apache-airflow/concepts/cluster-policies.rst:
##########
@@ -57,12 +57,17 @@ This policy checks if each DAG has at least one tag defined:
Task policies
-------------
-Here's an example of enforcing a maximum timeout policy on every task:
+Here's an example of enforcing a maximum timeout policy on every task::
-.. literalinclude:: /../../tests/cluster_policies/__init__.py
Review Comment:
@potiuk when using literalinclude, it raises `redefinition` error.
<img width="686" alt="image" src="https://user-images.githubusercontent.com/8676247/167551873-1c26791f-3246-4a84-b6bb-8c1d78124ffd.png">
Can I create new file `tests/cluster_policies/example_task_policy.py` for adding that code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #23468: Rename cluster_policy to task_policy
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #23468:
URL: https://github.com/apache/airflow/pull/23468#discussion_r867875723
##########
docs/apache-airflow/concepts/cluster-policies.rst:
##########
@@ -57,12 +57,17 @@ This policy checks if each DAG has at least one tag defined:
Task policies
-------------
-Here's an example of enforcing a maximum timeout policy on every task:
+Here's an example of enforcing a maximum timeout policy on every task::
-.. literalinclude:: /../../tests/cluster_policies/__init__.py
Review Comment:
Why not using literalinclude from cluster_policies?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] humit0 commented on a diff in pull request #23468: Rename cluster_policy to task_policy
Posted by GitBox <gi...@apache.org>.
humit0 commented on code in PR #23468:
URL: https://github.com/apache/airflow/pull/23468#discussion_r869125662
##########
docs/apache-airflow/concepts/cluster-policies.rst:
##########
@@ -57,12 +57,17 @@ This policy checks if each DAG has at least one tag defined:
Task policies
-------------
-Here's an example of enforcing a maximum timeout policy on every task:
+Here's an example of enforcing a maximum timeout policy on every task::
-.. literalinclude:: /../../tests/cluster_policies/__init__.py
Review Comment:
Could you explain it more?
If I rename `task_policy` method (eg. `two_days_task_policy`), then the code in the document and the contents of the document do not match. I think it is weird.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] humit0 commented on a diff in pull request #23468: Rename cluster_policy to task_policy
Posted by GitBox <gi...@apache.org>.
humit0 commented on code in PR #23468:
URL: https://github.com/apache/airflow/pull/23468#discussion_r869754416
##########
docs/apache-airflow/concepts/cluster-policies.rst:
##########
@@ -57,12 +57,17 @@ This policy checks if each DAG has at least one tag defined:
Task policies
-------------
-Here's an example of enforcing a maximum timeout policy on every task:
+Here's an example of enforcing a maximum timeout policy on every task::
-.. literalinclude:: /../../tests/cluster_policies/__init__.py
Review Comment:
I was just thinking of a way to declare a method only in `airflow_local_settings.py` file...
I will change method name `cluster_policy` to `example_task_policy`!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #23468: Rename cluster_policy to task_policy
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #23468:
URL: https://github.com/apache/airflow/pull/23468#discussion_r869499349
##########
docs/apache-airflow/concepts/cluster-policies.rst:
##########
@@ -57,12 +57,17 @@ This policy checks if each DAG has at least one tag defined:
Task policies
-------------
-Here's an example of enforcing a maximum timeout policy on every task:
+Here's an example of enforcing a maximum timeout policy on every task::
-.. literalinclude:: /../../tests/cluster_policies/__init__.py
Review Comment:
Why don't you replace it cluster_policy() with "example_task_policy" for example?
There is no reason why task_policy should be named always "task_policy". This is why it was named "cluster_policy" (historically we only had cluster policy). But this is quite ok to have a method named differently and import it as different method name in python. There is a "import x as y" and patching also work this way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #23468: Rename cluster_policy to task_policy
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #23468:
URL: https://github.com/apache/airflow/pull/23468#discussion_r869052524
##########
docs/apache-airflow/concepts/cluster-policies.rst:
##########
@@ -57,12 +57,17 @@ This policy checks if each DAG has at least one tag defined:
Task policies
-------------
-Here's an example of enforcing a maximum timeout policy on every task:
+Here's an example of enforcing a maximum timeout policy on every task::
-.. literalinclude:: /../../tests/cluster_policies/__init__.py
Review Comment:
Just rename the other task_policy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #23468: Rename cluster_policy to task_policy
Posted by GitBox <gi...@apache.org>.
potiuk merged PR #23468:
URL: https://github.com/apache/airflow/pull/23468
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org