You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/02/02 07:28:47 UTC
[GitHub] [airflow] chenglongyan opened a new pull request #21264: Refactor TriggerRule & WeightRule classes to inherit from Enum
chenglongyan opened a new pull request #21264:
URL: https://github.com/apache/airflow/pull/21264
closes: #19905
related: #5302,#18627
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21264: Refactor TriggerRule & WeightRule classes to inherit from Enum
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21264:
URL: https://github.com/apache/airflow/pull/21264#discussion_r798239432
##########
File path: airflow/utils/trigger_rule.py
##########
@@ -34,20 +36,16 @@ class TriggerRule:
ALWAYS = 'always'
NONE_FAILED_MIN_ONE_SUCCESS = "none_failed_min_one_success"
- _ALL_TRIGGER_RULES: Set[str] = set()
-
@classmethod
- def is_valid(cls, trigger_rule):
+ def is_valid(cls, trigger_rule: str) -> bool:
"""Validates a trigger rule."""
return trigger_rule in cls.all_triggers()
@classmethod
- def all_triggers(cls):
+ @cache
+ def all_triggers(cls) -> Set[str]:
"""Returns all trigger rules."""
- if not cls._ALL_TRIGGER_RULES:
- cls._ALL_TRIGGER_RULES = {
- getattr(cls, attr)
- for attr in dir(cls)
- if not attr.startswith("_") and not callable(getattr(cls, attr))
- }
- return cls._ALL_TRIGGER_RULES
+ return {getattr(cls, attr) for attr in dir(cls) if not attr.startswith("_")}
Review comment:
```suggestion
return set(cls.__members__.values())
```
The caching seems excessive and unnecessary though, this function is only called when a DAG is being initiated. I would remove `all_triggers` entirely and just rely on `__members__`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21264: Refactor TriggerRule & WeightRule classes to inherit from Enum
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21264:
URL: https://github.com/apache/airflow/pull/21264#discussion_r798239432
##########
File path: airflow/utils/trigger_rule.py
##########
@@ -34,20 +36,16 @@ class TriggerRule:
ALWAYS = 'always'
NONE_FAILED_MIN_ONE_SUCCESS = "none_failed_min_one_success"
- _ALL_TRIGGER_RULES: Set[str] = set()
-
@classmethod
- def is_valid(cls, trigger_rule):
+ def is_valid(cls, trigger_rule: str) -> bool:
"""Validates a trigger rule."""
return trigger_rule in cls.all_triggers()
@classmethod
- def all_triggers(cls):
+ @cache
+ def all_triggers(cls) -> Set[str]:
"""Returns all trigger rules."""
- if not cls._ALL_TRIGGER_RULES:
- cls._ALL_TRIGGER_RULES = {
- getattr(cls, attr)
- for attr in dir(cls)
- if not attr.startswith("_") and not callable(getattr(cls, attr))
- }
- return cls._ALL_TRIGGER_RULES
+ return {getattr(cls, attr) for attr in dir(cls) if not attr.startswith("_")}
Review comment:
```suggestion
return set(cls.__members__.values())
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21264: Refactor TriggerRule & WeightRule classes to inherit from Enum
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21264:
URL: https://github.com/apache/airflow/pull/21264#discussion_r798520060
##########
File path: tests/utils/test_weight_rule.py
##########
@@ -18,12 +18,18 @@
import unittest
+import pytest
+
from airflow.utils.weight_rule import WeightRule
class TestWeightRule(unittest.TestCase):
def test_valid_weight_rules(self):
- assert WeightRule.is_valid(WeightRule.DOWNSTREAM)
- assert WeightRule.is_valid(WeightRule.UPSTREAM)
- assert WeightRule.is_valid(WeightRule.ABSOLUTE)
- assert len(WeightRule.all_weight_rules()) == 3
+ weight_rules = WeightRule.__members__.values()
+ for wr in weight_rules:
+ assert isinstance(wr, str)
+ assert WeightRule.is_valid(wr)
+ assert len(WeightRule.all_weight_rules()) == len(weight_rules)
+
+ with pytest.raises(ValueError):
+ WeightRule("NOT_EXIST_WEIGHT_RULE")
Review comment:
Same here, the original test is better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chenglongyan closed pull request #21264: Refactor TriggerRule & WeightRule classes to inherit from Enum
Posted by GitBox <gi...@apache.org>.
chenglongyan closed pull request #21264:
URL: https://github.com/apache/airflow/pull/21264
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #21264: Refactor TriggerRule & WeightRule classes to inherit from Enum
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #21264:
URL: https://github.com/apache/airflow/pull/21264
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21264: Refactor TriggerRule & WeightRule classes to inherit from Enum
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21264:
URL: https://github.com/apache/airflow/pull/21264#issuecomment-1046212910
Nice one !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #21264: Refactor TriggerRule & WeightRule classes to inherit from Enum
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #21264:
URL: https://github.com/apache/airflow/pull/21264#discussion_r798519843
##########
File path: tests/utils/test_trigger_rule.py
##########
@@ -18,20 +18,18 @@
import unittest
+import pytest
+
from airflow.utils.trigger_rule import TriggerRule
class TestTriggerRule(unittest.TestCase):
def test_valid_trigger_rules(self):
- assert TriggerRule.is_valid(TriggerRule.ALL_SUCCESS)
- assert TriggerRule.is_valid(TriggerRule.ALL_FAILED)
- assert TriggerRule.is_valid(TriggerRule.ALL_DONE)
- assert TriggerRule.is_valid(TriggerRule.ONE_SUCCESS)
- assert TriggerRule.is_valid(TriggerRule.ONE_FAILED)
- assert TriggerRule.is_valid(TriggerRule.NONE_FAILED)
- assert TriggerRule.is_valid(TriggerRule.NONE_FAILED_OR_SKIPPED)
- assert TriggerRule.is_valid(TriggerRule.NONE_SKIPPED)
- assert TriggerRule.is_valid(TriggerRule.DUMMY)
- assert TriggerRule.is_valid(TriggerRule.ALWAYS)
- assert TriggerRule.is_valid(TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS)
- assert len(TriggerRule.all_triggers()) == 11
+ trigger_rules = TriggerRule.__members__.values()
+ for tr in trigger_rules:
+ assert isinstance(tr, str)
+ assert TriggerRule.is_valid(tr)
+ assert len(TriggerRule.all_triggers()) == len(trigger_rules)
+
+ with pytest.raises(ValueError):
+ TriggerRule("NOT_EXIST_TRIGGER_RULE")
Review comment:
I would keep the original test; the new test is basically checking `enum`’s implementation, which is not really what we need. The original tests is more useful because it actually checks the `TriggerRule` class is behaving as we expect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #21264: Refactor TriggerRule & WeightRule classes to inherit from Enum
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #21264:
URL: https://github.com/apache/airflow/pull/21264#issuecomment-1029118981
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org