You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/28 22:53:07 UTC
[GitHub] [airflow] mik-laj opened a new pull request #15582: Add more typing to airflow.utils.helpers
mik-laj opened a new pull request #15582:
URL: https://github.com/apache/airflow/pull/15582
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#issuecomment-857948370
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#discussion_r622667263
##########
File path: airflow/utils/helpers.py
##########
@@ -181,7 +184,7 @@ def merge_dicts(dict1, dict2):
return merged
-def partition(pred: Callable, iterable: Iterable):
+def partition(pred: Callable[[T], bool], iterable: Iterable[T]):
Review comment:
```suggestion
def partition(pred: Callable[[T], bool], iterable: Iterable[T]) -> Tuple[Iterable[T], Iterable[T]]:
```
(I think?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#discussion_r637057400
##########
File path: airflow/utils/helpers.py
##########
@@ -21,7 +21,7 @@
from datetime import datetime
from functools import reduce
from itertools import filterfalse, tee
-from typing import Any, Callable, Dict, Generator, Iterable, List, Optional, TypeVar
+from typing import TYPE_CHECKING, Any, Callable, Dict, Generator, Iterable, List, Optional, TypeVar
Review comment:
Thanks.Added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj merged pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
mik-laj merged pull request #15582:
URL: https://github.com/apache/airflow/pull/15582
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#issuecomment-829372792
[The Workflow run](https://github.com/apache/airflow/actions/runs/796535728) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#issuecomment-847290858
I guess it needs rebase @mik-laj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on a change in pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on a change in pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#discussion_r624472183
##########
File path: airflow/utils/helpers.py
##########
@@ -21,7 +21,7 @@
from datetime import datetime
from functools import reduce
from itertools import filterfalse, tee
-from typing import Any, Callable, Dict, Generator, Iterable, List, Optional, TypeVar
+from typing import TYPE_CHECKING, Any, Callable, Dict, Generator, Iterable, List, Optional, TypeVar
Review comment:
```suggestion
from typing import TYPE_CHECKING, Any, Callable, Dict, Generator, Iterable, List, Optional, Tuple, TypeVar
```
This is needed to address the static check error caused by line 187
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jhtimmins commented on pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
jhtimmins commented on pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#issuecomment-857899639
@XD-DENG Can you take another look at validate that your requested changes have been addressed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj merged pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
mik-laj merged pull request #15582:
URL: https://github.com/apache/airflow/pull/15582
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#issuecomment-829373193
[The Workflow run](https://github.com/apache/airflow/actions/runs/796537907) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj merged pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
mik-laj merged pull request #15582:
URL: https://github.com/apache/airflow/pull/15582
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#discussion_r622666746
##########
File path: airflow/utils/helpers.py
##########
@@ -31,10 +31,17 @@
from airflow.exceptions import AirflowException
from airflow.utils.module_loading import import_string
+if TYPE_CHECKING:
+ from airflow.models import TaskInstance
+
KEY_REGEX = re.compile(r'^[\w.-]+$')
+CAMELCASE_TO_SNAKE_CASE_REGEX = re.compile('(?!^)([A-Z]+)')
Review comment:
I’m surprised for all the opinions pylint have, it does not bother to enforce a regex pattern must always use an r-string.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#discussion_r622667450
##########
File path: airflow/utils/helpers.py
##########
@@ -137,7 +140,7 @@ def parse_template_string(template_string):
return template_string, None
-def render_log_filename(ti, try_number, filename_template):
+def render_log_filename(ti: "TaskInstance", try_number, filename_template):
Review comment:
```suggestion
def render_log_filename(ti: "TaskInstance", try_number, filename_template) -> str:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#discussion_r623146456
##########
File path: airflow/utils/helpers.py
##########
@@ -31,10 +31,17 @@
from airflow.exceptions import AirflowException
from airflow.utils.module_loading import import_string
+if TYPE_CHECKING:
+ from airflow.models import TaskInstance
+
KEY_REGEX = re.compile(r'^[\w.-]+$')
+CAMELCASE_TO_SNAKE_CASE_REGEX = re.compile('(?!^)([A-Z]+)')
Review comment:
```suggestion
CAMELCASE_TO_SNAKE_CASE_REGEX = re.compile(r'(?!^)([A-Z]+)')
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15582: Add more typing to airflow.utils.helpers
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15582:
URL: https://github.com/apache/airflow/pull/15582#issuecomment-829373535
[The Workflow run](https://github.com/apache/airflow/actions/runs/796620491) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org