You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/02/15 15:56:45 UTC
[GitHub] [airflow] frankcash opened a new pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
frankcash opened a new pull request #21567:
URL: https://github.com/apache/airflow/pull/21567
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
~~Uses built in util to handle import of literal until python 3.8 is phased out.~~
Changes `setup.cfg` to install `typing-extensions` will all versions of python, not just `<3.8`
closes: #21566
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040448456
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1041857334
Sounds good, thank you so much for your time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1049049106
Keeps timing out at `Build Images / Build CI image 3.7 (pull_request_target) In progress — This check has started...` 🙃
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash edited a comment on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash edited a comment on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1050825318
Thanks for linking that @potiuk , I will provide a comment on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806269328
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
https://github.com/apache/airflow/pull/16582 - in 2.1.1 is when this was added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040451781
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on a change in pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806889512
##########
File path: setup.cfg
##########
@@ -137,7 +137,7 @@ install_requires =
tabulate>=0.7.5, <0.9
tenacity>=6.2.0
termcolor>=1.1.0
- typing-extensions>=3.7.4;python_version<"3.8"
Review comment:
Addressed in latest commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806268329
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
We'll need to check airflow versions here.
What is the minumum version that the provider now needs, and when did we release a version of Airflow core that contained the "Literal" compat.
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
https://github.com/apache/airflow/pull/16582 - in 2.1.1 is when this was added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806274029
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
Minimum version appears to be 2.1.0 https://github.com/apache/airflow/blob/main/airflow/providers/amazon/provider.yaml#L40
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
Minimum version appears to be 2.1.0 for the provider https://github.com/apache/airflow/blob/main/airflow/providers/amazon/provider.yaml#L40
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
Sounds good, thanks for the feedback. Is there a preferred way I should go about solving this @potiuk ?
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
As @uranusjr pointed out we could change the line in setup.cfg that doesn't install the lib if on python 3.8+
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
As @uranusjr pointed out we could change the line in setup.cfg that doesn't install the lib if on python 3.8+
I included that as a potential fix in the issue https://github.com/apache/airflow/issues/21566#issuecomment-1039537580
##########
File path: setup.cfg
##########
@@ -137,7 +137,7 @@ install_requires =
tabulate>=0.7.5, <0.9
tenacity>=6.2.0
termcolor>=1.1.0
- typing-extensions>=3.7.4;python_version<"3.8"
Review comment:
Addressed in latest commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash edited a comment on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash edited a comment on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040552530
> It could be that the typing extension actually causes some too long running constraint generation.
looks like it timed out again. Let me know how I can help resolve this / if I need to change the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash edited a comment on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash edited a comment on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1041554670
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1050652253
> it looks like a lot of other builds are failing at the build images / build ci image {python_version} step..
Not all - only those that change "setup.py". This is pip getting into backtracking loop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806274029
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
Minimum version appears to be 2.1.0 for the provider https://github.com/apache/airflow/blob/main/airflow/providers/amazon/provider.yaml#L40
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806782127
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
Sounds good, thanks for the feedback. Is there a preferred way I should go about solving this @potiuk ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040451781
Closed/reopened to check if this was a flaky fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1041844036
> I am confused on what I need to change / do to get this to resolve -- would someone point me in the right direction?
Yeah. I was afraid it is not "accident". I will have to take a look at that more closely, but I think the problem is that it creates unsolvable conflict with one of the relased providers (we prepare few variants of the constraints and apparently this one gets triggered.
What happens under the hood `pip` gets into backtracking and tries to find a solution to impossible conflict resolution and it takes awfully lot of time to resolve it. I am afraid this one will need to wait until we take a bit closer look what's going on in detail. I should find time for it this week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
uranusjr commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040020125
Why not just add typing_extension as a dependency?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806684747
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
The eror is clear about it https://github.com/apache/airflow/runs/5191554253?check_suite_focus=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1051178892
@frankcash I think I managed to workaround the pip resolver issue with https://github.com/apache/airflow/pull/21824 - please rebase to latest main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806783562
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
As @uranusjr pointed out we could change the line in setup.cfg that doesn't install the lib if on python 3.8+
I included that as a potential fix in the issue https://github.com/apache/airflow/issues/21566#issuecomment-1039537580
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040346226
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #21567:
URL: https://github.com/apache/airflow/pull/21567
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1047763777
Looks like the failing tests will be fixed with https://github.com/apache/airflow/pull/21733
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1047048253
> Sounds good, thank you so much for your time
I think it should work fine now. We released new providers, and increased timeout for constraint generation, so it should work as both potential reasons for the hanging should have been addressed. Rebased and let's see if it succeeds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1050825318
Thanks for linking that Jarek, I will provide a comment on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806783562
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
As @uranusjr pointed out we could change the line in setup.cfg that doesn't install the lib if on python 3.8+
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #21567:
URL: https://github.com/apache/airflow/pull/21567
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash edited a comment on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash edited a comment on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040552530
> It could be that the typing extension actually causes some too long running constraint generation.
looks like it timed out again. Let me know how I can help resolve this / if I need to change the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #21567:
URL: https://github.com/apache/airflow/pull/21567
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1041554670
@potiuk / @uranusjr I've re-run this a few times but it continues to time out at the  step
I am confused on what I need to change / do to get this to resolve
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1047237852
Looks like the provider tests failed with the rebase `Tests / MSSQL2019-latest, Py3.7: Always API Core Other CLI Providers WWW Integration (pull_request) Failing after 19m — MSSQL2019-latest, Py3.7: Always API Core Other CLI Providers WWW Integration`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1051281925
@potiuk it worked yay
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1050340714
it looks like a lot of other builds are failing at the build images / build ci image {python_version} step..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040452424
It could be that the typing extension actually causes some too long running constraint generation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806684159
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
Then we can't use it, I am afraid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806684159
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
Then we can't use it, I am afraid.
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
The eror is clear about it https://github.com/apache/airflow/runs/5191554253?check_suite_focus=true
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
I am for this solution as well:
```
typing-extensions>=3.7.4;python_version<"3.8"
```
##########
File path: setup.cfg
##########
@@ -137,7 +137,7 @@ install_requires =
tabulate>=0.7.5, <0.9
tenacity>=6.2.0
termcolor>=1.1.0
- typing-extensions>=3.7.4;python_version<"3.8"
Review comment:
I think we should add a comment here that we can remove this dependency when one of two things happen:
1) We drop Pyhon 3.7
2) We add min airflow version for all providers to be 2.2 - in which case we can replace it with compat.
##########
File path: setup.cfg
##########
@@ -137,7 +137,7 @@ install_requires =
tabulate>=0.7.5, <0.9
tenacity>=6.2.0
termcolor>=1.1.0
- typing-extensions>=3.7.4;python_version<"3.8"
Review comment:
BTW. The 2. might happen in a few months, so it's good to note it to remember and remove it eventually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806799128
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
I am for this solution as well:
```
typing-extensions>=3.7.4;python_version<"3.8"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806274029
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
Minimum version appears to be 2.1.0 https://github.com/apache/airflow/blob/main/airflow/providers/amazon/provider.yaml#L40
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
uranusjr commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040020125
Why not just add typing_extension as a dependency?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1047887380
@potiuk / @uranusjr lgtm if one of ya'll want to merge it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1050662989
@frankcash -> I would really love if you could upvote and comment here: https://github.com/pypa/pip/pull/10258#issuecomment-1025125753.
The problem is that we have no way currently to determine what is the root cause of the problem. The problem is that `pip` is not able to solve dependencies in a reasonable time and runs backtracking for hours - but it does not really give us the possibilty to get the root cause message that could allow us to determine how we could help `pip`.
I proposed to add optional timeout on `pip install` execution producing a "conflict" message if the timeout is reached. That could give us at least a chance to understand what is the problem (someone released a new dependency that makes `pip` fail installing airflow from the scratch and we need to help `pip` to determine the right set of dependencies because it is unable to figure it out on its own).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806881110
##########
File path: setup.cfg
##########
@@ -137,7 +137,7 @@ install_requires =
tabulate>=0.7.5, <0.9
tenacity>=6.2.0
termcolor>=1.1.0
- typing-extensions>=3.7.4;python_version<"3.8"
Review comment:
I think we should add a comment here that we can remove this dependency when one of two things happen:
1) We drop Pyhon 3.7
2) We add min airflow version for all providers to be 2.2 - in which case we can replace it with compat.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #21567: FIX: updates type imports to use typing_compat #21566
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806268329
##########
File path: airflow/providers/amazon/aws/sensors/sqs.py
##########
@@ -21,11 +21,11 @@
from typing import TYPE_CHECKING, Any, Optional, Sequence
from jsonpath_ng import parse
-from typing_extensions import Literal
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.sqs import SqsHook
from airflow.sensors.base import BaseSensorOperator
+from airflow.typing_compat import Literal
Review comment:
We'll need to check airflow versions here.
What is the minumum version that the provider now needs, and when did we release a version of Airflow core that contained the "Literal" compat.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#discussion_r806881880
##########
File path: setup.cfg
##########
@@ -137,7 +137,7 @@ install_requires =
tabulate>=0.7.5, <0.9
tenacity>=6.2.0
termcolor>=1.1.0
- typing-extensions>=3.7.4;python_version<"3.8"
Review comment:
BTW. The 2. might happen in a few months, so it's good to note it to remember and remove it eventually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040448456
Looks like the Github Actions are "failed" because one was cancelled? `Error: The operation was canceled.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040552530
> It could be that the typing extension actually causes some too long running constraint generation.
looks like it timed out again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1040346226
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash commented on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash commented on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1051182254
🤞
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] frankcash edited a comment on pull request #21567: FIX: extends typing-extensions to be installed with python 3.8+ #21566
Posted by GitBox <gi...@apache.org>.
frankcash edited a comment on pull request #21567:
URL: https://github.com/apache/airflow/pull/21567#issuecomment-1047237852
Looks like the provider tests failed with the rebase `Tests / MSSQL2019-latest, Py3.7: Always API Core Other CLI Providers WWW Integration (pull_request) Failing after 19m — MSSQL2019-latest, Py3.7: Always API Core Other CLI Providers WWW Integration`
Rebasing this to see if it was caused by a one off
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org