You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/07/31 09:33:36 UTC
[GitHub] [airflow] eladkal opened a new pull request, #25426: Update azure-storage-blob version
eladkal opened a new pull request, #25426:
URL: https://github.com/apache/airflow/pull/25426
We restricted version in https://github.com/apache/airflow/pull/18443 due to error in static checks https://github.com/apache/airflow/pull/17068#issuecomment-925264410
Tests pass locally with the updated version
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1204888672
> Just one static-check left!
Yes. This is the one that we must wait to 12.14.0
The issue already fixed upstream but they didn't release it yet. I wrote to them about it:
https://github.com/Azure/azure-sdk-for-python/issues/24661#issuecomment-1204300000
Once 12.14.0 I'll modify the PR to use that version as minimum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on code in PR #25426:
URL: https://github.com/apache/airflow/pull/25426#discussion_r938513313
##########
airflow/providers/microsoft/azure/provider.yaml:
##########
@@ -56,8 +56,7 @@ dependencies:
- azure-mgmt-datafactory>=1.0.0,<2.0
- azure-mgmt-datalake-store>=0.5.0
- azure-mgmt-resource>=2.2.0
- # limited due to https://github.com/Azure/azure-sdk-for-python/pull/18801 implementation released in 12.9
- - azure-storage-blob>=12.7.0,<12.9.0
+ - azure-storage-blob>=12.9.0,<12.14.0
Review Comment:
https://github.com/apache/airflow/pull/25426/commits/e9d7714d0205bfc0686354f08ed0df057828f516
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
uranusjr commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1200636509
Any reason versions between 12.9 and 12.13 need to be excluded?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
uranusjr commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1207656883
Yeah let’s wait for upstream.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
josh-fell commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1233152203
> Testing with `12.14.0b2`
>
> ```
> Run mypy for providers.................................................................Failed
> - hook id: run-mypy
> - exit code: 1
>
> airflow/providers/microsoft/azure/hooks/wasb.py:364: error: Argument "offset"
> to "download_blob" of "BlobClient" has incompatible type "Optional[int]";
> expected "int" [arg-type]
> return blob_client.download_blob(offset=offset, length=length,...
> ^
> airflow/providers/microsoft/azure/hooks/wasb.py:364: error: Argument "length"
> to "download_blob" of "BlobClient" has incompatible type "Optional[int]";
> expected "int" [arg-type]
> ...turn blob_client.download_blob(offset=offset, length=length, **kwargs)
> ^
> Found 2 errors in 1 file (checked 951 source files)
> ```
>
> Looks like [Azure/azure-sdk-for-python@8c24354](https://github.com/Azure/azure-sdk-for-python/commit/8c243548334e2d58bce77caa35ede6a792b75a93) changed also type hints for another function.
```
def download_blob(
self, offset: int = None,
length: int = None,
...
```
^ I'm by no means a mypy expert, but the typing for `offset` and `length` are incorrect?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1204278656
mmm We have
```
airflow/providers/microsoft/azure/hooks/wasb.py:302: error: Argument 1 to
"write" of "BufferedWriter" has incompatible type "Union[bytes, str]"; expected
"Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]]"
```
which seems to be related to https://github.com/Azure/azure-sdk-for-python/issues/24661 and will be fixed only in `12.14.0b1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1204328507
OK delimiter is solved.
So now :
1. Test failures are not related to this PR. It will be fixed in https://github.com/apache/airflow/pull/25511 - I'll rebase after it's merged.
2. write function issue has been fixed upstream we must wait to `12.14.0`
after that we should get a green build
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
josh-fell commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1207196743
> So it solved the static issue but created problems for the unit tests.
>
> I think the best option here is just to wait for upstream to release 12.14.0 should be in a month+-
Maybe if the assertion was under `TYPE_CHECKING` scope the unit test wouldn't fail? Just an idea since the assert statement is to hopefully make mypy happy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on code in PR #25426:
URL: https://github.com/apache/airflow/pull/25426#discussion_r938509436
##########
airflow/providers/microsoft/azure/provider.yaml:
##########
@@ -56,8 +56,7 @@ dependencies:
- azure-mgmt-datafactory>=1.0.0,<2.0
- azure-mgmt-datalake-store>=0.5.0
- azure-mgmt-resource>=2.2.0
- # limited due to https://github.com/Azure/azure-sdk-for-python/pull/18801 implementation released in 12.9
- - azure-storage-blob>=12.7.0,<12.9.0
+ - azure-storage-blob>=12.9.0,<12.14.0
Review Comment:
sure we can do that.. I planned to bump to 12.14.0 to avoid the extra code. Are you OK with leaving the extra code in `wasb.py` for good?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #25426:
URL: https://github.com/apache/airflow/pull/25426#discussion_r938500731
##########
airflow/providers/microsoft/azure/provider.yaml:
##########
@@ -56,8 +56,7 @@ dependencies:
- azure-mgmt-datafactory>=1.0.0,<2.0
- azure-mgmt-datalake-store>=0.5.0
- azure-mgmt-resource>=2.2.0
- # limited due to https://github.com/Azure/azure-sdk-for-python/pull/18801 implementation released in 12.9
- - azure-storage-blob>=12.7.0,<12.9.0
+ - azure-storage-blob>=12.9.0,<12.14.0
Review Comment:
I think the workaround should continue to work after 12.14.0 (unless they change things _drastically_, which is very unlikely)
```suggestion
- azure-storage-blob>=12.9.0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on code in PR #25426:
URL: https://github.com/apache/airflow/pull/25426#discussion_r938509436
##########
airflow/providers/microsoft/azure/provider.yaml:
##########
@@ -56,8 +56,7 @@ dependencies:
- azure-mgmt-datafactory>=1.0.0,<2.0
- azure-mgmt-datalake-store>=0.5.0
- azure-mgmt-resource>=2.2.0
- # limited due to https://github.com/Azure/azure-sdk-for-python/pull/18801 implementation released in 12.9
- - azure-storage-blob>=12.7.0,<12.9.0
+ - azure-storage-blob>=12.9.0,<12.14.0
Review Comment:
sure we can do that.. I planned to bump to 12.14.0 to avoid this extra code. Are you OK with leaving it as is?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1270301171
New release of azure-storage-blob is expected next week:
https://github.com/Azure/azure-sdk-for-python/issues/24661#issuecomment-1268952696
then this PR will be green :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
josh-fell commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1204302313
> As for the second static failure it's kinda strange. mypy fails with:
>
> ```
> airflow/providers/microsoft/azure/hooks/wasb.py:232: error: Argument
> "delimiter" to "walk_blobs" of "ContainerClient" has incompatible type
> "Optional[str]"; expected "str" [arg-type]
> ...me_starts_with=prefix, include=include, delimiter=delimiter, **kwargs)
> ```
>
> Code line:
>
> https://github.com/apache/airflow/blob/f18c609d127f54fbbf4dae6b290c6cdcfc7f98d0/airflow/providers/microsoft/azure/hooks/wasb.py#L216
>
> but the types did not change between the versions: https://github.com/Azure/azure-sdk-for-python/blob/azure-storage-blob_12.8.1/sdk/storage/azure-storage-blob/azure/storage/blob/_container_client.py#L779
>
> https://github.com/Azure/azure-sdk-for-python/blob/azure-storage-blob_12.13.0/sdk/storage/azure-storage-blob/azure/storage/blob/_container_client.py#L780
>
> In 12.8.1 (when mypy pass) it’s the same as in 12.13.0 which mypy fails!
The `delimiter` typing should really be `delimiter: str = '/',` though. I think we've been getting away with this in mypy for a while 🙂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1202242636
> Any reason versions between 12.9 and 12.13 need to be excluded?
I'll check if I can relax the version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1207175736
So it solved the static issue but created problems for the unit tests.
I think the best option here is just to wait for upstream to release 12.14.0 should be in a month+-
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1342725709
> This change broke the [wasb log task handler](https://github.com/apache/airflow/blob/main/airflow/providers/microsoft/azure/log/wasb_task_handler.py).
>
> The `download` method now requires `offset` and `length` arguments, but these are not provided from the task handler code. Also, how would it provide anything else than "read the whole file".
I think you shoudl open an issue (or better PR :) for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1207198979
> Maybe if the assertion was under TYPE_CHECKING scope the unit test wouldn't fail? Just an idea since the assert statement is to hopefully make mypy happy.
I think not really - the idea is to make it consistent, I think :). It's going to change when the fixed version of liubrary is out, so it will change then and someone might rely on it being bytes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1204875452
Just one static-check left!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal merged pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal merged PR #25426:
URL: https://github.com/apache/airflow/pull/25426
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1204306838
> The `delimiter` typing should really be `delimiter: str = '/',` though. I think we've been getting away with this in mypy for a while 🙂
Yes this is what i did as it also required param in the upstream lib but I'm not sure I get why mypy behaves differently on the same function signature and typing? a mystery.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
uranusjr commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1206264058
I think Mypy is too smart and can tell that `open(..., "wb")` does not accept str, so that would also fail. This is probably the solution with the least overhead
```python
content = stream.readall()
assert isinstance(content, bytes)
fileblob.write(content)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] malthe commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
malthe commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1342705839
This change broke the [wasb log task handler](https://github.com/apache/airflow/blob/main/airflow/providers/microsoft/azure/log/wasb_task_handler.py).
The `download` method now requires `offset` and `length` arguments, but these are not provided from the task handler code. Also, how would it provide anything else than "read the whole file".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1277372637
[azure-storage-blob](https://pypi.org/project/azure-storage-blob/) 12.14.0 was released so this PR should be ready
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1204285393
As for the second static failure it's kinda strange.
mypy fails with:
```
airflow/providers/microsoft/azure/hooks/wasb.py:232: error: Argument
"delimiter" to "walk_blobs" of "ContainerClient" has incompatible type
"Optional[str]"; expected "str" [arg-type]
...me_starts_with=prefix, include=include, delimiter=delimiter, **kwargs)
```
but the types did not change between the versions:
https://github.com/Azure/azure-sdk-for-python/blob/azure-storage-blob_12.8.1/sdk/storage/azure-storage-blob/azure/storage/blob/_container_client.py#L779
https://github.com/Azure/azure-sdk-for-python/blob/azure-storage-blob_12.13.0/sdk/storage/azure-storage-blob/azure/storage/blob/_container_client.py#L780
In 12.8.1 (when mypy pass) it’s the same as in 12.13.0 which mypy fails!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1204363391
> * Test failures are not related to this PR. It will be fixed in [Limit Flask to <2.3 in the wake of 2.2 breaking our tests #25511](https://github.com/apache/airflow/pull/25511) - I'll rebase after it's merged.
Merged . Rebasing this one on top :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on code in PR #25426:
URL: https://github.com/apache/airflow/pull/25426#discussion_r938509436
##########
airflow/providers/microsoft/azure/provider.yaml:
##########
@@ -56,8 +56,7 @@ dependencies:
- azure-mgmt-datafactory>=1.0.0,<2.0
- azure-mgmt-datalake-store>=0.5.0
- azure-mgmt-resource>=2.2.0
- # limited due to https://github.com/Azure/azure-sdk-for-python/pull/18801 implementation released in 12.9
- - azure-storage-blob>=12.7.0,<12.9.0
+ - azure-storage-blob>=12.9.0,<12.14.0
Review Comment:
sure we can do that.. I planned to bump to 12.14.0 to avoid the extra code. Are you OK with leaving it as is?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1206154900
Still failing with
```
Run mypy for providers.................................................................Failed
- hook id: run-mypy
- exit code: 1
airflow/providers/microsoft/azure/hooks/wasb.py:305: error: Incompatible types
in assignment (expression has type "Union[bytes, str]", variable has type
"bytes") [assignment]
content: bytes = stream.readall()
```
will push a fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on pull request #25426: Update azure-storage-blob version
Posted by GitBox <gi...@apache.org>.
eladkal commented on PR #25426:
URL: https://github.com/apache/airflow/pull/25426#issuecomment-1233137324
```
Run mypy for providers.................................................................Failed
- hook id: run-mypy
- exit code: 1
airflow/providers/microsoft/azure/hooks/wasb.py:364: error: Argument "offset"
to "download_blob" of "BlobClient" has incompatible type "Optional[int]";
expected "int" [arg-type]
return blob_client.download_blob(offset=offset, length=length,...
^
airflow/providers/microsoft/azure/hooks/wasb.py:364: error: Argument "length"
to "download_blob" of "BlobClient" has incompatible type "Optional[int]";
expected "int" [arg-type]
...turn blob_client.download_blob(offset=offset, length=length, **kwargs)
^
Found 2 errors in 1 file (checked 951 source files)
```
Looks like https://github.com/Azure/azure-sdk-for-python/commit/8c243548334e2d58bce77caa35ede6a792b75a93 changed also type hints for another function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org