You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "eladkal (via GitHub)" <gi...@apache.org> on 2023/11/08 21:08:02 UTC
[I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
eladkal opened a new issue, #35540:
URL: https://github.com/apache/airflow/issues/35540
### Body
I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?
The guidelines on how to test providers can be found in
[Verify providers by contributors](https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#verify-the-release-candidate-by-contributors)
Let us know in the comment, whether the issue is addressed.
Those are providers that require testing as there were some substantial changes introduced:
## Provider [amazon: 8.11.0rc1](https://pypi.org/project/apache-airflow-providers-amazon/8.11.0rc1)
- [ ] [Add support for anonymous access to s3 buckets for objectstorage (#35273)](https://github.com/apache/airflow/pull/35273): @bolkedebruin
- [ ] [ECS Executor Health Check (#35412)](https://github.com/apache/airflow/pull/35412): @o-nikolas
- [ ] [Fix AWS RDS hook's DB instance state check (#34773)](https://github.com/apache/airflow/pull/34773): @AetherUnbound
- [ ] [Fix parameter syntax in Amazon docstrings (#35349)](https://github.com/apache/airflow/pull/35349): @josh-fell
- [ ] [Improve error handling in AWS Links (#35518)](https://github.com/apache/airflow/pull/35518): @Taragolis
- [ ] [Update ECS executor healthcheck with a catchall except (#35512)](https://github.com/apache/airflow/pull/35512): @o-nikolas
- [ ] [Move ECS Executor to its own file (#35418)](https://github.com/apache/airflow/pull/35418): @syedahsn
- [ ] [Clarify "task" in ECS Executor log messages (#35304)](https://github.com/apache/airflow/pull/35304): @ferruzzi
- [ ] [Make optional `output_location` attribute in `AthenaOperator` (#35265)](https://github.com/apache/airflow/pull/35265): @Taragolis
## Provider [apache.spark: 4.4.0rc1](https://pypi.org/project/apache-airflow-providers-apache-spark/4.4.0rc1)
- [ ] [Add pyspark decorator (#35247)](https://github.com/apache/airflow/pull/35247): @bolkedebruin
- [ ] [Add use_krb5ccache option to SparkSubmitOperator (#35331)](https://github.com/apache/airflow/pull/35331): @zeotuan
## Provider [cncf.kubernetes: 7.9.0rc1](https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/7.9.0rc1)
- [ ] [Add verification that provider docs are as expected (#35424)](https://github.com/apache/airflow/pull/35424): @potiuk
- [ ] [Add startup_check_interval_seconds to PodManager's await_pod_start (#34231)](https://github.com/apache/airflow/pull/34231): @stelsemeyer-m60
- [ ] [Remove before_log in KPO retry and add traceback when interrupted (#35423)](https://github.com/apache/airflow/pull/35423): @dstandish
- [ ] [Remove tenacity on KPO logs inner func consume_logs (#35504)](https://github.com/apache/airflow/pull/35504): @dstandish
- [ ] [Simplify KPO multi container log reconciliation logic (#35450)](https://github.com/apache/airflow/pull/35450): @dstandish
- [ ] [Remove PodLoggingStatus object (#35422)](https://github.com/apache/airflow/pull/35422): @dstandish
- [ ] [Improve clear_not_launched_queued_tasks call duration (#34985)](https://github.com/apache/airflow/pull/34985): @dirrao
- [ ] [Use constant for empty xcom result sentinel (#35451)](https://github.com/apache/airflow/pull/35451): @dstandish
## Provider [cohere: 1.0.0rc1](https://pypi.org/project/apache-airflow-providers-cohere/1.0.0rc1)
- [ ] [Add Cohere Provider (#34921)](https://github.com/apache/airflow/pull/34921): @utkarsharma2
## Provider [common.io: 1.0.1rc1](https://pypi.org/project/apache-airflow-providers-common-io/1.0.1rc1)
- [ ] [fix changelog of common-io (#35241)](https://github.com/apache/airflow/pull/35241): @eladkal
## Provider [databricks: 5.0.0rc1](https://pypi.org/project/apache-airflow-providers-databricks/5.0.0rc1)
- [ ] [Remove offset-based pagination from `list_jobs` function in `DatabricksHook` (#34926)](https://github.com/apache/airflow/pull/34926): @oleksiidav
## Provider [discord: 3.4.1rc1](https://pypi.org/project/apache-airflow-providers-discord/3.4.1rc1)
- [ ] [Add pagination to `HttpOperator` and make it more modular (#34669)](https://github.com/apache/airflow/pull/34669): @Joffreybvn
## Provider [docker: 3.8.1rc1](https://pypi.org/project/apache-airflow-providers-docker/3.8.1rc1)
- [ ] [fix `_DockerDecoratedOperator` module type attribute pickle error (#35293)](https://github.com/apache/airflow/pull/35293): @phi-friday
## Provider [elasticsearch: 5.1.1rc1](https://pypi.org/project/apache-airflow-providers-elasticsearch/5.1.1rc1)
- [ ] [Use None instead of empty data structures when no ElasticSearch logs (#34793)](https://github.com/apache/airflow/pull/34793): @dstandish
## Provider [ftp: 3.6.1rc1](https://pypi.org/project/apache-airflow-providers-ftp/3.6.1rc1)
- [ ] [FTPHook methods should not change working directory (#35105)](https://github.com/apache/airflow/pull/35105): @makrushin-evgenii
## Provider [google: 10.11.1rc1](https://pypi.org/project/apache-airflow-providers-google/10.11.1rc1)
- [ ] [Update Google Ads API version from v14 to v15 (#35295)](https://github.com/apache/airflow/pull/35295): @MaksYermak
## Provider [http: 4.7.0rc1](https://pypi.org/project/apache-airflow-providers-http/4.7.0rc1)
- [ ] [Add pagination to `HttpOperator` and make it more modular (#34669)](https://github.com/apache/airflow/pull/34669): @Joffreybvn
- [ ] [json data for async PUTs fixed (#35405)](https://github.com/apache/airflow/pull/35405): @andrew9691
## Provider [microsoft.azure: 8.2.0rc1](https://pypi.org/project/apache-airflow-providers-microsoft-azure/8.2.0rc1)
- [ ] [add managed identity support to AsyncDefaultAzureCredential (#35394)](https://github.com/apache/airflow/pull/35394): @Lee-W
- [ ] [ add managed identity support to azure container registry hook (#35320)](https://github.com/apache/airflow/pull/35320): @Lee-W
- [ ] [add managed identity support to wasb hook (#35326)](https://github.com/apache/airflow/pull/35326): @Lee-W
- [ ] [add managed identity support to asb hook (#35324)](https://github.com/apache/airflow/pull/35324): @Lee-W
- [ ] [add managed identity support to azure cosmos hook (#35323)](https://github.com/apache/airflow/pull/35323): @Lee-W
- [ ] [ add managed identity support to azure container volume hook (#35321)](https://github.com/apache/airflow/pull/35321): @Lee-W
- [ ] [add managed identity support to azure container instance hook (#35319)](https://github.com/apache/airflow/pull/35319): @Lee-W
- [ ] [add managed identity support to adx hook (#35325)](https://github.com/apache/airflow/pull/35325): @Lee-W
- [ ] [add managed identity support to azure batch hook (#35327)](https://github.com/apache/airflow/pull/35327): @Lee-W
- [ ] [add managed identity support to azure datalake hook (#35328)](https://github.com/apache/airflow/pull/35328): @Lee-W
- [ ] [add managed identity support to synapse hook (#35329)](https://github.com/apache/airflow/pull/35329): @Lee-W
- [ ] [add managed identity support to fileshare hook (#35330)](https://github.com/apache/airflow/pull/35330): @Lee-W
- [ ] [Fix AzureContainerInstanceOperator remove_on_error (#35212)](https://github.com/apache/airflow/pull/35212): @danielvdende
- [ ] [setting use_async=True for get_async_default_azure_credential (#35432)](https://github.com/apache/airflow/pull/35432): @Lee-W
- [ ] [Remove empty TYPE_CHECKING block into the Azure provider (#35477)](https://github.com/apache/airflow/pull/35477): @Taragolis
- [ ] [Refactor azure managed identity (#35367)](https://github.com/apache/airflow/pull/35367): @Lee-W
- [ ] [ Reuse get_default_azure_credential method from Azure utils for Azure key valut (#35318)](https://github.com/apache/airflow/pull/35318): @Lee-W
- [ ] [ make DefaultAzureCredential configurable in AzureKeyVaultBackend (#35052)](https://github.com/apache/airflow/pull/35052): @Lee-W
- [ ] [Make DefaultAzureCredential in AzureBaseHook configuration (#35051)](https://github.com/apache/airflow/pull/35051): @Lee-W
## Provider [openai: 1.0.0rc1](https://pypi.org/project/apache-airflow-providers-openai/1.0.0rc1)
- [ ] [Add OpenAI Provider (#35023)](https://github.com/apache/airflow/pull/35023): @utkarsharma2
## Provider [openlineage: 1.2.1rc1](https://pypi.org/project/apache-airflow-providers-openlineage/1.2.1rc1)
- [ ] [openlineage, snowflake: fix Snowflake information schema query (#35428)](https://github.com/apache/airflow/pull/35428): @JDarDagran
## Provider [pgvector: 1.0.0rc1](https://pypi.org/project/apache-airflow-providers-pgvector/1.0.0rc1)
- [ ] [Add pgvector provider implementation (#35399)](https://github.com/apache/airflow/pull/35399): @pankajkoti
## Provider [pinecone: 1.0.0rc1](https://pypi.org/project/apache-airflow-providers-pinecone/1.0.0rc1)
- [ ] [Add Pinecone Provider (#35094)](https://github.com/apache/airflow/pull/35094): @utkarsharma2
## Provider [postgres: 5.8.0rc1](https://pypi.org/project/apache-airflow-providers-postgres/5.8.0rc1)
- [ ] [Refactor cursor retrieval in PostgresHook. (#35498)](https://github.com/apache/airflow/pull/35498): @ylnsnv
## Provider [salesforce: 5.5.1rc1](https://pypi.org/project/apache-airflow-providers-salesforce/5.5.1rc1)
- [ ] [Fix SalesforceHook compatiblity with Pandas 2.x (#35145)](https://github.com/apache/airflow/pull/35145): @dave-pollock
## Provider [slack: 8.4.0rc1](https://pypi.org/project/apache-airflow-providers-slack/8.4.0rc1)
- [ ] [Reorganize SQL to Slack Operators (#35215)](https://github.com/apache/airflow/pull/35215): @Taragolis
## Provider [snowflake: 5.1.1rc1](https://pypi.org/project/apache-airflow-providers-snowflake/5.1.1rc1)
- [ ] [openlineage, snowflake: fix Snowflake information schema query (#35428)](https://github.com/apache/airflow/pull/35428): @JDarDagran
- [ ] [Bump min `snowflake-connector-python` version (#35440)](https://github.com/apache/airflow/pull/35440): @romsharon98
## Provider [weaviate: 1.0.0rc1](https://pypi.org/project/apache-airflow-providers-weaviate/1.0.0rc1)
- [ ] [Add Weaviate Provider (#35060)](https://github.com/apache/airflow/pull/35060): @utkarsharma2
## Provider [yandex: 3.6.0rc1](https://pypi.org/project/apache-airflow-providers-yandex/3.6.0rc1)
- [ ] [Yandex dataproc deduce default service account (#35059)](https://github.com/apache/airflow/pull/35059): @Piatachock
<!--
NOTE TO RELEASE MANAGER:
Please move here the providers that have doc-only changes or for which changes are trivial, and
you could assess that they are OK. In case
The providers are automatically installed on Airflow 2.3 and latest `main` during the CI, so we know they
are installable. Also, all classes within the providers are imported during the CI run so we know all
providers can be imported.
-->
All users involved in the PRs:
@ferruzzi @ylnsnv @syedahsn @stelsemeyer-m60 @dave-pollock @eladkal @zeotuan @MaksYermak @AetherUnbound @pankajkoti @makrushin-evgenii @andrew9691 @utkarsharma2 @Joffreybvn @phi-friday @Taragolis
@JDarDagran @danielvdende @bolkedebruin @dstandish @dirrao @Lee-W @oleksiidav @josh-fell @romsharon98 @Piatachock @potiuk @o-nikolas
### Committer
- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "utkarsharma2 (via GitHub)" <gi...@apache.org>.
utkarsharma2 commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1803908091
+1 Tested my changes, working as expected.
Thanks,
Utkarsh Sharma
On Thu, Nov 9, 2023 at 5:46 PM Jakub Dardzinski ***@***.***>
wrote:
> #35428 <https://github.com/apache/airflow/pull/35428> tested, working as
> expected
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/airflow/issues/35540#issuecomment-1803723441>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADDLAHIYTFESXLAS33T25DLYDTCTRAVCNFSM6AAAAAA7DQXVD6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBTG4ZDGNBUGE>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "Piatachock (via GitHub)" <gi...@apache.org>.
Piatachock commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1803645809
✅ for Yandex provider
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "Joffreybvn (via GitHub)" <gi...@apache.org>.
Joffreybvn commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1804313345
**There is something wrong with the HttpOperator**. The new pagination functionality doesn't behave like expected with `deferrable=False`. But, it works fine when deferred.
I opened a PR with a fix #35560
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "JDarDagran (via GitHub)" <gi...@apache.org>.
JDarDagran commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1803723441
#35428 tested, working as expected
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "MaksYermak (via GitHub)" <gi...@apache.org>.
MaksYermak commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1805461823
Tested (35295), working as expected
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "AetherUnbound (via GitHub)" <gi...@apache.org>.
AetherUnbound commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1806494221
Tested #34773 locally - Sensors which were failing previously are now being rescheduled as expected! Looks good on my end!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1803010365
My change was only in "tests" and it works :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "phi-friday (via GitHub)" <gi...@apache.org>.
phi-friday commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1807034724
tested #35293 locally. it works as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #35540: Status of testing Providers that were prepared on November 08, 2023
URL: https://github.com/apache/airflow/issues/35540
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "dirrao (via GitHub)" <gi...@apache.org>.
dirrao commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1806825702
Tested #34985 locally- it is working as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "pankajkoti (via GitHub)" <gi...@apache.org>.
pankajkoti commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1805717855
Tested #35399, working as expected. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1807211677
Thank you everyone.
Providers are released (http and discord providers are excluded and RC2 will follow shortly)
I invite everyone to help improve providers for the next release, a list of open issues can be found [here](https://github.com/apache/airflow/issues?q=is%3Aopen+is%3Aissue+label%3Aarea%3Aproviders).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1803545292
Tested with default Azure credential change and a few examples dags. All look good 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Status of testing Providers that were prepared on November 08, 2023 [airflow]
Posted by "danielvdende (via GitHub)" <gi...@apache.org>.
danielvdende commented on issue #35540:
URL: https://github.com/apache/airflow/issues/35540#issuecomment-1805202615
Tested my change (35212), works as expected
Daniel
On 9 Nov 2023 at 19:07 +0100, Joffrey Bienvenu ***@***.***>, wrote:
> There is something wrong with the HttpOperator. The new pagination functionality doesn't behave like expected with deferrable=False. But, it works fine when deferred.
> I opened a PR with a fix #35560
> —
> Reply to this email directly, view it on GitHub, or unsubscribe.
> You are receiving this because you were mentioned.Message ID: ***@***.***>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org