You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "eladkal (via GitHub)" <gi...@apache.org> on 2024/02/12 11:26:59 UTC

[I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

eladkal opened a new issue, #37358:
URL: https://github.com/apache/airflow/issues/37358

   ### Body
   
   
   I have a kind request for all the contributors to the latest provider packages release.
   Could you please help us to test the RC versions of the providers?
   
   The guidelines on how to test providers can be found in
   
   [Verify providers by contributors](https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#verify-the-release-candidate-by-contributors)
   
   Let us know in the comment, whether the issue is addressed.
   
   Those are providers that require testing as there were some substantial changes introduced:
   
   
   ## Provider [amazon: 8.18.0rc1](https://pypi.org/project/apache-airflow-providers-amazon/8.18.0rc1)
      - [ ] [ECS Executor - Add backoff on failed task retry (#37109)](https://github.com/apache/airflow/pull/37109): @ferruzzi
      - [ ] [SqlToS3Operator: feat/ add max_rows_per_file parameter (#37055)](https://github.com/apache/airflow/pull/37055): @selimchergui
      - [ ] [Adding Amazon Neptune Hook and Operators (#37000)](https://github.com/apache/airflow/pull/37000): @ellisms
      - [ ] [Fix `log_query` to format SQL statement correctly in `AthenaOperator` (#36962)](https://github.com/apache/airflow/pull/36962): @ferruzzi
      - [ ] [check sagemaker training job status before deferring SageMakerTrainingOperator (#36685)](https://github.com/apache/airflow/pull/36685): @Lee-W
      - [ ] [Merge all ECS executor configs following recursive python dict update (#37137)](https://github.com/apache/airflow/pull/37137): @o-nikolas
      - [ ] [Update default value for `BatchSensor` (#37234)](https://github.com/apache/airflow/pull/37234): @vincbeck
      - [ ] [Remove info log from amazon provider-> S3 Hook -> download_file (#37211)](https://github.com/apache/airflow/pull/37211): @rawwar
      - [ ] [feature: S3ToRedshiftOperator templating aws_conn_id (#37195)](https://github.com/apache/airflow/pull/37195): @raphaelauv
      - [ ] [Updates to ECS Executor Docs (#37125)](https://github.com/apache/airflow/pull/37125): @o-nikolas
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [apache.beam: 5.6.1rc1](https://pypi.org/project/apache-airflow-providers-apache-beam/5.6.1rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [apache.drill: 2.6.1rc1](https://pypi.org/project/apache-airflow-providers-apache-drill/2.6.1rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [apache.druid: 3.8.1rc1](https://pypi.org/project/apache-airflow-providers-apache-druid/3.8.1rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [apache.hive: 7.0.0rc1](https://pypi.org/project/apache-airflow-providers-apache-hive/7.0.0rc1)
      - [ ] [ Simplify hive client connection (#37043)](https://github.com/apache/airflow/pull/37043): @romsharon98
      - [ ] [Fix pyhive hive_pure_sasl extra name (#37323)](https://github.com/apache/airflow/pull/37323): @hussein-awala
   ## Provider [apache.livy: 3.7.2rc1](https://pypi.org/project/apache-airflow-providers-apache-livy/3.7.2rc1)
      - [ ] [Bump aiohttp min version to avoid CVE-2024-23829 and CVE-2024-23334 (#37110)](https://github.com/apache/airflow/pull/37110): @hussein-awala
   ## Provider [apprise: 1.2.2rc1](https://pypi.org/project/apache-airflow-providers-apprise/1.2.2rc1)
      - [ ] [Simplified configuration parsing in Apprise Connection (#37202)](https://github.com/apache/airflow/pull/37202): @infohash
   ## Provider [celery: 3.6.0rc1](https://pypi.org/project/apache-airflow-providers-celery/3.6.0rc1)
      - [ ] [Add `task_acks_late` configuration to Celery Executor (#37066)](https://github.com/apache/airflow/pull/37066): @ninsbl
      - [ ] [prevent celery command autoscale misconfig (#36576)](https://github.com/apache/airflow/pull/36576): @Bowrna
   ## Provider [cncf.kubernetes: 8.0.0rc1](https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/8.0.0rc1)
      - [ ] [The task is stuck in a queued state forever in case of pod launch errors  (#36882)](https://github.com/apache/airflow/pull/36882): @dirrao
      - [ ] [Modify KPO to log container log periodically  (#37279)](https://github.com/apache/airflow/pull/37279): @pankajastro
      - [ ] [Fix occasional attr-undefined for the python_kubernetes_script (#37318)](https://github.com/apache/airflow/pull/37318): @potiuk
      - [ ] [Fix hanging KPO on deferrable task with do_xcom_push (#37300)](https://github.com/apache/airflow/pull/37300): @vchiapaikeo
      - [ ] [Fix rendering `SparkKubernetesOperator.template_body` (#37271)](https://github.com/apache/airflow/pull/37271): @Taragolis
      - [ ] [Fix assignment of template field in `__init__` in `KubernetesPodOperator` (#37010)](https://github.com/apache/airflow/pull/37010): @romsharon98
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
      - [ ] [Kubernetes version bump (#37040)](https://github.com/apache/airflow/pull/37040): @dirrao
   ## Provider [common.io: 1.3.0rc1](https://pypi.org/project/apache-airflow-providers-common-io/1.3.0rc1)
      - [ ] [AIP-58: Add object storage backend for xcom (#37058)](https://github.com/apache/airflow/pull/37058): @bolkedebruin
   ## Provider [common.sql: 1.11.0rc2](https://pypi.org/project/apache-airflow-providers-common-sql/1.11.0rc2)
      - [ ] [Enhancement: Performance enhancement for insert_rows method DbApiHook with fast executemany + SAP Hana support (#37246)](https://github.com/apache/airflow/pull/37246): @dabla
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [databricks: 6.2.0rc1](https://pypi.org/project/apache-airflow-providers-databricks/6.2.0rc1)
      - [ ] [Update DatabricksSqlOperator to work with namedtuple (#37025)](https://github.com/apache/airflow/pull/37025): @SuperBo
      - [ ] [Bump aiohttp min version to avoid CVE-2024-23829 and CVE-2024-23334 (#37110)](https://github.com/apache/airflow/pull/37110): @hussein-awala
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
      - [ ] [Get rid of pytest-httpx as dependency (#37334)](https://github.com/apache/airflow/pull/37334): @potiuk
   ## Provider [dbt.cloud: 3.6.1rc1](https://pypi.org/project/apache-airflow-providers-dbt-cloud/3.6.1rc1)
      - [ ] [Bump aiohttp min version to avoid CVE-2024-23829 and CVE-2024-23334 (#37110)](https://github.com/apache/airflow/pull/37110): @hussein-awala
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [elasticsearch: 5.3.3rc1](https://pypi.org/project/apache-airflow-providers-elasticsearch/5.3.3rc1)
      - [ ] [Avoid `pendulum.from_timestamp` usage (#37160)](https://github.com/apache/airflow/pull/37160): @Taragolis
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [google: 10.15.0rc1](https://pypi.org/project/apache-airflow-providers-google/10.15.0rc1)
      - [ ] [add service_file support to GKEPodAsyncHook (#37081)](https://github.com/apache/airflow/pull/37081): @Lee-W
      - [ ] [Update GCP Dataproc ClusterGenerator to support GPU params (#37036)](https://github.com/apache/airflow/pull/37036): @dmedora
      - [ ] [Create DataprocStartClusterOperator and DataprocStopClusterOperator (#36996)](https://github.com/apache/airflow/pull/36996): @molcay
      - [ ] [Implement deferrable mode for CreateHyperparameterTuningJobOperator (#36594)](https://github.com/apache/airflow/pull/36594): @moiseenkov
      - [ ] [Enable '_enable_tcp_keepalive' functionality for GKEPodHook (#36999)](https://github.com/apache/airflow/pull/36999): @MaksYermak
      - [ ] [fix how GKEPodAsyncHook.service_file_as_context is used (#37306)](https://github.com/apache/airflow/pull/37306): @Lee-W
      - [ ] [Fix metadata override for ComputeEngineSSHHook (#37192)](https://github.com/apache/airflow/pull/37192): @molcay
      - [ ] [Fix assignment of template field in `__init__` in `custom_job` (#36789)](https://github.com/apache/airflow/pull/36789): @romsharon98
      - [ ] [Fix location requirement in DataflowTemplatedJobStartOperator (#37069)](https://github.com/apache/airflow/pull/37069): @shardbread
      - [ ] [Fix assignment of template field in `__init__` in `CloudDataTransferServiceCreateJobOperator` (#36909)](https://github.com/apache/airflow/pull/36909): @romsharon98
      - [ ] [Fixed the hardcoded default namespace value for GCP Data Fusion links. (#35379)](https://github.com/apache/airflow/pull/35379): @synsh
      - [ ] [Do not ignore the internal_ip_only if set to false in Dataproc cluster config (#37014)](https://github.com/apache/airflow/pull/37014): @ahzaz
      - [ ] [Revert protection against back-compatibilty issue with google-core-api (#37111)](https://github.com/apache/airflow/pull/37111): @potiuk
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [hashicorp: 3.6.3rc1](https://pypi.org/project/apache-airflow-providers-hashicorp/3.6.3rc1)
      - [ ] [feat(hashicorp): Add `VAULT_TOKEN` support (#37337)](https://github.com/apache/airflow/pull/37337): @V0lantis
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [http: 4.9.1rc1](https://pypi.org/project/apache-airflow-providers-http/4.9.1rc1)
      - [ ] [Bump aiohttp min version to avoid CVE-2024-23829 and CVE-2024-23334 (#37110)](https://github.com/apache/airflow/pull/37110): @hussein-awala
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
      - [ ] [Get rid of pytest-httpx as dependency (#37334)](https://github.com/apache/airflow/pull/37334): @potiuk
   ## Provider [influxdb: 2.4.1rc1](https://pypi.org/project/apache-airflow-providers-influxdb/2.4.1rc1)
      - [ ] [Get rid of pytest-httpx as dependency (#37334)](https://github.com/apache/airflow/pull/37334): @potiuk
   ## Provider [jdbc: 4.2.2rc1](https://pypi.org/project/apache-airflow-providers-jdbc/4.2.2rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [microsoft.mssql: 3.6.1rc1](https://pypi.org/project/apache-airflow-providers-microsoft-mssql/3.6.1rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [mongo: 4.0.0rc1](https://pypi.org/project/apache-airflow-providers-mongo/4.0.0rc1)
      - [ ] [Adding certificate validation for ssl in mongo hook (#37214)](https://github.com/apache/airflow/pull/37214): @amoghrajesh
      - [ ] [Change `MongoHook.conn_name_attr` to `mongo_conn_id` (#37104)](https://github.com/apache/airflow/pull/37104): @Taragolis
      - [ ] [Change docs type in insert_many to list of dicts (#37230)](https://github.com/apache/airflow/pull/37230): @MezentsevIlya
   ## Provider [mysql: 5.5.3rc1](https://pypi.org/project/apache-airflow-providers-mysql/5.5.3rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [oracle: 3.9.2rc1](https://pypi.org/project/apache-airflow-providers-oracle/3.9.2rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [pagerduty: 3.6.2rc1](https://pypi.org/project/apache-airflow-providers-pagerduty/3.6.2rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [postgres: 5.10.1rc1](https://pypi.org/project/apache-airflow-providers-postgres/5.10.1rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [qdrant: 1.0.0rc1](https://pypi.org/project/apache-airflow-providers-qdrant/1.0.0rc1)
      - [ ] [Add Qdrant Provider (#36805)](https://github.com/apache/airflow/pull/36805): @Anush008
   ## Provider [sftp: 4.9.0rc1](https://pypi.org/project/apache-airflow-providers-sftp/4.9.0rc1)
      - [ ] [Add deferrable param in SFTPSensor (#37117)](https://github.com/apache/airflow/pull/37117): @utkarsharma2
   ## Provider [slack: 8.6.1rc1](https://pypi.org/project/apache-airflow-providers-slack/8.6.1rc1)
      - [ ] [Remove the remaining references to use the token bypassing the Connection in the Slack provider. (#37112)](https://github.com/apache/airflow/pull/37112): @Taragolis
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [snowflake: 5.3.1rc1](https://pypi.org/project/apache-airflow-providers-snowflake/5.3.1rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [sqlite: 3.7.1rc1](https://pypi.org/project/apache-airflow-providers-sqlite/3.7.1rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [ssh: 3.10.1rc1](https://pypi.org/project/apache-airflow-providers-ssh/3.10.1rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [tableau: 4.4.2rc1](https://pypi.org/project/apache-airflow-providers-tableau/4.4.2rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [trino: 5.6.2rc1](https://pypi.org/project/apache-airflow-providers-trino/5.6.2rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [vertica: 3.7.1rc1](https://pypi.org/project/apache-airflow-providers-vertica/3.7.1rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [weaviate: 1.3.2rc1](https://pypi.org/project/apache-airflow-providers-weaviate/1.3.2rc1)
      - [ ] [Standardize deprecations in providers [part1] (#36876)](https://github.com/apache/airflow/pull/36876): @kacpermuda
   ## Provider [yandex: 3.9.0rc1](https://pypi.org/project/apache-airflow-providers-yandex/3.9.0rc1)
      - [ ] [Add secrets-backends section into the Yandex provider yaml definition (#37065)](https://github.com/apache/airflow/pull/37065): @Taragolis
      - [ ] [fix: using endpoint from connection if not specified (#37076)](https://github.com/apache/airflow/pull/37076): @ImpressionableRaccoon
   
   <!--
   
   NOTE TO RELEASE MANAGER:
   
   You can move here the providers that have doc-only changes or for which changes are trivial, and
   you could assess that they are OK.
   
   -->
   All users involved in the PRs:
   @ellisms @moiseenkov @o-nikolas @synsh @Anush008 @infohash @shardbread @raphaelauv @hussein-awala @molcay @pankajastro @selimchergui @dabla @rawwar @SuperBo @ninsbl @V0lantis @kacpermuda @ahzaz @dirrao
   
   
   ### Committer
   
   - [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.

potiuk commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938788250

   Added https://github.com/apache/airflow/pull/37362 to make the provider "ready" (we will need to do it when we come closer to 2.9.0 release.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "pankajkoti (via GitHub)" <gi...@apache.org>.

pankajkoti commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1943347598

   #37117 works as expected


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "utkarsharma2 (via GitHub)" <gi...@apache.org>.

utkarsharma2 commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1943727694

   Tested Add deferrable param in SFTPSensor (#37117)
   <https://github.com/apache/airflow/pull/37117> and Fix metadata override
   for ComputeEngineSSHHook (#37192)
   <https://github.com/apache/airflow/pull/37192>: @molcay
   <https://github.com/molcay> works as expected.
   
   Thanks,
   Utkarsh Sharma
   
   On Wed, Feb 14, 2024 at 2:41 PM Pankaj Koti ***@***.***>
   wrote:
   
   > #37117 <https://github.com/apache/airflow/pull/37117> works as expected
   >
   > —
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/airflow/issues/37358#issuecomment-1943347598>,
   > or unsubscribe
   > <https://github.com/notifications/unsubscribe-auth/ADDLAHM3XVNAOIW7XIDQ5GTYTR5SJAVCNFSM6AAAAABDERZ2CGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBTGM2DONJZHA>
   > .
   > You are receiving this because you were mentioned.Message ID:
   > ***@***.***>
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "moiseenkov (via GitHub)" <gi...@apache.org>.

moiseenkov commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938513725

   @potiuk , @eladkal , hi,
   Could we please include #37282 into the release?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "raphaelauv (via GitHub)" <gi...@apache.org>.

raphaelauv commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1943824457

   ` Fix hanging KPO on deferrable task with do_xcom_push (#37300): @vchiapaikeo`
    
    work :+1: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "Taragolis (via GitHub)" <gi...@apache.org>.

Taragolis commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1941326115

   > However it raises error with xcom enabled when serializing named tuple to json.
   
   Serializing named tuple should be add into the core first, e.g. it might be added by https://github.com/apache/airflow/pull/37168


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "kacpermuda (via GitHub)" <gi...@apache.org>.

kacpermuda commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938985436

   I tested a random sample of changes from #36876 and it works OK. The PR simply changed the warnings.warn to decorator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "infohash (via GitHub)" <gi...@apache.org>.

infohash commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938654442

   <!-- gh-issue-number: gh-37202 -->
   * gh-37202 is working with Provider [apprise: 1.2.2rc1](https://pypi.org/project/apache-airflow-providers-apprise/1.2.2rc1)
   <!-- /gh-issue-number -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "SuperBo (via GitHub)" <gi...@apache.org>.

SuperBo commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1941187375

   Databricks providers, confirm [#37025](https://github.com/apache/airflow/pull/37025) works when xcom is disabled. However it raises error with xcom enabled when serializing named tuple to json.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "ninsbl (via GitHub)" <gi...@apache.org>.

ninsbl commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938763329

   Hmm... I am struggeling to build a recent version of Airflow with `celery 3.6.0rc1`. The issue seems unrelated to the changes in the provider. I tried to build a container with 2.9.0 but there FAB provider seems to be missing / not installable...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.

potiuk commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938621258

   > Observing an issue with [common.sql: 1.11.0rc2](https://pypi.org/project/apache-airflow-providers-common-sql/1.11.0rc2)
   > 
   > Posted a [question](https://github.com/apache/airflow/pull/37246#discussion_r1486120101) on the PR
   
   Nice catch @pankajkoti 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "rawwar (via GitHub)" <gi...@apache.org>.

rawwar commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1939109668

   Tried out #37211 
   
   amazon provider - 8.17.0:
   ![image](https://github.com/apache/airflow/assets/20266953/f96c054b-a6fe-4021-a0e5-ecfa5a76c23b)
   
   amazon provider - 8.18.0rc1:
   ![image](https://github.com/apache/airflow/assets/20266953/f727596c-51d6-4c88-a06f-9a74e408a2a9)
   
   
   Log is no longer printed. hence, working as expected.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.

eladkal commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1940678655

   Providers `amazon`, `cncf.kubernetes` and `common.sql` will be excluded from RC1.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.

eladkal commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1944428660

   > However, this executor is still experimental so I'm not sure it should block this release.
   
   Since we are having RC2 for amazon provider there is time to fix it if you can raise a PR?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.

potiuk commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1939817893

   Checked all my changes. LGTM.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.

eladkal commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1946422517

   Thank you everyone. Providers are released.
   
   Providers: `amazon`, `common.sql`, `cncf.kubernetes` were excluded from this release.
   
   I invite everyone to help improve providers for the next release, a list of open issues can be found [here](https://github.com/apache/airflow/issues?q=is%3Aopen+is%3Aissue+label%3Aarea%3Aproviders).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "ahzaz (via GitHub)" <gi...@apache.org>.

ahzaz commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1942052614

   #37014 works as expected. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "molcay (via GitHub)" <gi...@apache.org>.

molcay commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938872616

   https://github.com/apache/airflow/pull/36996 and https://github.com/apache/airflow/pull/37192 works as expected


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.

amoghrajesh commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1939082982

   Tried out https://github.com/apache/airflow/pull/37214 using some sample DAGs in breeze environment. Looks good 👍🏽 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "V0lantis (via GitHub)" <gi...@apache.org>.

V0lantis commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1939420241

   Tried out #37337 <https://github.com/apache/airflow/pull/37337>, works like
   a charm.
   
   Provider apache-airflow-providers-hashicorp==3.6.3rc1
   
   Le lun. 12 févr. 2024 à 17:37, Kalyan ***@***.***> a écrit :
   
   > Tried out #37211 <https://github.com/apache/airflow/pull/37211>
   >
   > amazon provider - 8.17.0:
   > image.png (view on web)
   > <https://github.com/apache/airflow/assets/20266953/f96c054b-a6fe-4021-a0e5-ecfa5a76c23b>
   >
   > amazon provider - 8.18.0rc1:
   > image.png (view on web)
   > <https://github.com/apache/airflow/assets/20266953/f727596c-51d6-4c88-a06f-9a74e408a2a9>
   >
   > Log is no longer printed. hence, working as expected.
   >
   > —
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/airflow/issues/37358#issuecomment-1939109668>,
   > or unsubscribe
   > <https://github.com/notifications/unsubscribe-auth/AI7LNNVW4RMJ3ZBCIZAWTGDYTJANRAVCNFSM6AAAAABDERZ2CGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZZGEYDSNRWHA>
   > .
   > You are receiving this because you were mentioned.Message ID:
   > ***@***.***>
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "synsh (via GitHub)" <gi...@apache.org>.

synsh commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1939582721

   #35379 looks good.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "pankajastro (via GitHub)" <gi...@apache.org>.

pankajastro commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1944013309

   
   > However, I have deferrable tasks that should be completing that are not completing due to #37279. Will comment on the PR there. cc @pankajastro @pankajkoti
   > 
   > Link to comment: [#37279 (comment)](https://github.com/apache/airflow/pull/37279#issuecomment-1938757838)
   
   Hey @vchiapaikeo would it be possible for you to test https://github.com/apache/airflow/pull/37363 with your use case?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "moiseenkov (via GitHub)" <gi...@apache.org>.

moiseenkov commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938512042

   #36594 works as expected


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "pankajastro (via GitHub)" <gi...@apache.org>.

pankajastro commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938751873

   > However, I have deferrable tasks that should be completing that are not completing due to https://github.com/apache/airflow/pull/37279. Will comment on the PR there. cc @pankajastro @pankajkoti
   
   Thanks for testing it. please share the DAG and other details if possible. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.

potiuk commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938774710

   > Hmm... I am struggeling to build a recent version of Airflow with `celery 3.6.0rc1`. The issue seems unrelated to the changes in the provider. I tried to build a container with 2.9.0 but there FAB provider seems to be missing / not installable...
   
   Yes. FAB provider and Airflow 2.9.0 are not yet usable to install. We might want to change that and release FAB provider in the next wave


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "Taragolis (via GitHub)" <gi...@apache.org>.

Taragolis commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1941335389

   Checked all my changes, looking good


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.

Lee-W commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1945445200

   > #36685 doesn't work as expected. It stucks somehow. Will take a deeper look these days.
   > 
   > #37306 and #37081 works fine without [cncf.kubernetes: 8.0.0rc1](https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/8.0.0rc1) Looks like [#37279 (comment)](https://github.com/apache/airflow/pull/37279#issuecomment-1938782796) is the root cause
   
   I tested #36685 a few more times today. The issue can no longer be reproduced.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "pankajkoti (via GitHub)" <gi...@apache.org>.

pankajkoti commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938644369

   > Nice catch @pankajkoti
   
   Actually, it was the (Astronomer) Providers RC automation test suite that reported this failure :) 
   Thank you @potiuk for providing the fix PR https://github.com/apache/airflow/pull/37359 so quick. Appreciate it 🙏🏽 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "ellisms (via GitHub)" <gi...@apache.org>.

ellisms commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938804993

   #37000 is good.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.

Lee-W commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1940427194

   https://github.com/apache/airflow/pull/36685 doesn't work as expected. It stucks somehow. Will take a deeper look these days.
   
   https://github.com/apache/airflow/pull/37306 and https://github.com/apache/airflow/pull/37081 works fine without [cncf.kubernetes: 8.0.0rc1](https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/8.0.0rc1) Looks like https://github.com/apache/airflow/pull/37279#issuecomment-1938782796 is the root cause


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "Anush008 (via GitHub)" <gi...@apache.org>.

Anush008 commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1940528252

   > Provider [qdrant: 1.0.0rc1](https://pypi.org/project/apache-airflow-providers-qdrant/1.0.0rc1)
    https://github.com/apache/airflow/pull/36805
    
   Tested the DAGs. Works.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "Anush008 (via GitHub)" <gi...@apache.org>.

Anush008 commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1940528959

   > Provider [qdrant: 1.0.0rc1](https://pypi.org/project/apache-airflow-providers-qdrant/1.0.0rc1)
    https://github.com/apache/airflow/pull/36805
    
   Tested DAGs. Works.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.

o-nikolas commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1939765447

   While testing #37137 I encountered an unexpected behaviour with the ECS executor not retrying on failures. I don't think it was caused by this change, it's a separate regression I believe. However, this executor is still experimental so I'm not sure it should block this release.
   CC @ferruzzi @syedahsn @shubham22


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.

eladkal commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938516681

   > @potiuk , @eladkal , hi, Could we please include #37282 into the release?
   
   If PR missed cut off time there is no option to add it. It will have to wait for next release / RC2 if we will have one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "pankajkoti (via GitHub)" <gi...@apache.org>.

pankajkoti commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938596138

   Observing an issue with [common.sql: 1.11.0rc2](https://pypi.org/project/apache-airflow-providers-common-sql/1.11.0rc2)
   
   Posted a [question](https://github.com/apache/airflow/pull/37246#discussion_r1486120101) on the PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "vchiapaikeo (via GitHub)" <gi...@apache.org>.

vchiapaikeo commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1938743542

   https://github.com/apache/airflow/pull/37300 looks good. Also, @Lee-W , thank you for that fix to GKE providers.
   
   However, I have deferrable tasks that should be completing that are not completing due to https://github.com/apache/airflow/pull/37279. Will comment on the PR there.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.

hussein-awala commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1939477002

   Tested #37110 and #37323 and they both work as expected.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "ninsbl (via GitHub)" <gi...@apache.org>.

ninsbl commented on issue #37358:
URL: https://github.com/apache/airflow/issues/37358#issuecomment-1945112880

   #37066 Seems to work fine. The environment variable is acknowledged in the airflow config and long running tasks have been finnishing successfull with Celery 3.6.0rc1.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Status of testing Providers that were prepared on February 12, 2024 [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.

eladkal closed issue #37358: Status of testing Providers that were prepared on February 12, 2024
URL: https://github.com/apache/airflow/issues/37358


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org