You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/09/28 14:52:15 UTC
[GitHub] [airflow] potiuk opened a new issue, #26752: Status of testing Providers that were prepared on September 28, 2022
potiuk opened a new issue, #26752:
URL: https://github.com/apache/airflow/issues/26752
### Body
I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?
Let us know in the comment, whether the issue is addressed.
Those are providers that require testing as there were some substantial changes introduced:
## Provider [alibaba: 2.1.0rc2](https://pypi.org/project/apache-airflow-providers-alibaba/2.1.0rc2)
- [ ] [Auto tail file logs in Web UI (#26169)](https://github.com/apache/airflow/pull/26169): @jens-scheffler-bosch
## Provider [amazon: 6.0.0rc2](https://pypi.org/project/apache-airflow-providers-amazon/6.0.0rc2)
- [ ] [Remove Amazon S3 Connection Type (#25980)](https://github.com/apache/airflow/pull/25980): @Taragolis
- [ ] [Add RdsDbSensor to amazon provider package (#26003)](https://github.com/apache/airflow/pull/26003): @hankehly
- [ ] [Set template_fields on RDS operators (#26005)](https://github.com/apache/airflow/pull/26005): @hankehly
- [ ] [Auto tail file logs in Web UI (#26169)](https://github.com/apache/airflow/pull/26169): @jens-scheffler-bosch
- [ ] [Fix SageMakerEndpointConfigOperator's return value (#26541)](https://github.com/apache/airflow/pull/26541): @ferruzzi
- [ ] [EMR Serverless Fix for Jobs marked as success even on failure (#26218)](https://github.com/apache/airflow/pull/26218): @syedahsn
- [ ] [Fix AWS Connection warn condition for invalid `profile_name` argument (#26464)](https://github.com/apache/airflow/pull/26464): @Taragolis
- [ ] [Athena and EMR operator max_retries mix-up fix (#25971)](https://github.com/apache/airflow/pull/25971): @syedahsn
- [ ] [Fixes SageMaker operator return values (#23628)](https://github.com/apache/airflow/pull/23628): @ferruzzi
- [ ] [Remove redundant catch exceptions in Amazon Log Task Handlers (#26442)](https://github.com/apache/airflow/pull/26442): @Taragolis
- [ ] [Remove duplicated connection-type within the provider (#26628)](https://github.com/apache/airflow/pull/26628): @Taragolis
## Provider [apache.drill: 2.2.1rc2](https://pypi.org/project/apache-airflow-providers-apache-drill/2.2.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [apache.druid: 3.2.1rc2](https://pypi.org/project/apache-airflow-providers-apache-druid/3.2.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [apache.hive: 4.0.1rc2](https://pypi.org/project/apache-airflow-providers-apache-hive/4.0.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [apache.pinot: 3.2.1rc2](https://pypi.org/project/apache-airflow-providers-apache-pinot/3.2.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
- [ ] [Fix PinotDB dependencies (#26705)](https://github.com/apache/airflow/pull/26705): @potiuk
## Provider [databricks: 3.3.0rc2](https://pypi.org/project/apache-airflow-providers-databricks/3.3.0rc2)
- [ ] [DatabricksSubmitRunOperator dbt task support (#25623)](https://github.com/apache/airflow/pull/25623): @gaborratky-db
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
- [ ] [Remove duplicated connection-type within the provider (#26628)](https://github.com/apache/airflow/pull/26628): @Taragolis
- [ ] [Databricks: fix provider name in the User-Agent string (#25873)](https://github.com/apache/airflow/pull/25873): @alexott
## Provider [dbt.cloud: 2.2.0rc2](https://pypi.org/project/apache-airflow-providers-dbt-cloud/2.2.0rc2)
- [ ] [Add `DbtCloudListJobsOperator` (#26475)](https://github.com/apache/airflow/pull/26475): @phanikumv
## Provider [docker: 3.2.0rc2](https://pypi.org/project/apache-airflow-providers-docker/3.2.0rc2)
- [ ] [Add logging options to docker operator (#26653)](https://github.com/apache/airflow/pull/26653): @nshetty15
- [ ] [Add pre-commit hook for custom_operator_name (#25786)](https://github.com/apache/airflow/pull/25786): @uranusjr
- [ ] [Implement ExternalPythonOperator (#25780)](https://github.com/apache/airflow/pull/25780): @potiuk
## Provider [elasticsearch: 4.2.1rc2](https://pypi.org/project/apache-airflow-providers-elasticsearch/4.2.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [exasol: 4.0.1rc2](https://pypi.org/project/apache-airflow-providers-exasol/4.0.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [google: 8.4.0rc2](https://pypi.org/project/apache-airflow-providers-google/8.4.0rc2)
- [ ] [Add BigQuery Column and Table Check Operators (#26368)](https://github.com/apache/airflow/pull/26368): @denimalpaca
- [ ] [Add deferrable big query operators and sensors (#26156)](https://github.com/apache/airflow/pull/26156): @phanikumv
- [ ] [Add `output` property to MappedOperator (#25604)](https://github.com/apache/airflow/pull/25604): @josh-fell
- [ ] [Added append_job_name parameter to DataflowTemplatedJobStartOperator (#25746)](https://github.com/apache/airflow/pull/25746): @Voldurk
- [ ] [Adding a parameter for exclusion of trashed files in GoogleDriveHook (#25675)](https://github.com/apache/airflow/pull/25675): @alexkruc
- [ ] [Cloud Data Loss Prevention Operators assets (AIP-47) (#26618)](https://github.com/apache/airflow/pull/26618): @bkossakowska
- [ ] [Cloud Storage Transfer Operators assets & system tests migration (AIP-47) (#26072)](https://github.com/apache/airflow/pull/26072): @bkossakowska
- [ ] [Merge deferrable BigQuery operators to exisitng one (#26433)](https://github.com/apache/airflow/pull/26433): @lwyszomi
- [ ] [specifying project id when calling wait_for_operation in delete/create cluster (#26418)](https://github.com/apache/airflow/pull/26418): @SPTKL
- [ ] [Auto tail file logs in Web UI (#26169)](https://github.com/apache/airflow/pull/26169): @jens-scheffler-bosch
- [ ] [Cloud Functions Operators assets & system tests migration (AIP-47) (#26073)](https://github.com/apache/airflow/pull/26073): @bkossakowska
- [ ] [GCSToBigQueryOperator Resolve `max_id_key` job retrieval and xcom return (#26285)](https://github.com/apache/airflow/pull/26285): @patricker
- [ ] [Allow for the overriding of `stringify_dict` for json export format on BaseSQLToGCSOperator (#26277)](https://github.com/apache/airflow/pull/26277): @patricker
- [ ] [Use common url base for Google links (#26057)](https://github.com/apache/airflow/pull/26057): @bhirsz
- [ ] [Cloud Video Intelligence Operators assets & system tests migration (AIP-47) (#26132)](https://github.com/apache/airflow/pull/26132): @bkossakowska
- [ ] [Life Science assets & system tests migration (AIP-47) (#25548)](https://github.com/apache/airflow/pull/25548): @bkossakowska
- [ ] [GCSToBigQueryOperator allow for schema_object in alternate GCS Bucket (#26190)](https://github.com/apache/airflow/pull/26190): @patricker
- [ ] [Use AsyncClient for Composer Operators in deferrable mode (#25951)](https://github.com/apache/airflow/pull/25951): @lwyszomi
- [ ] [Fix BigQueryCreateEmptyDatasetOperator to use the provided project_id (#25984)](https://github.com/apache/airflow/pull/25984): @phanikumv
- [ ] [Cloud Build assets & system tests migration (AIP-47) (#25895)](https://github.com/apache/airflow/pull/25895): @bkossakowska
- [ ] [Dataproc submit job operator async (#25302)](https://github.com/apache/airflow/pull/25302): @bjankie1
- [ ] [Support project_id argument in BigQueryGetDataOperator (#25782)](https://github.com/apache/airflow/pull/25782): @sudohainguyen
- [ ] [Fix JSONDecodeError in Datafusion operators (#26202)](https://github.com/apache/airflow/pull/26202): @VladaZakharova
- [ ] [Fixed never ending loop in CreateWorkflowInvocation (#25737)](https://github.com/apache/airflow/pull/25737): @lwyszomi
- [ ] [Fix error message on gcs.py (#26570)](https://github.com/apache/airflow/pull/26570): @Corea
- [ ] [Don't throw an exception when a BQ cursor job has no schema (#26096)](https://github.com/apache/airflow/pull/26096): @patricker
- [ ] [Google Cloud Tasks Sensor for queue being empty (#25622)](https://github.com/apache/airflow/pull/25622): @vinitpayal
- [ ] [SQL to GCS operators update docs for parquet format (#25878)](https://github.com/apache/airflow/pull/25878): @pierrejeambrun
- [ ] [Limit Google Protobuf for compatibility with biggtable client (#25886)](https://github.com/apache/airflow/pull/25886): @potiuk
- [ ] [Make GoogleBaseHook credentials functions public (#25785)](https://github.com/apache/airflow/pull/25785): @feluelle
- [ ] [Consolidate to one `schedule` param (#25410)](https://github.com/apache/airflow/pull/25410): @dstandish
## Provider [jdbc: 3.2.1rc2](https://pypi.org/project/apache-airflow-providers-jdbc/3.2.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [jira: 3.1.0rc2](https://pypi.org/project/apache-airflow-providers-jira/3.1.0rc2)
- [ ] [Deprecate jira provider in favor of atlassian.jira provider (#25930)](https://github.com/apache/airflow/pull/25930): @eladkal
## Provider [microsoft.azure: 4.3.0rc2](https://pypi.org/project/apache-airflow-providers-microsoft-azure/4.3.0rc2)
- [ ] [Add DataFlow operations to Azure DataFactory hook (#26345)](https://github.com/apache/airflow/pull/26345): @phanikumv
- [ ] [Add `network_profile` param in AzureContainerInstancesOperator (#26117)](https://github.com/apache/airflow/pull/26117): @pankajastro
- [ ] [Implement Azure synapse operator (#26038)](https://github.com/apache/airflow/pull/26038): @phanikumv
- [ ] [Auto tail file logs in Web UI (#26169)](https://github.com/apache/airflow/pull/26169): @jens-scheffler-bosch
- [ ] [Implement Azure Service Bus Topic (Create, Delete) Operators (#25436)](https://github.com/apache/airflow/pull/25436): @bharanidharan14
- [ ] [Fix AzureBatchOperator false negative status (#25844)](https://github.com/apache/airflow/pull/25844): @pankajastro
## Provider [microsoft.mssql: 3.2.1rc2](https://pypi.org/project/apache-airflow-providers-microsoft-mssql/3.2.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [mysql: 3.2.1rc2](https://pypi.org/project/apache-airflow-providers-mysql/3.2.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [odbc: 3.1.2rc2](https://pypi.org/project/apache-airflow-providers-odbc/3.1.2rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [oracle: 3.4.0rc2](https://pypi.org/project/apache-airflow-providers-oracle/3.4.0rc2)
- [ ] [Add oracledb thick mode support for oracle provider (#26576)](https://github.com/apache/airflow/pull/26576): @pauldalewilliams
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [postgres: 5.2.2rc2](https://pypi.org/project/apache-airflow-providers-postgres/5.2.2rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
- [ ] [Rename schema to database in PostgresHook (#26436)](https://github.com/apache/airflow/pull/26436): @feluelle
- [ ] [Revert "Rename schema to database in `PostgresHook` (#26436)" (#26734)](https://github.com/apache/airflow/pull/26734): @jedcunningham
## Provider [qubole: 3.2.1rc2](https://pypi.org/project/apache-airflow-providers-qubole/3.2.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
- [ ] [D400 first line should end with period batch02 (#25268)](https://github.com/apache/airflow/pull/25268): @edithturn
## Provider [sftp: 4.1.0rc2](https://pypi.org/project/apache-airflow-providers-sftp/4.1.0rc2)
- [ ] [SFTPOperator - add support for list of file paths (#26666)](https://github.com/apache/airflow/pull/26666): @pauldalewilliams
## Provider [slack: 6.0.0rc2](https://pypi.org/project/apache-airflow-providers-slack/6.0.0rc2)
- [ ] [Refactor `SlackWebhookOperator`: Get rid of mandatory http-provider dependency (#26648)](https://github.com/apache/airflow/pull/26648): @Taragolis
- [ ] [Refactor SlackWebhookHook in order to use `slack_sdk` instead of HttpHook methods (#26452)](https://github.com/apache/airflow/pull/26452): @Taragolis
- [ ] [Move send_file method into SlackHook (#26118)](https://github.com/apache/airflow/pull/26118): @Taragolis
- [ ] [Refactor Slack API Hook and add new Connection type (#25852)](https://github.com/apache/airflow/pull/25852): @Taragolis
- [ ] [Remove unsafe imports in Slack API Connection (#26459)](https://github.com/apache/airflow/pull/26459): @Taragolis
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
## Provider [snowflake: 3.3.0rc2](https://pypi.org/project/apache-airflow-providers-snowflake/3.3.0rc2)
- [ ] [Add custom handler param in SnowflakeOperator (#25983)](https://github.com/apache/airflow/pull/25983): @pankajastro
- [ ] [Fix wrong deprecation warning for `S3ToSnowflakeOperator` (#26047)](https://github.com/apache/airflow/pull/26047): @eladkal
## Provider [ssh: 3.2.0rc2](https://pypi.org/project/apache-airflow-providers-ssh/3.2.0rc2)
- [ ] [SSHHook: Saving new host keys as described in docs (#25979)](https://github.com/apache/airflow/pull/25979): @doiken
## Provider [trino: 4.1.0rc2](https://pypi.org/project/apache-airflow-providers-trino/4.1.0rc2)
- [ ] [trino: Support CertificateAuthentication in the trino hook (#26246)](https://github.com/apache/airflow/pull/26246): @c2zwdjnlcg
## Provider [vertica: 3.2.1rc2](https://pypi.org/project/apache-airflow-providers-vertica/3.2.1rc2)
- [ ] [Add common-sql lower bound for common-sql (#25789)](https://github.com/apache/airflow/pull/25789): @potiuk
The guidelines on how to test providers can be found in
[Verify providers by contributors](https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#verify-by-contributors)
### Committer
- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
Taragolis commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1262928362
#26169 - @jens-scheffler-bosch @potiuk It work with `S3TaskHandler` in breeze with some small artifacts: during fetching logs continuously printed `*** Falling back to local log`
```text
[2022-09-29, 23:02:50 UTC] {subprocess.py:93} INFO - 70
[2022-09-29, 23:02:51 UTC] {subprocess.py:93} INFO - 71
[2022-09-29, 23:02:52 UTC] {subprocess.py:93} INFO - 72
*** Falling back to local log
[2022-09-29, 23:02:53 UTC] {subprocess.py:93} INFO - 73
[2022-09-29, 23:02:54 UTC] {subprocess.py:93} INFO - 74
*** Falling back to local log
[2022-09-29, 23:02:55 UTC] {subprocess.py:93} INFO - 75
[2022-09-29, 23:02:56 UTC] {subprocess.py:93} INFO - 76
*** Falling back to local log
[2022-09-29, 23:02:57 UTC] {subprocess.py:93} INFO - 77
[2022-09-29, 23:02:58 UTC] {subprocess.py:93} INFO - 78
*** Falling back to local log
[2022-09-29, 23:02:59 UTC] {subprocess.py:93} INFO - 79
[2022-09-29, 23:03:00 UTC] {subprocess.py:93} INFO - 80
```
If something wrong with connection than also spam with this message
https://github.com/apache/airflow/blob/ce071172e22fba018889db7dcfac4a4d0fc41cda/airflow/providers/amazon/aws/log/s3_task_handler.py#L112-L113
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1264369717
> If something wrong with connection than also spam with this message
@Taragolis I think it's a small thing, they indicate that something is wrong with configuration. We might want to configure it in Breeze to make it less annoying.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261415183
> I was testing https://github.com/apache/airflow/pull/26285, and there is something off with the way I wrote the value to XCom. The Max Value shows correctly in the text logs, but when it's written to XCom there is some kind of serialization issue, and it throws an error in the web ui when I try to view it.
Hmm. I think we need to take closer look. Can you describe a way you did it? It kinda looks like you have a "table" object serialized somehow rather than primitive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jens-scheffler-bosch commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
jens-scheffler-bosch commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261495246
I'd like to contribute to the RC tests but for all 4 provider packages touched by my PR I just did code checks for consistency of the logging on API. I have no back-end environment for alibaba, azure, google and amazon in my availability :-( Is somebody besides me able to test?
Test is simple if you have a running environment, can use any existing longer running DAG and check the log view ,as described in PR https://github.com/apache/airflow/pull/26169
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261533733
> I'd like to contribute to the RC tests but for all 4 provider packages touched by my PR I just did code checks for consistency of the logging on API. I have no back-end environment for alibaba, azure, google and amazon in my availability :-( Is somebody besides me able to test? Test is simple if you have a running environment, can use any existing longer running DAG and check the log view ,as described in PR #26169
Yep. If others can test it - it would be great (especially while testing other issue). Thanks for notifying :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alexkruc commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
alexkruc commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1263333204
#25675 - verified
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1264368343
> #25852: Try to test in Airflow 2.2.5 and got an issues that it not work correctly with extra in the UI:
>
> 1. Do not allow create connection with empty value for timeout
> 2. If field not specified than write empty value to the DB, as result slack_sdk can't build correct URL
>
> Need to check is it affect 2.3, because in 2.4 no problem. No problem with connection if it created via API or provided as URL (env var, secrets backend)
Ok. I will wait for the check to relase an RC2 and will remove Amazon alongside Google from the release and we will have RC2 together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261532586
> This serialized row then causes issues in XCom. I'll see about fixing it.
Thanks for flagging/testing/fixing. We can remove google provider and re-release it as rc2 (but let's continue testing it for now by others).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
Taragolis commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1264472911
> I am releasing all but google and slack. I am at ApacheCon now. And I will look for @Taragolis comments on the connection for Slack to release RC2s
@potiuk Let me check is might only related to specific version of Airflow, like only for 2.2.5
I have an idea about workaround if I sure that it related for all 2.2.x
Also as an option this version of provider might be released after October 11 with min airflow version 2.3.
In this case we do not need to create workaround which we remove in next version of provider
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1264494097
It might still be cool to fix it as the 'last' version working for 2.2.5 :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bharanidharan14 commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
bharanidharan14 commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1262155202
Tested both [Azure Service Bus (Create and Delete) Topic Operators](https://github.com/apache/airflow/pull/25436), working fine 👍
<img width="405" alt="Screenshot 2022-09-29 at 5 07 08 PM" src="https://user-images.githubusercontent.com/94612827/193021569-73351aeb-de52-48bd-9b33-a2714f92b039.png">
<img width="251" alt="Screenshot 2022-09-29 at 5 07 18 PM" src="https://user-images.githubusercontent.com/94612827/193021575-ee4f5c6a-599a-4320-97b7-e6e4a16c478d.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261109556
Copy-pasted a wrong command :). All links are fixed now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
Taragolis commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1265140345
> It might still be cool to fix it as the 'last' version working for 2.2.5 :)
I try to check it again **today** to determine with which version it actually has a problem (personally I have a problem with create connection in all providers which define additional fields in `get_connection_form_widgets` on different 2.2.5 environments and do not have in 2.3.4)
And create a workaround which allow to create Slack Connection in the UI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
Taragolis commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1263625826
#25852: Try to test in Airflow 2.2.5 and got an issues that it not work correctly with extra in the UI:
1. Do not allow create connection with empty value for timeout
2. If field not specified than write empty value to the DB, as result slack_sdk can't build correct URL
Need to check is it affect 2.3, because in 2.4 no problem. No problem with connection if it created via API or provided as URL (env var, secrets backend)
https://github.com/apache/airflow/blob/4b456d56c9962c8428ad07185ca054fa88f66a0d/airflow/providers/slack/hooks/slack.py#L297-L314
#26648: Same as previous
#26118: Working fine
#26459: Unsafe imports not presented. Fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pauldalewilliams commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
pauldalewilliams commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261655569
I tested both #26666 and #26576 - both are good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
Taragolis commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1264462482
> @Taragolis I think it's a small thing, they indicate that something is wrong with configuration. We might want to configure it in Breeze to make it less annoying.
I also think it a small things. I use breeze for test because part related to actual print a log during task execution in a core and seem like it not added in any current releases yet (milestones is Airflow 2.5.0)
The issue itself how work S3TaskHandler (as well as other Object storage):
1. During task execution logs received from worker and after task execution write to S3
2. If logs received from logger it print message that `*** Falling back to local log`
3. If also user incorrectly configure AWS credentials it also write `*** Failed to verify remote log exists`
So.. _Auto tail file logs_ just continuously execute `_read` which write this message (only in the UI, this information not presented into the actual logs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
hankehly commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1264544094
Manually tested the following on 2022/10/02, both OK
#26003
#26005
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] patricker commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
patricker commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261334878
I've tested #26277, it's good.
I was testing #26285, and there is something off with the way I wrote the value to XCom. The Max Value shows correctly in the text logs, but when it's written to XCom there is some kind of serialization issue, and it throws an error in the web ui when I try to view it.
```
File "/usr/local/lib/python3.9/site-packages/airflow/models/xcom.py", line 600, in deserialize_value
return pickle.loads(result.value)
File "/usr/local/lib/python3.9/site-packages/google/cloud/bigquery/table.py", line 1409, in __getattr__
value = self._xxx_field_to_index.get(name)
File "/usr/local/lib/python3.9/site-packages/google/cloud/bigquery/table.py", line 1409, in __getattr__
value = self._xxx_field_to_index.get(name)
File "/usr/local/lib/python3.9/site-packages/google/cloud/bigquery/table.py", line 1409, in __getattr__
value = self._xxx_field_to_index.get(name)
[Previous line repeated 957 more times]
RecursionError: maximum recursion depth exceeded
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] c2zwdjnlcg commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
c2zwdjnlcg commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261336383
Tested trino: 4.1.0rc1 and did not hit any issues
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nshetty15 commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
nshetty15 commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1262505540
#26653 - (apache-airflow-providers-docker 3.2.0rc1) Looks good. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] patricker commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
patricker commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261689446
@potiuk I'll try and get my other two tested tomorrow. Here is the PR with the fix for #26285, https://github.com/apache/airflow/pull/26768.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
Taragolis commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1262828270
#26442 - Only could proof that changed code persists. It is hardly possible to make loggers fail on create hook
#26628 - Check that `amazon` and `databricks` do not change anything in airflow.providers_manager.ProvidersManager after changes. (related to removed listed hooks)
#25980 - As expected deprecated `s3` connection can be use in aws hooks with warnings. No connection in the UI anymore.
#26464 - Working as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
josh-fell commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1263919394
#25604 looks good!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1264370641
> I ran into an issue while testing #26096, but which isn't a "bug" with the original PR's implementation. Just a new bug in a similar part of the code. I cut a PR for it, but doesn't have to go into RC2 if it doesn't make it.
> #26786
We are going to have RC2 of Google and I just merged #26786 so it will make it :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1264371340
Thanks all who tested it ! As usual we have great help from our community.
And thanks a lot for all the "critical" checks and doubts. Keep having them. It helps to make the right decisions for release!
I am releasing all but google and slack. I am at ApacheCon now. And I will look for @Taragolis comments on the connection for Slack to release RC2s
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #26752: Status of testing Providers that were prepared on September 28, 2022
URL: https://github.com/apache/airflow/issues/26752
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261107744
Thanks Elad :). Fixing :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] patricker commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
patricker commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261522787
> > I was testing #26285, and there is something off with the way I wrote the value to XCom. The Max Value shows correctly in the text logs, but when it's written to XCom there is some kind of serialization issue, and it throws an error in the web ui when I try to view it.
>
> Hmm. I think we need to take closer look. Can you describe a way you did it? It kinda looks like you have a "table" object serialized somehow rather than primitive.
@potiuk In looking at the code again, and the actual Log message, it looks like I accidentally tried to serialize a whole table row instead of the value of the first column of the table row.
>Loaded BQ data with max data-landing.api.users_20220928_.updated_at=Row((datetime.datetime(2022, 9, 28, 16, 57, 18, 752695, tzinfo=datetime.timezone.utc),), {'f0_': 0})
This serialized row then causes issues in XCom. I'll see about fixing it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1261069255
links are wrong? all are RC1 not RC2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] patricker commented on issue #26752: Status of testing Providers that were prepared on September 28, 2022
Posted by GitBox <gi...@apache.org>.
patricker commented on issue #26752:
URL: https://github.com/apache/airflow/issues/26752#issuecomment-1262726383
I tested #29096 and #26190. Both tested fine.
I ran into a _related_ issue while testing #29096, but which isn't a "bug" with the original PR's implementation. Just a new bug in a similar part of the code. I cut a PR for it, but doesn't have to go into RC2 if it doesn't make it.
https://github.com/apache/airflow/pull/26786
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org