You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/02/15 21:54:58 UTC
[GitHub] [airflow] eladkal opened a new issue #21599: Explore the possibility of generic `SqlToGcsOperator`
eladkal opened a new issue #21599:
URL: https://github.com/apache/airflow/issues/21599
### Body
Recently `SqlToS3Operator` was added in PR https://github.com/apache/airflow/pull/20807 (It also deprecated `MySqlToS3Operator`)
In Google provider Currently we have [BaseSQLToGCSOperator](https://github.com/apache/airflow/blob/1b568d73e1dfb838a3a0446e3a6063b9f27f04b8/airflow/providers/google/cloud/transfers/sql_to_gcs.py#L36) with `PrestoToGCSOperator`, `MSSQLToGCSOperator`, `MySQLToGCSOperator`, `OracleToGCSOperator`, `PostgresToGCSOperator`, `TrinoToGCSOperator`, `DummySQLToGCSOperator`
This task is to check if we can have a generic `SqlToGCSOperator` just like there is for AWS so we won't need class per each provider...
### Committer
- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kadai0308 commented on issue #21599: Explore the possibility of generic `SqlToGcsOperator`
Posted by GitBox <gi...@apache.org>.
kadai0308 commented on issue #21599:
URL: https://github.com/apache/airflow/issues/21599#issuecomment-1046180277
Hi eladkal, Can I take this one as my first issue? I already went through the PR [#20807](https://github.com/apache/airflow/pull/20807)
and later I will check each implementation of XXXToGCSOperator to see is it possible to implement a generic `SqlToGCSOperator` to replace all of them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #21599: Explore the possibility of generic `SqlToGcsOperator`
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #21599:
URL: https://github.com/apache/airflow/issues/21599#issuecomment-1046181651
sure assigned to you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kadai0308 commented on issue #21599: Explore the possibility of generic `SqlToGcsOperator`
Posted by GitBox <gi...@apache.org>.
kadai0308 commented on issue #21599:
URL: https://github.com/apache/airflow/issues/21599#issuecomment-1046504788
Hi @eladkal, when I investigated the presto_to_gcs operator & trino_to_gcs operator found the key of integer & float in their type map is different with others to_gcs operator, I think it might be a bug, so raise a PR #21700.
Can you help me check on that? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org