You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/04/08 13:00:02 UTC
[GitHub] [airflow] LennyKLB opened a new issue, #22859: Add .sql template extension rendering for BigQueryInsertJobOperator
LennyKLB opened a new issue, #22859:
URL: https://github.com/apache/airflow/issues/22859
### Description
Add `.sql` as value to `template_ext` in `airflow.providers.google.cloud.operators.bigquery.BigQueryInsertJobOperator`
### Use case/motivation
It would be helpful if Jinja would also read `.sql` templates in `airflow.providers.google.cloud.operators.bigquery.BigQueryInsertJobOperator`, especially for its `configuration.query.query` parameter. It will render that parameter as `sql` correctly, but only the outer `configuration` body -- which is based on [Bigquery/rest/v2/Job#jobconfiguration](https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#jobconfiguration) -- is considered for extension-based rendering and only `json` is considered.
I think there is a clear use-case to support templating SQL-only files. I know that `BigQueryExecuteQueryOperator` has this but it's deprecated in favour of `BigQueryInsertJobOperator`.
I currently work around this using a really basic override:
```
class BigQueryTemplateFileOperator(BigQueryInsertJobOperator):
template_ext = (".json", ".sql")
```
However, I'd like to avoid having to replicate this operator everywhere in our codebase if this might actually already be implemented soon.
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on issue #22859: Add .sql template extension rendering for BigQueryInsertJobOperator
Posted by GitBox <gi...@apache.org>.
josh-fell commented on issue #22859:
URL: https://github.com/apache/airflow/issues/22859#issuecomment-1095555636
Closing via #22920.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell closed issue #22859: Add .sql template extension rendering for BigQueryInsertJobOperator
Posted by GitBox <gi...@apache.org>.
josh-fell closed issue #22859: Add .sql template extension rendering for BigQueryInsertJobOperator
URL: https://github.com/apache/airflow/issues/22859
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] LennyKLB commented on issue #22859: Add .sql template extension rendering for BigQueryInsertJobOperator
Posted by GitBox <gi...@apache.org>.
LennyKLB commented on issue #22859:
URL: https://github.com/apache/airflow/issues/22859#issuecomment-1095481943
The change itself is so small that I refrained from adding/editing tests. I found that the BigQuery-related tests didn't feature any file-based templating.
If this is nonetheless still necessary, could someone point me in the right direction in terms of inspiration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on issue #22859: Add .sql template extension rendering for BigQueryInsertJobOperator
Posted by GitBox <gi...@apache.org>.
josh-fell commented on issue #22859:
URL: https://github.com/apache/airflow/issues/22859#issuecomment-1093516567
@LennyKLB Assigned to you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #22859: Add .sql template extension rendering for BigQueryInsertJobOperator
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #22859:
URL: https://github.com/apache/airflow/issues/22859#issuecomment-1092834074
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org