You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/24 13:24:27 UTC
[GitHub] [airflow] MohammedDiab opened a new issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
MohammedDiab opened a new issue #10508:
URL: https://github.com/apache/airflow/issues/10508
**Description**
it is really useful to have the body parameter as templated field, the parameter can have attributes related to file path, table name, database and many more which usually comes from a previous operator that do the work and then send the information to the CloudSqlInstanceImportOperator
**Use case / motivation**
I have the following case :
1. GoogleDataFlowOperator that runs first to do some transformations and then stores the result as csv file on Google Bucket
2. The CloudSqlInstanceImportOperator is supposed to load the csv into a a cloud sql instance but the import body isn't a templated field so it is challenging to pass the files dynamically to the operator, I had to override the operator just to add the body as templated field
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] MohammedDiab commented on issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
Posted by GitBox <gi...@apache.org>.
MohammedDiab commented on issue #10508:
URL: https://github.com/apache/airflow/issues/10508#issuecomment-699899537
@yashhd the body field was already there, the issue was you couldn't render Airflow variables inside the `body` parameter. examples of Airflow variables here https://airflow.apache.org/docs/stable/macros-ref.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #10508:
URL: https://github.com/apache/airflow/issues/10508#issuecomment-679128477
> If you create pull-request, you don't have to create an issue first, but if you want, you can do it. Creating an issue will allow you to collect feedback or share plans with other people.
https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #10508:
URL: https://github.com/apache/airflow/issues/10508#issuecomment-679128862
Part-of: https://github.com/apache/airflow/issues/8893
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yashhd commented on issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
Posted by GitBox <gi...@apache.org>.
yashhd commented on issue #10508:
URL: https://github.com/apache/airflow/issues/10508#issuecomment-699618524
@MohammedDiab Hi I'm a little new to airflow and have been implementing a project of my own where I'm using cloudsqlinstanceimport operator. What exactly did you mean by having the body parameter as a templated field?
In this link - https://airflow.apache.org/docs/stable/howto/operator/gcp/sql.html#cloudsqlinstanceimportoperator , the body parameter is already present. So were you saying that it wouldn't have worked before your pull request got merged?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yashhd commented on issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
Posted by GitBox <gi...@apache.org>.
yashhd commented on issue #10508:
URL: https://github.com/apache/airflow/issues/10508#issuecomment-700629468
@MohammedDiab Oh yes, I understood. Thanks for clarifying!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #10508:
URL: https://github.com/apache/airflow/issues/10508#issuecomment-679126537
Would you be so kind to add a PR with this change? I am happy to review it once you ping me - this seems to be super -easy change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #10508:
URL: https://github.com/apache/airflow/issues/10508#issuecomment-679123933
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] MohammedDiab commented on issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
Posted by GitBox <gi...@apache.org>.
MohammedDiab commented on issue #10508:
URL: https://github.com/apache/airflow/issues/10508#issuecomment-679139408
@potiuk here is the PR,
https://github.com/apache/airflow/pull/10510
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #10508:
URL: https://github.com/apache/airflow/issues/10508
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] MohammedDiab commented on issue #10508: Adding 'body' as templated field to CloudSqlInstanceImportOperator
Posted by GitBox <gi...@apache.org>.
MohammedDiab commented on issue #10508:
URL: https://github.com/apache/airflow/issues/10508#issuecomment-679127996
@potiuk yes sure, i'm currently creating the PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org