You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/08/27 08:08:54 UTC
[GitHub] [airflow] hankehly opened a new pull request, #26005: Set template_fields on RDS operators
hankehly opened a new pull request, #26005:
URL: https://github.com/apache/airflow/pull/26005
Related: #24099 #25952
### Summary
This PR sets the `template_fields` property on `RdsCreateDbInstanceOperator` and `RdsDeleteDbInstanceOperator` to allow users to programmatically change the database id, instance size, allocated storage, etc..
It also replaces use of `@task` decorated functions with their appropriate operator.
### Todo
- [x] Add `template_fields` to RDS operators
- [x] Update/run system tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ferruzzi commented on a diff in pull request #26005: Set template_fields on RDS operators
Posted by GitBox <gi...@apache.org>.
ferruzzi commented on code in PR #26005:
URL: https://github.com/apache/airflow/pull/26005#discussion_r957548732
##########
airflow/providers/amazon/aws/operators/rds.py:
##########
@@ -585,6 +585,8 @@ class RdsCreateDbInstanceOperator(RdsBaseOperator):
:param wait_for_completion: If True, waits for creation of the DB instance to complete. (default: True)
"""
+ template_fields = ("db_instance_identifier", "db_instance_class", "rds_kwargs")
Review Comment:
```suggestion
template_fields = ("db_instance_identifier", "db_instance_class", "engine", "rds_kwargs")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ferruzzi commented on a diff in pull request #26005: Set template_fields on RDS operators
Posted by GitBox <gi...@apache.org>.
ferruzzi commented on code in PR #26005:
URL: https://github.com/apache/airflow/pull/26005#discussion_r957548732
##########
airflow/providers/amazon/aws/operators/rds.py:
##########
@@ -585,6 +585,8 @@ class RdsCreateDbInstanceOperator(RdsBaseOperator):
:param wait_for_completion: If True, waits for creation of the DB instance to complete. (default: True)
"""
+ template_fields = ("db_instance_identifier", "db_instance_class", "rds_kwargs")
Review Comment:
Could you throw `engine` in there while you are at it?
```suggestion
template_fields = ("db_instance_identifier", "db_instance_class", "engine", "rds_kwargs")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil merged pull request #26005: Set template_fields on RDS operators
Posted by GitBox <gi...@apache.org>.
kaxil merged PR #26005:
URL: https://github.com/apache/airflow/pull/26005
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ferruzzi commented on a diff in pull request #26005: Set template_fields on RDS operators
Posted by GitBox <gi...@apache.org>.
ferruzzi commented on code in PR #26005:
URL: https://github.com/apache/airflow/pull/26005#discussion_r957547444
##########
tests/system/providers/amazon/aws/example_rds_event.py:
##########
@@ -41,35 +43,6 @@ def create_sns_topic(env_id) -> str:
return boto3.client('sns').create_topic(Name=f'{env_id}-topic')['TopicArn']
-@task
-def create_rds_instance(db_name, instance_name) -> None:
Review Comment:
Thanks for this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on a diff in pull request #26005: Set template_fields on RDS operators
Posted by GitBox <gi...@apache.org>.
hankehly commented on code in PR #26005:
URL: https://github.com/apache/airflow/pull/26005#discussion_r957895161
##########
airflow/providers/amazon/aws/operators/rds.py:
##########
@@ -585,6 +585,8 @@ class RdsCreateDbInstanceOperator(RdsBaseOperator):
:param wait_for_completion: If True, waits for creation of the DB instance to complete. (default: True)
"""
+ template_fields = ("db_instance_identifier", "db_instance_class", "rds_kwargs")
Review Comment:
Added, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on a diff in pull request #26005: Set template_fields on RDS operators
Posted by GitBox <gi...@apache.org>.
hankehly commented on code in PR #26005:
URL: https://github.com/apache/airflow/pull/26005#discussion_r956554307
##########
tests/system/providers/amazon/aws/example_rds_event.py:
##########
@@ -41,35 +43,6 @@ def create_sns_topic(env_id) -> str:
return boto3.client('sns').create_topic(Name=f'{env_id}-topic')['TopicArn']
-@task
-def create_rds_instance(db_name, instance_name) -> None:
Review Comment:
I made the same updates in each system test file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on pull request #26005: Set template_fields on RDS operators
Posted by GitBox <gi...@apache.org>.
hankehly commented on PR #26005:
URL: https://github.com/apache/airflow/pull/26005#issuecomment-1229183627
@o-nikolas @vincbeck @ferruzzi @potiuk
Please review this at your earliest convenience.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ferruzzi commented on a diff in pull request #26005: Set template_fields on RDS operators
Posted by GitBox <gi...@apache.org>.
ferruzzi commented on code in PR #26005:
URL: https://github.com/apache/airflow/pull/26005#discussion_r957548285
##########
airflow/providers/amazon/aws/operators/rds.py:
##########
@@ -585,6 +585,8 @@ class RdsCreateDbInstanceOperator(RdsBaseOperator):
:param wait_for_completion: If True, waits for creation of the DB instance to complete. (default: True)
"""
+ template_fields = ("db_instance_identifier", "db_instance_class", "rds_kwargs")
Review Comment:
Could you throw 'engine' in here while you are at it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on pull request #26005: Set template_fields on RDS operators
Posted by GitBox <gi...@apache.org>.
hankehly commented on PR #26005:
URL: https://github.com/apache/airflow/pull/26005#issuecomment-1234920752
@o-nikolas @vincbeck @ferruzzi @potiuk
Could someone please merge this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ferruzzi commented on pull request #26005: Set template_fields on RDS operators
Posted by GitBox <gi...@apache.org>.
ferruzzi commented on PR #26005:
URL: https://github.com/apache/airflow/pull/26005#issuecomment-1231001857
Cool, thanks for the change. LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org