You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "vincbeck (via GitHub)" <gi...@apache.org> on 2023/02/09 16:48:42 UTC
[GitHub] [airflow] vincbeck opened a new pull request, #29443: Use an Amazon Linux 2 image to run EMR job
vincbeck opened a new pull request, #29443:
URL: https://github.com/apache/airflow/pull/29443
For internal operational reasons, it is better to use Amazon Linux 2 images on EC2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on a diff in pull request #29443: Use an Amazon Linux 2 image to run EMR job
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on code in PR #29443:
URL: https://github.com/apache/airflow/pull/29443#discussion_r1101801403
##########
tests/system/providers/amazon/aws/example_emr.py:
##########
@@ -90,6 +90,16 @@
# [END howto_operator_emr_steps_config]
+@task
+def get_ami_id():
+ """
+ Returns an AL2 AMI compatible with EMR
+ """
+ return boto3.client("ssm").get_parameter(
+ Name="/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-ebs",
+ )["Parameter"]["Value"]
Review Comment:
I like that. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on a diff in pull request #29443: Use an Amazon Linux 2 image to run EMR job
Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis commented on code in PR #29443:
URL: https://github.com/apache/airflow/pull/29443#discussion_r1101768568
##########
tests/system/providers/amazon/aws/example_emr.py:
##########
@@ -90,6 +90,16 @@
# [END howto_operator_emr_steps_config]
+@task
+def get_ami_id():
+ """
+ Returns an AL2 AMI compatible with EMR
+ """
+ return boto3.client("ssm").get_parameter(
+ Name="/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-ebs",
+ )["Parameter"]["Value"]
Review Comment:
```suggestion
return SsmHook(aws_conn_id=None).get_parameter_value(
"/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-ebs"
)
```
Just an idea for use SsmHook here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on a diff in pull request #29443: Use an Amazon Linux 2 image to run EMR job
Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis commented on code in PR #29443:
URL: https://github.com/apache/airflow/pull/29443#discussion_r1101768568
##########
tests/system/providers/amazon/aws/example_emr.py:
##########
@@ -90,6 +90,16 @@
# [END howto_operator_emr_steps_config]
+@task
+def get_ami_id():
+ """
+ Returns an AL2 AMI compatible with EMR
+ """
+ return boto3.client("ssm").get_parameter(
+ Name="/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-ebs",
+ )["Parameter"]["Value"]
Review Comment:
```suggestion
return SsmHook(aws_conn_id=None).get_parameter_value(
"/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-ebs"
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas merged pull request #29443: Use an Amazon Linux 2 image to run EMR job
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas merged PR #29443:
URL: https://github.com/apache/airflow/pull/29443
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org