You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "o-nikolas (via GitHub)" <gi...@apache.org> on 2023/02/02 20:36:25 UTC
[GitHub] [airflow] o-nikolas commented on a diff in pull request #29341: Add clarification about AWS Connection behaviour
o-nikolas commented on code in PR #29341:
URL: https://github.com/apache/airflow/pull/29341#discussion_r1095054455
##########
docs/apache-airflow-providers-amazon/connections/aws.rst:
##########
@@ -20,8 +20,21 @@
Amazon Web Services Connection
==============================
-The Amazon Web Services connection type enables the :ref:`AWS Integrations
-<AWS>`.
+The Amazon Web Services connection type enables the :ref:`AWS Integrations <AWS>`.
+
+.. important:: Amazon Web Services Connection could be tested in the UI/API or by call
Review Comment:
```suggestion
.. important:: The Amazon Web Services Connection can be tested in the UI/API or by calling
```
##########
docs/apache-airflow-providers-amazon/connections/aws.rst:
##########
@@ -20,8 +20,21 @@
Amazon Web Services Connection
==============================
-The Amazon Web Services connection type enables the :ref:`AWS Integrations
-<AWS>`.
+The Amazon Web Services connection type enables the :ref:`AWS Integrations <AWS>`.
+
+.. important:: Amazon Web Services Connection could be tested in the UI/API or by call
+ :meth:`~airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.test_connection`,
+ it is **important** to correct interpret result of this test.
Review Comment:
```suggestion
it is **important** to correctly interpret the result of this test.
```
##########
docs/apache-airflow-providers-amazon/connections/aws.rst:
##########
@@ -32,19 +45,26 @@ Alternatively, one can pass credentials in as a Connection initialisation parame
To use IAM instance profile, create an "empty" connection (i.e. one with no AWS Access Key ID or AWS Secret Access Key
specified, or ``aws://``).
-
Default Connection IDs
-----------------------
The default connection ID is ``aws_default``. If the environment/machine where you are running Airflow has the
file credentials in ``/home/.aws/``, and the default connection has user and pass fields empty, it will take
automatically the credentials from there.
-.. note:: Previously, the ``aws_default`` connection had the "extras" field set to ``{"region_name": "us-east-1"}``
+.. important:: Previously, the ``aws_default`` connection had the "extras" field set to ``{"region_name": "us-east-1"}``
on install. This means that by default the ``aws_default`` connection used the ``us-east-1`` region.
This is no longer the case and the region needs to be set manually, either in the connection screens in Airflow,
or via the ``AWS_DEFAULT_REGION`` environment variable.
+.. caution:: If you do not set ``[database] load_default_connections`` to ``True``
+ most probably you do not have ``aws_default``. By historical reason Amazon Provider
Review Comment:
```suggestion
most probably you do not have ``aws_default``. For historical reasons, the Amazon Provider
```
##########
docs/apache-airflow-providers-amazon/connections/aws.rst:
##########
@@ -32,19 +45,26 @@ Alternatively, one can pass credentials in as a Connection initialisation parame
To use IAM instance profile, create an "empty" connection (i.e. one with no AWS Access Key ID or AWS Secret Access Key
specified, or ``aws://``).
-
Default Connection IDs
-----------------------
The default connection ID is ``aws_default``. If the environment/machine where you are running Airflow has the
file credentials in ``/home/.aws/``, and the default connection has user and pass fields empty, it will take
automatically the credentials from there.
-.. note:: Previously, the ``aws_default`` connection had the "extras" field set to ``{"region_name": "us-east-1"}``
+.. important:: Previously, the ``aws_default`` connection had the "extras" field set to ``{"region_name": "us-east-1"}``
on install. This means that by default the ``aws_default`` connection used the ``us-east-1`` region.
This is no longer the case and the region needs to be set manually, either in the connection screens in Airflow,
or via the ``AWS_DEFAULT_REGION`` environment variable.
+.. caution:: If you do not set ``[database] load_default_connections`` to ``True``
+ most probably you do not have ``aws_default``. By historical reason Amazon Provider
+ components (Hooks, Operators, Sensors, etc.) fallback to default boto3 credentials strategy
+ in case of missing Connection ID, this behaviour is deprecated and will be removed in a future releases.
Review Comment:
```suggestion
components (Hooks, Operators, Sensors, etc.) fallback to the default boto3 credentials strategy
in case of a missing Connection ID. This behaviour is deprecated and will be removed in a future releases.
```
##########
docs/apache-airflow-providers-amazon/connections/aws.rst:
##########
@@ -32,19 +45,26 @@ Alternatively, one can pass credentials in as a Connection initialisation parame
To use IAM instance profile, create an "empty" connection (i.e. one with no AWS Access Key ID or AWS Secret Access Key
specified, or ``aws://``).
-
Default Connection IDs
-----------------------
The default connection ID is ``aws_default``. If the environment/machine where you are running Airflow has the
file credentials in ``/home/.aws/``, and the default connection has user and pass fields empty, it will take
automatically the credentials from there.
-.. note:: Previously, the ``aws_default`` connection had the "extras" field set to ``{"region_name": "us-east-1"}``
+.. important:: Previously, the ``aws_default`` connection had the "extras" field set to ``{"region_name": "us-east-1"}``
on install. This means that by default the ``aws_default`` connection used the ``us-east-1`` region.
This is no longer the case and the region needs to be set manually, either in the connection screens in Airflow,
or via the ``AWS_DEFAULT_REGION`` environment variable.
+.. caution:: If you do not set ``[database] load_default_connections`` to ``True``
+ most probably you do not have ``aws_default``. By historical reason Amazon Provider
+ components (Hooks, Operators, Sensors, etc.) fallback to default boto3 credentials strategy
+ in case of missing Connection ID, this behaviour is deprecated and will be removed in a future releases.
+
+ If you need use default boto3 credential strategy (credentials in environment variables, IAM Profile, etc.)
+ please provide ``None`` instead of connection id.
Review Comment:
```suggestion
If you need to use the default boto3 credential strategy (credentials in environment variables, IAM Profile, etc.)
please provide ``None`` instead of a connection ID.
```
##########
docs/apache-airflow-providers-amazon/connections/aws.rst:
##########
@@ -20,8 +20,21 @@
Amazon Web Services Connection
==============================
-The Amazon Web Services connection type enables the :ref:`AWS Integrations
-<AWS>`.
+The Amazon Web Services connection type enables the :ref:`AWS Integrations <AWS>`.
+
+.. important:: Amazon Web Services Connection could be tested in the UI/API or by call
+ :meth:`~airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.test_connection`,
+ it is **important** to correct interpret result of this test.
+ During this test components of Amazon Provider invoke AWS Security Token Service API
+ `GetCallerIdentity <https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html>`__.
+ This service **only** could check is your credentials valid or not.
+ Unfortunately it is not possible to validate is this credentials has access to specific AWS service or not.
+
+ If you use Amazon Provider for communicate with AWS API compatible services (MinIO, LocalStack, etc.)
Review Comment:
```suggestion
If you use the Amazon Provider to communicate with AWS API compatible services (MinIO, LocalStack, etc.)
```
##########
docs/apache-airflow-providers-amazon/connections/aws.rst:
##########
@@ -20,8 +20,21 @@
Amazon Web Services Connection
==============================
-The Amazon Web Services connection type enables the :ref:`AWS Integrations
-<AWS>`.
+The Amazon Web Services connection type enables the :ref:`AWS Integrations <AWS>`.
+
+.. important:: Amazon Web Services Connection could be tested in the UI/API or by call
+ :meth:`~airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.test_connection`,
+ it is **important** to correct interpret result of this test.
+ During this test components of Amazon Provider invoke AWS Security Token Service API
+ `GetCallerIdentity <https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html>`__.
+ This service **only** could check is your credentials valid or not.
+ Unfortunately it is not possible to validate is this credentials has access to specific AWS service or not.
+
+ If you use Amazon Provider for communicate with AWS API compatible services (MinIO, LocalStack, etc.)
+ test connection failure **doesn't mean** that your connection has wrong credentials.
+ Many of compatible services provide only limited amount of AWS API services,
+ and most of them not implement AWS STS GetCallerIdentity method.
Review Comment:
```suggestion
Many compatible services provide only a limited number of AWS API services,
and most of them do not implement the AWS STS GetCallerIdentity method.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org