You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/07/07 12:20:03 UTC
[GitHub] [airflow] gitstart-airflow opened a new pull request, #24898: Refactor and fix AWS secret manager invalid exception
gitstart-airflow opened a new pull request, #24898:
URL: https://github.com/apache/airflow/pull/24898
closes: [#24773](https://github.com/apache/airflow/issues/24773)
related: [#24773](https://github.com/apache/airflow/issues/24773)
---
This PR removes the invalid AccessDeniedException in get_secret_value and handle more exceptions.
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragement file, named `{pr_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #24898: Refactor and fix AWS secret manager invalid exception
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #24898:
URL: https://github.com/apache/airflow/pull/24898#issuecomment-1177860964
(and fix the static check along the way. I recommend pre-commit, it will do it for you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #24898: Refactor and fix AWS secret manager invalid exception
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #24898:
URL: https://github.com/apache/airflow/pull/24898#discussion_r916397272
##########
airflow/providers/amazon/aws/secrets/secrets_manager.py:
##########
@@ -257,15 +258,32 @@ def _get_secret(self, path_prefix, secret_id: str) -> Optional[str]:
return response.get('SecretString')
except self.client.exceptions.ResourceNotFoundException:
self.log.debug(
- "An error occurred (ResourceNotFoundException) when calling the "
- "get_secret_value operation: "
+ f"ResourceNotFoundException: {error_msg}: "
"Secret %s not found.",
secret_id,
)
return None
- except self.client.exceptions.AccessDeniedException:
+ except self.client.exceptions.InvalidParameterException:
self.log.debug(
- "An error occurred (AccessDeniedException) when calling the get_secret_value operation",
+ f"InvalidParameterException: {error_msg}",
+ exc_info=True,
+ )
+ return None
+ except self.client.exceptions.InvalidRequestException:
+ self.log.debug(
+ f"InvalidRequestException: {error_msg}",
Review Comment:
@josh-fell Finally I got to automate this check. I really do not like wasting mental capacity for things that can be automated during review :) https://github.com/apache/airflow/pull/24910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #24898: Refactor and fix AWS secret manager invalid exception
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #24898:
URL: https://github.com/apache/airflow/pull/24898#issuecomment-1177858917
OK. you should rebase now @gitstart-airflow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #24898: Refactor and fix AWS secret manager invalid exception
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #24898:
URL: https://github.com/apache/airflow/pull/24898#issuecomment-1177789574
You will have to rebase (static checks are failing) but also wait until https://github.com/apache/airflow/pull/24901 is merged (it should solve the test failures - my mistake introduced yesterday).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a diff in pull request #24898: Refactor and fix AWS secret manager invalid exception
Posted by GitBox <gi...@apache.org>.
josh-fell commented on code in PR #24898:
URL: https://github.com/apache/airflow/pull/24898#discussion_r916049648
##########
airflow/providers/amazon/aws/secrets/secrets_manager.py:
##########
@@ -257,15 +258,32 @@ def _get_secret(self, path_prefix, secret_id: str) -> Optional[str]:
return response.get('SecretString')
except self.client.exceptions.ResourceNotFoundException:
self.log.debug(
- "An error occurred (ResourceNotFoundException) when calling the "
- "get_secret_value operation: "
+ f"ResourceNotFoundException: {error_msg}: "
"Secret %s not found.",
secret_id,
)
return None
- except self.client.exceptions.AccessDeniedException:
+ except self.client.exceptions.InvalidParameterException:
self.log.debug(
- "An error occurred (AccessDeniedException) when calling the get_secret_value operation",
+ f"InvalidParameterException: {error_msg}",
+ exc_info=True,
+ )
+ return None
+ except self.client.exceptions.InvalidRequestException:
+ self.log.debug(
+ f"InvalidRequestException: {error_msg}",
Review Comment:
Can you change the the logging statements to use %-formatting instead of f-strings please? This makes sure that the string formatting is deferred until it cannot be avoided. [This is a helpful reference](https://blog.pilosus.org/posts/2020/01/24/python-f-strings-in-logging/) which hopefully explains the practice here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #24898: Refactor and fix AWS secret manager invalid exception
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #24898:
URL: https://github.com/apache/airflow/pull/24898#discussion_r916353226
##########
airflow/providers/amazon/aws/secrets/secrets_manager.py:
##########
@@ -257,15 +258,32 @@ def _get_secret(self, path_prefix, secret_id: str) -> Optional[str]:
return response.get('SecretString')
except self.client.exceptions.ResourceNotFoundException:
self.log.debug(
- "An error occurred (ResourceNotFoundException) when calling the "
- "get_secret_value operation: "
+ f"ResourceNotFoundException: {error_msg}: "
"Secret %s not found.",
secret_id,
)
return None
- except self.client.exceptions.AccessDeniedException:
+ except self.client.exceptions.InvalidParameterException:
self.log.debug(
- "An error occurred (AccessDeniedException) when calling the get_secret_value operation",
+ f"InvalidParameterException: {error_msg}",
+ exc_info=True,
+ )
+ return None
+ except self.client.exceptions.InvalidRequestException:
+ self.log.debug(
+ f"InvalidRequestException: {error_msg}",
Review Comment:
Right!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #24898: Refactor and fix AWS secret manager invalid exception
Posted by GitBox <gi...@apache.org>.
potiuk merged PR #24898:
URL: https://github.com/apache/airflow/pull/24898
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org