You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/12/16 20:42:03 UTC
[GitHub] [airflow] subkanthi opened a new pull request #20353: Fix mypy in providers/aws/hooks
subkanthi opened a new pull request #20353:
URL: https://github.com/apache/airflow/pull/20353
Fix mypy errors in providers/aws/hooks
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r779572254
##########
File path: airflow/providers/amazon/aws/hooks/athena.py
##########
@@ -228,7 +228,7 @@ def poll_query_status(self, query_execution_id: str, max_tries: Optional[int] =
sleep(self.sleep_time)
return final_query_state
- def get_output_location(self, query_execution_id: str) -> str:
+ def get_output_location(self, query_execution_id: str) -> Optional[str]:
Review comment:
The reason its Optional is because it can return None as output_location can be None
` output_location = None
`
Fixes this mypy error
```
airflow/providers/amazon/aws/hooks/athena.py:253: error: Incompatible return value type (got "Optional[Any]", expected "str")
return output_location
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#issuecomment-1008427702
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r779572254
##########
File path: airflow/providers/amazon/aws/hooks/athena.py
##########
@@ -228,7 +228,7 @@ def poll_query_status(self, query_execution_id: str, max_tries: Optional[int] =
sleep(self.sleep_time)
return final_query_state
- def get_output_location(self, query_execution_id: str) -> str:
+ def get_output_location(self, query_execution_id: str) -> Optional[str]:
Review comment:
The reason its Optional is because it can return None as output_location can be None
` output_location = None
`
Fixes this mypy error
```airflow/providers/amazon/aws/hooks/athena.py:253: error: Incompatible return value type (got "Optional[Any]", expected "str")
return output_location```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #20353:
URL: https://github.com/apache/airflow/pull/20353
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r773559560
##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -366,6 +366,8 @@ def get_cluster_state(self, clusterName: str) -> ClusterStates:
except ClientError as ex:
if ex.response.get("Error").get("Code") == "ResourceNotFoundException":
return ClusterStates.NONEXISTENT
+ else:
+ raise
Review comment:
```suggestion
raise
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi closed pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi closed pull request #20353:
URL: https://github.com/apache/airflow/pull/20353
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r779605574
##########
File path: airflow/providers/amazon/aws/hooks/athena.py
##########
@@ -228,7 +228,7 @@ def poll_query_status(self, query_execution_id: str, max_tries: Optional[int] =
sleep(self.sleep_time)
return final_query_state
- def get_output_location(self, query_execution_id: str) -> str:
+ def get_output_location(self, query_execution_id: str) -> Optional[str]:
Review comment:
Correct, We just have to cover the `else` for the `if response` to fix the mypy error.
```
airflow/providers/amazon/aws/hooks/athena.py:253: error: Incompatible return value type (got "Optional[Any]", expected "str")
return output_location
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi closed pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi closed pull request #20353:
URL: https://github.com/apache/airflow/pull/20353
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#issuecomment-1000144823
@ashb ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r770948413
##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -349,7 +349,7 @@ def describe_fargate_profile(
)
return response
- def get_cluster_state(self, clusterName: str) -> ClusterStates:
+ def get_cluster_state(self, clusterName: str) -> Optional[ClusterStates]:
Review comment:
This function shouldn't be optional -- it always returns something.
##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -360,14 +360,19 @@ def get_cluster_state(self, clusterName: str) -> ClusterStates:
:rtype: ClusterStates
"""
eks_client = self.conn
+ result = None
try:
- return ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
+ result = ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
Review comment:
This seems like a change for change's sake -- the previous version should have been okay, so lets stick with that please.
##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -360,14 +360,19 @@ def get_cluster_state(self, clusterName: str) -> ClusterStates:
:rtype: ClusterStates
"""
eks_client = self.conn
+ result = None
try:
- return ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
+ result = ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
except ClientError as ex:
if ex.response.get("Error").get("Code") == "ResourceNotFoundException":
- return ClusterStates.NONEXISTENT
+ result = ClusterStates.NONEXISTENT
- def get_fargate_profile_state(self, clusterName: str, fargateProfileName: str) -> FargateProfileStates:
+ return result
+
+ def get_fargate_profile_state(
+ self, clusterName: str, fargateProfileName: str
+ ) -> Optional[FargateProfileStates]:
Review comment:
Again, not optional -- always has a return.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r779609704
##########
File path: airflow/providers/amazon/aws/hooks/athena.py
##########
@@ -228,7 +228,7 @@ def poll_query_status(self, query_execution_id: str, max_tries: Optional[int] =
sleep(self.sleep_time)
return final_query_state
- def get_output_location(self, query_execution_id: str) -> str:
+ def get_output_location(self, query_execution_id: str) -> Optional[str]:
Review comment:
You can raise an error in the else part
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r779572254
##########
File path: airflow/providers/amazon/aws/hooks/athena.py
##########
@@ -228,7 +228,7 @@ def poll_query_status(self, query_execution_id: str, max_tries: Optional[int] =
sleep(self.sleep_time)
return final_query_state
- def get_output_location(self, query_execution_id: str) -> str:
+ def get_output_location(self, query_execution_id: str) -> Optional[str]:
Review comment:
The reason its Optional is because it can return None as output_location can be None
` output_location = None
`
Fixes this mypy error
```airflow/providers/amazon/aws/hooks/athena.py:253: error: Incompatible return value type (got "Optional[Any]", expected "str")
return output_location```
##########
File path: airflow/providers/amazon/aws/hooks/athena.py
##########
@@ -228,7 +228,7 @@ def poll_query_status(self, query_execution_id: str, max_tries: Optional[int] =
sleep(self.sleep_time)
return final_query_state
- def get_output_location(self, query_execution_id: str) -> str:
+ def get_output_location(self, query_execution_id: str) -> Optional[str]:
Review comment:
The reason its Optional is because it can return None as output_location can be None
` output_location = None
`
Fixes this mypy error
```airflow/providers/amazon/aws/hooks/athena.py:253: error: Incompatible return value type (got "Optional[Any]", expected "str")
return output_location
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r779605574
##########
File path: airflow/providers/amazon/aws/hooks/athena.py
##########
@@ -228,7 +228,7 @@ def poll_query_status(self, query_execution_id: str, max_tries: Optional[int] =
sleep(self.sleep_time)
return final_query_state
- def get_output_location(self, query_execution_id: str) -> str:
+ def get_output_location(self, query_execution_id: str) -> Optional[str]:
Review comment:
Correct, We just have to cover the `else` for the `if response` to fix the mypy error.
```airflow/providers/amazon/aws/hooks/athena.py:253: error: Incompatible return value type (got "Optional[Any]", expected "str")
return output_location```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r771885329
##########
File path: airflow/providers/amazon/aws/hooks/base_aws.py
##########
@@ -129,30 +130,30 @@ def _create_session_with_assume_role(self, session_kwargs: Dict[str, Any]) -> bo
)
session = botocore.session.get_session()
session._credentials = credentials
+
if self.basic_session is None:
raise RuntimeError("The basic session should be created here!")
+
region_name = self.basic_session.region_name
session.set_config_variable("region", region_name)
+
return boto3.session.Session(botocore_session=session, **session_kwargs)
def _refresh_credentials(self) -> Dict[str, Any]:
self.log.info('Refreshing credentials')
assume_role_method = self.extra_config.get('assume_role_method', 'assume_role')
sts_session = self.basic_session
+
+ if sts_session is None:
Review comment:
Not sure I understand - basic_session is a "**create+fash** on first use" property?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#issuecomment-998943387
Resolved conflicts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r774244569
##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -366,6 +366,8 @@ def get_cluster_state(self, clusterName: str) -> ClusterStates:
except ClientError as ex:
if ex.response.get("Error").get("Code") == "ResourceNotFoundException":
return ClusterStates.NONEXISTENT
+ else:
+ raise
Review comment:
Changed, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r770952251
##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -360,14 +360,19 @@ def get_cluster_state(self, clusterName: str) -> ClusterStates:
:rtype: ClusterStates
"""
eks_client = self.conn
+ result = None
try:
- return ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
+ result = ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
Review comment:
Dont really want to change anything π , these are the errors and in the except there is no return for the if /else clause. Maybe Im missing something
^
airflow/providers/amazon/aws/hooks/eks.py:352: error: Missing return statement
def get_cluster_state(self, clusterName: str) -> ClusterStates:
^
airflow/providers/amazon/aws/hooks/eks.py:370: error: Missing return statement
def get_fargate_profile_state(self, clusterName: str, fargateProfileName: str) -> FargateProfileStates:
^
airflow/providers/amazon/aws/hooks/eks.py:396: error: Missing return statement
def get_nodegroup_state(self, clusterName: str, nodegroupName: str) -> NodegroupStates:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r770952251
##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -360,14 +360,19 @@ def get_cluster_state(self, clusterName: str) -> ClusterStates:
:rtype: ClusterStates
"""
eks_client = self.conn
+ result = None
try:
- return ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
+ result = ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
Review comment:
Dont really want to change anything π , these are the errors and in the except there is no return for the if clause.
^
airflow/providers/amazon/aws/hooks/eks.py:352: error: Missing return statement
def get_cluster_state(self, clusterName: str) -> ClusterStates:
^
airflow/providers/amazon/aws/hooks/eks.py:370: error: Missing return statement
def get_fargate_profile_state(self, clusterName: str, fargateProfileName: str) -> FargateProfileStates:
^
airflow/providers/amazon/aws/hooks/eks.py:396: error: Missing return statement
def get_nodegroup_state(self, clusterName: str, nodegroupName: str) -> NodegroupStates:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r770967315
##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -360,14 +360,19 @@ def get_cluster_state(self, clusterName: str) -> ClusterStates:
:rtype: ClusterStates
"""
eks_client = self.conn
+ result = None
try:
- return ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
+ result = ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
except ClientError as ex:
if ex.response.get("Error").get("Code") == "ResourceNotFoundException":
- return ClusterStates.NONEXISTENT
+ result = ClusterStates.NONEXISTENT
Review comment:
Oh, we are missing an `else: raise` -- that was the case that didn't have a return.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r779669349
##########
File path: airflow/providers/amazon/aws/hooks/athena.py
##########
@@ -228,7 +228,7 @@ def poll_query_status(self, query_execution_id: str, max_tries: Optional[int] =
sleep(self.sleep_time)
return final_query_state
- def get_output_location(self, query_execution_id: str) -> str:
+ def get_output_location(self, query_execution_id: str) -> Optional[str]:
Review comment:
Thanks, changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r770952251
##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -360,14 +360,19 @@ def get_cluster_state(self, clusterName: str) -> ClusterStates:
:rtype: ClusterStates
"""
eks_client = self.conn
+ result = None
try:
- return ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
+ result = ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
Review comment:
Dont really want to change anything thats not related to mypy errorsπ , these are the errors and in the except there is no return for the if /else clause. Maybe Im missing something
^
airflow/providers/amazon/aws/hooks/eks.py:352: error: Missing return statement
def get_cluster_state(self, clusterName: str) -> ClusterStates:
^
airflow/providers/amazon/aws/hooks/eks.py:370: error: Missing return statement
def get_fargate_profile_state(self, clusterName: str, fargateProfileName: str) -> FargateProfileStates:
^
airflow/providers/amazon/aws/hooks/eks.py:396: error: Missing return statement
def get_nodegroup_state(self, clusterName: str, nodegroupName: str) -> NodegroupStates:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r770947943
##########
File path: airflow/providers/amazon/aws/hooks/base_aws.py
##########
@@ -129,30 +130,30 @@ def _create_session_with_assume_role(self, session_kwargs: Dict[str, Any]) -> bo
)
session = botocore.session.get_session()
session._credentials = credentials
+
if self.basic_session is None:
raise RuntimeError("The basic session should be created here!")
+
region_name = self.basic_session.region_name
session.set_config_variable("region", region_name)
+
return boto3.session.Session(botocore_session=session, **session_kwargs)
def _refresh_credentials(self) -> Dict[str, Any]:
self.log.info('Refreshing credentials')
assume_role_method = self.extra_config.get('assume_role_method', 'assume_role')
sts_session = self.basic_session
+
+ if sts_session is None:
Review comment:
We can create a basic_session with no extra arguments.
How about changing it so that basic_session is a "create+fash on first use" property?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r770952251
##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -360,14 +360,19 @@ def get_cluster_state(self, clusterName: str) -> ClusterStates:
:rtype: ClusterStates
"""
eks_client = self.conn
+ result = None
try:
- return ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
+ result = ClusterStates(eks_client.describe_cluster(name=clusterName).get('cluster').get('status'))
Review comment:
Dont really want to change anything π , these are the errors and in the except there is no return for the if clause. Maybe Im missing something
^
airflow/providers/amazon/aws/hooks/eks.py:352: error: Missing return statement
def get_cluster_state(self, clusterName: str) -> ClusterStates:
^
airflow/providers/amazon/aws/hooks/eks.py:370: error: Missing return statement
def get_fargate_profile_state(self, clusterName: str, fargateProfileName: str) -> FargateProfileStates:
^
airflow/providers/amazon/aws/hooks/eks.py:396: error: Missing return statement
def get_nodegroup_state(self, clusterName: str, nodegroupName: str) -> NodegroupStates:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r779601819
##########
File path: airflow/providers/amazon/aws/hooks/athena.py
##########
@@ -228,7 +228,7 @@ def poll_query_status(self, query_execution_id: str, max_tries: Optional[int] =
sleep(self.sleep_time)
return final_query_state
- def get_output_location(self, query_execution_id: str) -> str:
+ def get_output_location(self, query_execution_id: str) -> Optional[str]:
Review comment:
In the following block, the only way the `output_location` can be `None` is if `response` is a false-y value which does not look like it is possible from https://boto3.amazonaws.com/v1/documentation/api/1.9.42/reference/services/athena.html#Athena.Client.get_query_execution and https://docs.aws.amazon.com/athena/latest/APIReference/API_GetQueryExecution.html
https://github.com/apache/airflow/blob/d9259a556e9ae4c60edad9d1374ca0c7a9d2fa58/airflow/providers/amazon/aws/hooks/athena.py#L240-L251
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #20353: Fix mypy in providers/aws/hooks
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #20353:
URL: https://github.com/apache/airflow/pull/20353#discussion_r779509625
##########
File path: airflow/providers/amazon/aws/hooks/athena.py
##########
@@ -228,7 +228,7 @@ def poll_query_status(self, query_execution_id: str, max_tries: Optional[int] =
sleep(self.sleep_time)
return final_query_state
- def get_output_location(self, query_execution_id: str) -> str:
+ def get_output_location(self, query_execution_id: str) -> Optional[str]:
Review comment:
This should not be optional, if it does not return, it will raise. So `str` is correct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org