[GitHub] [airflow] mik-laj commented on a change in pull request #16571: Implemented Basic EKS Integration

[GitBox <gi...@apache.org>]

mik-laj commented on a change in pull request #16571:
URL: https://github.com/apache/airflow/pull/16571#discussion_r683872000



##########
File path: airflow/providers/amazon/aws/hooks/eks.py
##########
@@ -0,0 +1,420 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Interact with Amazon EKS, using the boto3 library."""
+import base64
+import json
+import re
+import tempfile
+from contextlib import contextmanager
+from functools import partial
+from typing import Callable, Dict, List, Optional
+
+import yaml
+from botocore.signers import RequestSigner
+
+from airflow.providers.amazon.aws.hooks.base_aws import AwsBaseHook
+from airflow.utils.json import AirflowJsonEncoder
+
+DEFAULT_CONTEXT_NAME = 'aws'
+DEFAULT_PAGINATION_TOKEN = ''
+DEFAULT_POD_USERNAME = 'aws'
+STS_TOKEN_EXPIRES_IN = 60
+
+
+class EKSHook(AwsBaseHook):
+    """
+    Interact with Amazon EKS, using the boto3 library.
+
+    Additional arguments (such as ``aws_conn_id``) may be specified and
+    are passed down to the underlying AwsBaseHook.
+
+    .. seealso::
+        :class:`~airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook`
+    """
+
+    client_type = 'eks'
+
+    def __init__(self, *args, **kwargs) -> None:
+        kwargs["client_type"] = self.client_type
+        super().__init__(*args, **kwargs)
+
+    def create_cluster(self, name: str, roleArn: str, resourcesVpcConfig: Dict, **kwargs) -> Dict:
+        """
+        Creates an Amazon EKS control plane.
+
+        :param name: The unique name to give to your Amazon EKS Cluster.
+        :type name: str
+        :param roleArn: The Amazon Resource Name (ARN) of the IAM role that provides permissions
+          for the Kubernetes control plane to make calls to AWS API operations on your behalf.
+        :type roleArn: str
+        :param resourcesVpcConfig: The VPC configuration used by the cluster control plane.
+        :type resourcesVpcConfig: Dict
+
+        :return: Returns descriptive information about the created EKS Cluster.
+        :rtype: Dict
+        """
+        eks_client = self.conn
+
+        response = eks_client.create_cluster(
+            name=name, roleArn=roleArn, resourcesVpcConfig=resourcesVpcConfig, **kwargs
+        )
+
+        self.log.info("Created cluster with the name %s.", response.get('cluster').get('name'))
+        return response
+
+    def create_nodegroup(
+        self, clusterName: str, nodegroupName: str, subnets: List[str], nodeRole: str, **kwargs
+    ) -> Dict:
+        """
+        Creates an Amazon EKS Managed Nodegroup for an EKS Cluster.
+
+        :param clusterName: The name of the cluster to create the EKS Managed Nodegroup in.
+        :type clusterName: str
+        :param nodegroupName: The unique name to give your managed nodegroup.
+        :type nodegroupName: str
+        :param subnets: The subnets to use for the Auto Scaling group that is created for your nodegroup.
+        :type subnets: List[str]
+        :param nodeRole: The Amazon Resource Name (ARN) of the IAM role to associate with your nodegroup.
+        :type nodeRole: str
+
+        :return: Returns descriptive information about the created EKS Managed Nodegroup.
+        :rtype: Dict
+        """
+        eks_client = self.conn
+        # The below tag is mandatory and must have a value of either 'owned' or 'shared'
+        # A value of 'owned' denotes that the subnets are exclusive to the nodegroup.
+        # The 'shared' value allows more than one resource to use the subnet.
+        tags = {'kubernetes.io/cluster/' + clusterName: 'owned'}
+        if "tags" in kwargs:
+            tags = {**tags, **kwargs["tags"]}
+            kwargs.pop("tags")
+
+        response = eks_client.create_nodegroup(
+            clusterName=clusterName,
+            nodegroupName=nodegroupName,
+            subnets=subnets,
+            nodeRole=nodeRole,
+            tags=tags,
+            **kwargs,
+        )
+
+        self.log.info(
+            "Created a managed nodegroup named %s in cluster %s",
+            response.get('nodegroup').get('nodegroupName'),
+            response.get('nodegroup').get('clusterName'),
+        )
+        return response
+
+    def delete_cluster(self, name: str) -> Dict:
+        """
+        Deletes the Amazon EKS Cluster control plane.
+
+        :param name: The name of the cluster to delete.
+        :type name: str
+
+        :return: Returns descriptive information about the deleted EKS Cluster.
+        :rtype: Dict
+        """
+        eks_client = self.conn
+
+        response = eks_client.delete_cluster(name=name)
+
+        self.log.info("Deleted cluster with the name %s.", response.get('cluster').get('name'))
+        return response
+
+    def delete_nodegroup(self, clusterName: str, nodegroupName: str) -> Dict:
+        """
+        Deletes an Amazon EKS Nodegroup from a specified cluster.
+
+        :param clusterName: The name of the Amazon EKS Cluster that is associated with your nodegroup.
+        :type clusterName: str
+        :param nodegroupName: The name of the nodegroup to delete.
+        :type nodegroupName: str
+
+        :return: Returns descriptive information about the deleted EKS Managed Nodegroup.
+        :rtype: Dict
+        """
+        eks_client = self.conn
+
+        response = eks_client.delete_nodegroup(clusterName=clusterName, nodegroupName=nodegroupName)
+
+        self.log.info(
+            "Deleted nodegroup named %s from cluster %s.",
+            response.get('nodegroup').get('nodegroupName'),
+            response.get('nodegroup').get('clusterName'),
+        )
+        return response
+
+    def describe_cluster(self, name: str, verbose: Optional[bool] = False) -> Dict:

Review comment:
       > You can use the Optional type modifier to define a type variant that allows None, such as Optional[int] (Optional[X] is the preferred shorthand for Union[X, None]):
   
   https://mypy.readthedocs.io/en/latest/kinds_of_types.html#optional-types-and-the-none-type
   
   I agree. We should use `bool` type here.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org