[GitHub] mik-laj commented on a change in pull request #4667: [AIRFLOW-3826] Group SageMaker sensors and operators into 2 files

[GitBox <gi...@apache.org>]

mik-laj commented on a change in pull request #4667: [AIRFLOW-3826] Group SageMaker sensors and operators into 2 files
URL: https://github.com/apache/airflow/pull/4667#discussion_r254886590
 
 

 ##########
 File path: airflow/contrib/operators/sagemaker_base_operator.py
 ##########
 @@ -17,84 +17,11 @@
 # specific language governing permissions and limitations
 # under the License.
 
-import json
+import warnings
 
-from airflow.contrib.hooks.sagemaker_hook import SageMakerHook
-from airflow.models import BaseOperator
-from airflow.utils.decorators import apply_defaults
+from airflow.contrib.operators.sagamaker_operator import SageMakerBaseOperator # noqa
 
-
-class SageMakerBaseOperator(BaseOperator):
-    """
-    This is the base operator for all SageMaker operators.
-
-    :param config: The configuration necessary to start a training job (templated)
-    :type config: dict
-    :param aws_conn_id: The AWS connection ID to use.
-    :type aws_conn_id: str
-    """
-
-    template_fields = ['config']
-    template_ext = ()
-    ui_color = '#ededed'
-
-    integer_fields = []
-
-    @apply_defaults
-    def __init__(self,
-                 config,
-                 aws_conn_id='aws_default',
-                 *args, **kwargs):
-        super(SageMakerBaseOperator, self).__init__(*args, **kwargs)
-
-        self.aws_conn_id = aws_conn_id
-        self.config = config
-        self.hook = None
-
-    def parse_integer(self, config, field):
-        if len(field) == 1:
-            if isinstance(config, list):
-                for sub_config in config:
-                    self.parse_integer(sub_config, field)
-                return
-            head = field[0]
-            if head in config:
-                config[head] = int(config[head])
-            return
-
-        if isinstance(config, list):
-            for sub_config in config:
-                self.parse_integer(sub_config, field)
-            return
-
-        head, tail = field[0], field[1:]
-        if head in config:
-            self.parse_integer(config[head], tail)
-        return
-
-    def parse_config_integers(self):
-        # Parse the integer fields of training config to integers
-        # in case the config is rendered by Jinja and all fields are str
-        for field in self.integer_fields:
-            self.parse_integer(self.config, field)
-
-    def expand_role(self):
-        pass
-
-    def preprocess_config(self):
-        self.log.info(
-            'Preprocessing the config and doing required s3_operations'
-        )
-        self.hook = SageMakerHook(aws_conn_id=self.aws_conn_id)
-
-        self.hook.configure_s3_resources(self.config)
-        self.parse_config_integers()
-        self.expand_role()
-
-        self.log.info(
-            'After preprocessing the config is:\n {}'.format(
-                json.dumps(self.config, sort_keys=True, indent=4, separators=(',', ': ')))
-        )
-
-    def execute(self, context):
-        raise NotImplementedError('Please implement execute() in sub class!')
+warnings.warn("The operator `SageMakerBaseOperator` has been moved to another "
+              "module. You should write "
+              "'from airflow.contrib.operators.segamaker_operator import SageMakerBaseOperator`"
+              " to load this operator.", DeprecationWarning)
 
 Review comment:
   > The other thing to check out is `zope.deprecation` which is a module we already use in Airflow
   
   `warnings` is module from Python. `zone.deprecation` uses `warnings` internally.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services