You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/12/24 18:10:50 UTC
[GitHub] [airflow] Taragolis commented on a diff in pull request #28569: Add general-purpose "notifier" concept to DAGs
Taragolis commented on code in PR #28569:
URL: https://github.com/apache/airflow/pull/28569#discussion_r1056860177
##########
airflow/providers/slack/notifier.py:
##########
@@ -0,0 +1,80 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
Review Comment:
There is second ability for send message into slack by use SlackWebhook so maybe better move `airflow/providers/slack/notifier.py` -> `airflow/providers/slack/notifiers/slack.py`.
In this case we will have dumb relatable name in path but it granted consistency with hooks and operators
##########
airflow/notifications/notifier.py:
##########
@@ -0,0 +1,108 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+from abc import abstractmethod
+from typing import TYPE_CHECKING, Sequence
+
+import jinja2
+
+from airflow.template.templater import Templater
+from airflow.utils.context import Context, context_merge
+
+if TYPE_CHECKING:
+ from airflow import DAG
+
+
+class Notifier(Templater):
+ """
+ Base Notifier class for sending notifications
+
+ :param message: The message to send
+
+ The message or template_file can be used to send a notification. If both are set, the message
+ will be used.
+ """
+
+ template_fields: Sequence[str] = ("message",)
+ template_ext: Sequence[str] = (".txt",)
+
+ def __init__(
+ self,
+ message: str | None = "This is a default message",
+ ):
+ super().__init__()
+ self.message = message
+ self.resolve_template_files()
Review Comment:
Do we need to specify any default `template_fields` and `template_ext` as well as default argument `message`.
I think it could be define on specific implementation of Notifier.
##########
airflow/notifications/notifier.py:
##########
@@ -0,0 +1,108 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+from abc import abstractmethod
+from typing import TYPE_CHECKING, Sequence
+
+import jinja2
+
+from airflow.template.templater import Templater
+from airflow.utils.context import Context, context_merge
+
+if TYPE_CHECKING:
+ from airflow import DAG
+
+
+class Notifier(Templater):
+ """
+ Base Notifier class for sending notifications
+
+ :param message: The message to send
+
+ The message or template_file can be used to send a notification. If both are set, the message
+ will be used.
+ """
+
+ template_fields: Sequence[str] = ("message",)
+ template_ext: Sequence[str] = (".txt",)
+
+ def __init__(
+ self,
+ message: str | None = "This is a default message",
+ ):
+ super().__init__()
+ self.message = message
+ self.resolve_template_files()
+
+ def _update_context(self, context: Context) -> Context:
+ """
+ Add additional context to the context
+
+ :param context: The airflow context
+ """
+ additional_context = {}
+ for field in self.template_fields:
+ additional_context[field] = getattr(self, field)
+ context_merge(context, additional_context)
+ return context
+
+ def _render(self, template, context, dag: DAG | None = None):
+ dag = context["dag"] if dag is None else dag
+ return super()._render(template, context, dag)
+
+ def render_template_fields(
+ self,
+ context: Context,
+ jinja_env: jinja2.Environment | None = None,
+ ) -> None:
+ """Template all attributes listed in *self.template_fields*.
+
+ This mutates the attributes in-place and is irreversible.
+
+ :param context: Context dict with values to apply on content.
+ :param jinja_env: Jinja environment to use for rendering.
+ """
+ context = self._update_context(context)
+ dag = context["dag"]
+ if not jinja_env:
+ jinja_env = self.get_template_env(dag=dag)
+ self._do_render_template_fields(self, self.template_fields, context, jinja_env, set())
+
+ @abstractmethod
+ def notify(self, context: Context) -> None:
+ """
+ Sends a notification
+
+ subclasses should always call super().notify(context) first to ensure
+ that the template is rendered with the context.
+
+ :param context: The airflow context
+ """
+ self.render_template_fields(context)
+
+ def __call__(self, context: Context) -> None:
+ """
+ Send a notification
+
+ :param context: The airflow context
+ """
+ try:
+ self.notify(context)
Review Comment:
If it possible to move render_template_fields line into `__call__` method? If so then we don't need to call `super().notify(context)` if directly subclass of Notifier
##########
airflow/notifications/notifier.py:
##########
@@ -0,0 +1,108 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+from abc import abstractmethod
+from typing import TYPE_CHECKING, Sequence
+
+import jinja2
+
+from airflow.template.templater import Templater
+from airflow.utils.context import Context, context_merge
+
+if TYPE_CHECKING:
+ from airflow import DAG
+
+
+class Notifier(Templater):
Review Comment:
BaseNotifier?
##########
airflow/providers/slack/notifier.py:
##########
@@ -0,0 +1,80 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+import json
+
+from airflow.compat.functools import cached_property
+from airflow.notifications.notifier import Notifier
+from airflow.providers.slack.hooks.slack import SlackHook
+
+
+class SlackNotifier(Notifier):
+ """
+ Slack Notifier
+
+ :param slack_conn_id: Slack API token (https://api.slack.com/web). Optional
Review Comment:
`slack_conn_id` should be mandatory attribute.
In SlackHook and some operators this attribute optional by historical reason - user could provide token (secret value) directly to hook which a bit insecure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org