You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "cjames23 (via GitHub)" <gi...@apache.org> on 2023/06/15 14:29:33 UTC
[GitHub] [airflow] cjames23 opened a new pull request, #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
cjames23 opened a new pull request, #31939:
URL: https://github.com/apache/airflow/pull/31939
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
Creating a Hook to call a Chime Web Hook. This is preliminary work to be able to enable Chime notifications similar to https://github.com/apache/airflow/pull/31273
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] cjames23 commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "cjames23 (via GitHub)" <gi...@apache.org>.
cjames23 commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1231336068
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
Review Comment:
I was not entirely sure, I think we might need to have 2 different classes for this? You cannot do webhook stuff in boto3 but chat bot stuff is available via boto3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1231440525
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ message: str = "",
+ *args: Any,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(*args, **kwargs)
+ self.message = message
+ self.webhook_endpoint = self._get_webhook_endpoint(http_conn_id, webhook_endpoint)
+
+ def _get_webhook_endpoint(self, http_conn_id: str | None, webhook_endpoint: str | None) -> str:
+ """
+ Given a Chime http_conn_id return the default webhook endpoint or override if
+ webhook_endpoint is manually provided.
+
+ :param http_conn_id: The provided connection ID
+ :param webhook_endpoint: The manually provided webhook endpoint
+ :return: Webhook Endpoint(str) to use with Chime
+ """
+ if webhook_endpoint:
+ endpoint = webhook_endpoint
+ elif http_conn_id:
+ conn = self.get_connection(http_conn_id)
+ extra = conn.extra_dejson
+ endpoint = extra.get("webhook_endpoint", "")
+ else:
+ raise AirflowException()
+
+ # Check to make sure the endpoint matches what Chime expects
+ if not re.match("^incomingwebhooks/[a-zA-Z0-9_-]+\?token=[a-zA-Z0-9_-]+$", endpoint):
+ raise AirflowException(
+ "Expected Chime webhook endpoint in the form of "
+ '"incomingwebhooks/{webhook.id}?token={webhook.token}".'
Review Comment:
this can expose the token to the log doesn't it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1232392626
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Chime webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Chime room.
Review Comment:
This is not a parameter of `__init__` method. Please move it to `send_message` method
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Chime webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Chime room.
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ *args: Any,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(*args, **kwargs)
+ self.webhook_endpoint = self._get_webhook_endpoint(http_conn_id, webhook_endpoint)
+
+ def _get_webhook_endpoint(self, http_conn_id: str | None, webhook_endpoint: str | None) -> str:
+ """
+ Given a Chime http_conn_id return the default webhook endpoint or override if
+ webhook_endpoint is manually provided.
+
+ :param http_conn_id: The provided connection ID
+ :param webhook_endpoint: The manually provided webhook endpoint
+ :return: Webhook Endpoint(str) to use with Chime
+ """
+ if webhook_endpoint:
+ endpoint = webhook_endpoint
+ elif http_conn_id:
+ conn = self.get_connection(http_conn_id)
+ extra = conn.extra_dejson
+ endpoint = extra.get("webhook_endpoint", "")
Review Comment:
In case the key does not exist, `endpoint` value would be empty string which would result in the exception raised on line 81 with message `Expected Chime webhook endpoint in the form of`. It might be better user experience to raise an exception mentioning `webhook_endpoint` is not defined in configuration in that case. That might be easier for users to understand the issue. WDYT?
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Chime webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Chime room.
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ *args: Any,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(*args, **kwargs)
+ self.webhook_endpoint = self._get_webhook_endpoint(http_conn_id, webhook_endpoint)
+
+ def _get_webhook_endpoint(self, http_conn_id: str | None, webhook_endpoint: str | None) -> str:
+ """
+ Given a Chime http_conn_id return the default webhook endpoint or override if
+ webhook_endpoint is manually provided.
+
+ :param http_conn_id: The provided connection ID
+ :param webhook_endpoint: The manually provided webhook endpoint
+ :return: Webhook Endpoint(str) to use with Chime
+ """
+ if webhook_endpoint:
+ endpoint = webhook_endpoint
+ elif http_conn_id:
+ conn = self.get_connection(http_conn_id)
+ extra = conn.extra_dejson
+ endpoint = extra.get("webhook_endpoint", "")
+ else:
+ raise AirflowException(
+ "Missing one of http_conn_id or webhook_endpoint arguments which are required."
+ )
+
+ # Check to make sure the endpoint matches what Chime expects
+ if not re.match("^incomingwebhooks/[a-zA-Z0-9_-]+\?token=[a-zA-Z0-9_-]+$", endpoint):
+ raise AirflowException(
+ "Expected Chime webhook endpoint in the form of "
+ '"incomingwebhooks/{webhook.id}?token={webhook.token}".'
+ )
+
+ return endpoint
+
+ def _build_chime_payload(self, message: str) -> str:
+ """Builds payload for Chime and ensures messages do not exceed max length allowed."""
+ payload: dict[str, Any] = {}
+ # We need to make sure that the message does not exceed the max length for Chime
+ if len(message) <= 4096:
+ payload["Content"] = message
+ else:
+ raise AirflowException("Chime message must be 4096 characters or less.")
+
+ return json.dumps(payload)
Review Comment:
Just a question of style but I feel this is simpler that way? Feel free to keep your version if you want.
```suggestion
# We need to make sure that the message does not exceed the max length for Chime
if len(message) > 4096:
raise AirflowException("Chime message must be 4096 characters or less.")
return json.dumps({"Content": message})
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #31939: Add `ChimeWebhookHook`
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #31939:
URL: https://github.com/apache/airflow/pull/31939#issuecomment-1608831805
Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1231460207
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
Review Comment:
> Or do you see the need for two distinct Chime hooks?
I think it should be 2 hooks. They don't really share the same logic and combining it to one hook may be very confusing to users as some functions (like chatbots) will work only with one of the options.
We have same case with Slack: `SlackHook`, `SlackWebhookHook`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1231438052
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ message: str = "",
+ *args: Any,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(*args, **kwargs)
+ self.message = message
+ self.webhook_endpoint = self._get_webhook_endpoint(http_conn_id, webhook_endpoint)
+
+ def _get_webhook_endpoint(self, http_conn_id: str | None, webhook_endpoint: str | None) -> str:
+ """
+ Given a Chime http_conn_id return the default webhook endpoint or override if
+ webhook_endpoint is manually provided.
+
+ :param http_conn_id: The provided connection ID
+ :param webhook_endpoint: The manually provided webhook endpoint
+ :return: Webhook Endpoint(str) to use with Chime
+ """
+ if webhook_endpoint:
+ endpoint = webhook_endpoint
+ elif http_conn_id:
+ conn = self.get_connection(http_conn_id)
+ extra = conn.extra_dejson
+ endpoint = extra.get("webhook_endpoint", "")
+ else:
+ raise AirflowException()
+
+ # Check to make sure the endpoint matches what Chime expects
+ if not re.match("^incomingwebhooks/[a-zA-Z0-9_-]+\?token=[a-zA-Z0-9_-]+$", endpoint):
+ raise AirflowException(
+ "Expected Chime webhook endpoint in the form of "
+ '"incomingwebhooks/{webhook.id}?token={webhook.token}".'
+ )
+
+ return endpoint
+
+ def _build_chime_payload(self) -> str:
+ """Builds payload for Chime and ensures messages do not exceed max length allowed."""
+ payload: dict[str, Any] = {}
+ # We need to make sure that the message does not exceed the max length for Chime
+ if len(self.message) <= 4096:
+ payload["Content"] = self.message
+ else:
+ raise AirflowException("Chime message must be 4096 characters or less.")
+
+ return json.dumps(payload)
Review Comment:
I assume that if we submit message longer than 4096 chars the API itself will return error and as a result Airflow will fail the task and show the error traceback.
Is there value in doing this check in advance?
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ message: str = "",
+ *args: Any,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(*args, **kwargs)
+ self.message = message
+ self.webhook_endpoint = self._get_webhook_endpoint(http_conn_id, webhook_endpoint)
+
+ def _get_webhook_endpoint(self, http_conn_id: str | None, webhook_endpoint: str | None) -> str:
+ """
+ Given a Chime http_conn_id return the default webhook endpoint or override if
+ webhook_endpoint is manually provided.
+
+ :param http_conn_id: The provided connection ID
+ :param webhook_endpoint: The manually provided webhook endpoint
+ :return: Webhook Endpoint(str) to use with Chime
+ """
+ if webhook_endpoint:
+ endpoint = webhook_endpoint
+ elif http_conn_id:
+ conn = self.get_connection(http_conn_id)
+ extra = conn.extra_dejson
+ endpoint = extra.get("webhook_endpoint", "")
+ else:
+ raise AirflowException()
+
+ # Check to make sure the endpoint matches what Chime expects
+ if not re.match("^incomingwebhooks/[a-zA-Z0-9_-]+\?token=[a-zA-Z0-9_-]+$", endpoint):
+ raise AirflowException(
+ "Expected Chime webhook endpoint in the form of "
+ '"incomingwebhooks/{webhook.id}?token={webhook.token}".'
Review Comment:
this can expose the token to the log doesn't it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #31939:
URL: https://github.com/apache/airflow/pull/31939#issuecomment-1593179007
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (ruff, mypy and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] cjames23 commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "cjames23 (via GitHub)" <gi...@apache.org>.
cjames23 commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1231336488
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ message: str = "",
+ *args: Any,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(*args, **kwargs)
+ self.message = message
+ self.webhook_endpoint = self._get_webhook_endpoint(http_conn_id, webhook_endpoint)
+
+ def _get_webhook_endpoint(self, http_conn_id: str | None, webhook_endpoint: str | None) -> str:
+ """
+ Given a Chime http_conn_id return the default webhook endpoint or override if
+ webhook_endpoint is manually provided.
+
+ :param http_conn_id: The provided connection ID
+ :param webhook_endpoint: The manually provided webhook endpoint
+ :return: Webhook Endpoint(str) to use with Chime
+ """
+ if webhook_endpoint:
+ endpoint = webhook_endpoint
+ elif http_conn_id:
+ conn = self.get_connection(http_conn_id)
+ extra = conn.extra_dejson
+ endpoint = extra.get("webhook_endpoint", "")
+ else:
+ raise AirflowException()
Review Comment:
Thanks, I must have missed adding in the error message about requiring one of the two args.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1232681082
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,113 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with web hooks and not chat bots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ ``{"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}``
+ :param webhook_endpoint: Chime webhook endpoint in the form of
+ ``"incomingwebhooks/{webhook.id}?token={webhook.token}"``
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ *args: Any,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(*args, **kwargs)
+ self.webhook_endpoint = self._get_webhook_endpoint(http_conn_id, webhook_endpoint)
+
+ def _get_webhook_endpoint(self, http_conn_id: str | None, webhook_endpoint: str | None) -> str:
+ """
+ Given a Chime http_conn_id return the default webhook endpoint or override if
+ webhook_endpoint is manually provided.
+
+ :param http_conn_id: The provided connection ID.
+ :param webhook_endpoint: The manually provided endpoint for the chime webhook.
+ :return: Endpoint(str) for chime webhook.
+ """
+ if webhook_endpoint:
+ endpoint = webhook_endpoint
+ elif http_conn_id:
+ conn = self.get_connection(http_conn_id)
+ extra = conn.extra_dejson
+ endpoint = extra.get("webhook_endpoint", "")
+ if endpoint == "":
+ raise AirflowException("webhook_endpoint missing from extras and is required.")
Review Comment:
nit. Setting a default value and then testing on that default value feels odd to me
```suggestion
if "webhook_endpoint" not in extra:
raise AirflowException("webhook_endpoint missing from extras and is required.")
endpoint = extra.get("webhook_endpoint")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #31939:
URL: https://github.com/apache/airflow/pull/31939#issuecomment-1607081022
Hey @cjames23 -> you need to update `breeze` selective tests. Apparently the change caused changes in dependencies between providers: https://github.com/apache/airflow/pull/31939/files#diff-e1c023651fed5dc86ec9d3303308052dd32afa65a80bfbc6d6c4554d9aae7c1d -> so expected output of seme selective check tests should be updated to match it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ferruzzi commented on pull request #31939: Add `ChimeWebhookHook`
Posted by "ferruzzi (via GitHub)" <gi...@apache.org>.
ferruzzi commented on PR #31939:
URL: https://github.com/apache/airflow/pull/31939#issuecomment-1608917697
Nice! Congrats.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1231304684
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
Review Comment:
I'd just call this Chime? In the future this hook could also be used for chat bots. Or do you see the need for two distinct Chime hooks?
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
Review Comment:
```suggestion
"""This module contains a Hook for interacting with Chime Web Hooks."""
```
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
Review Comment:
copy/pasta
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ message: str = "",
Review Comment:
Shouldn't the message be an argument to execute? Having one message per Hook construction feels odd.
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ message: str = "",
+ *args: Any,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(*args, **kwargs)
+ self.message = message
+ self.webhook_endpoint = self._get_webhook_endpoint(http_conn_id, webhook_endpoint)
+
+ def _get_webhook_endpoint(self, http_conn_id: str | None, webhook_endpoint: str | None) -> str:
+ """
+ Given a Chime http_conn_id return the default webhook endpoint or override if
+ webhook_endpoint is manually provided.
+
+ :param http_conn_id: The provided connection ID
+ :param webhook_endpoint: The manually provided webhook endpoint
+ :return: Webhook Endpoint(str) to use with Chime
+ """
+ if webhook_endpoint:
+ endpoint = webhook_endpoint
+ elif http_conn_id:
+ conn = self.get_connection(http_conn_id)
+ extra = conn.extra_dejson
+ endpoint = extra.get("webhook_endpoint", "")
+ else:
+ raise AirflowException()
+
+ # Check to make sure the endpoint matches what Chime expects
+ if not re.match("^incomingwebhooks/[a-zA-Z0-9_-]+\?token=[a-zA-Z0-9_-]+$", endpoint):
+ raise AirflowException(
+ "Expected Chime webhook endpoint in the form of "
+ '"incomingwebhooks/{webhook.id}?token={webhook.token}".'
+ )
+
+ return endpoint
+
+ def _build_chime_payload(self) -> str:
+ """Builds payload for Chime and ensures messages do not exceed max length allowed."""
+ payload: dict[str, Any] = {}
+ # We need to make sure that the message does not exceed the max length for Chime
+ if len(self.message) <= 4096:
+ payload["Content"] = self.message
+ else:
+ raise AirflowException("Chime message must be 4096 characters or less.")
+
+ return json.dumps(payload)
+
+ def execute(self) -> None:
Review Comment:
This kind of feels like an Operator execute method. Maybe this should be migrated to a new ChimeWebHookOperator?
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ message: str = "",
+ *args: Any,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(*args, **kwargs)
+ self.message = message
+ self.webhook_endpoint = self._get_webhook_endpoint(http_conn_id, webhook_endpoint)
+
+ def _get_webhook_endpoint(self, http_conn_id: str | None, webhook_endpoint: str | None) -> str:
+ """
+ Given a Chime http_conn_id return the default webhook endpoint or override if
+ webhook_endpoint is manually provided.
+
+ :param http_conn_id: The provided connection ID
+ :param webhook_endpoint: The manually provided webhook endpoint
+ :return: Webhook Endpoint(str) to use with Chime
+ """
+ if webhook_endpoint:
+ endpoint = webhook_endpoint
+ elif http_conn_id:
+ conn = self.get_connection(http_conn_id)
+ extra = conn.extra_dejson
+ endpoint = extra.get("webhook_endpoint", "")
+ else:
+ raise AirflowException()
Review Comment:
Add a useful error here to explain to the caller what was missing or needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] cjames23 commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "cjames23 (via GitHub)" <gi...@apache.org>.
cjames23 commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1231337850
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
Review Comment:
Yeah I caught that shortly after pushing the review. I have a fix coming this afternoon including adding the rst for the chime connection documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1231436675
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
+
+ def __init__(
+ self,
+ http_conn_id: str | None = None,
+ webhook_endpoint: str | None = None,
+ message: str = "",
+ *args: Any,
+ **kwargs: Any,
+ ) -> None:
+ super().__init__(*args, **kwargs)
+ self.message = message
+ self.webhook_endpoint = self._get_webhook_endpoint(http_conn_id, webhook_endpoint)
+
+ def _get_webhook_endpoint(self, http_conn_id: str | None, webhook_endpoint: str | None) -> str:
+ """
+ Given a Chime http_conn_id return the default webhook endpoint or override if
+ webhook_endpoint is manually provided.
+
+ :param http_conn_id: The provided connection ID
+ :param webhook_endpoint: The manually provided webhook endpoint
+ :return: Webhook Endpoint(str) to use with Chime
+ """
+ if webhook_endpoint:
+ endpoint = webhook_endpoint
+ elif http_conn_id:
+ conn = self.get_connection(http_conn_id)
+ extra = conn.extra_dejson
+ endpoint = extra.get("webhook_endpoint", "")
+ else:
+ raise AirflowException()
+
+ # Check to make sure the endpoint matches what Chime expects
+ if not re.match("^incomingwebhooks/[a-zA-Z0-9_-]+\?token=[a-zA-Z0-9_-]+$", endpoint):
+ raise AirflowException(
+ "Expected Chime webhook endpoint in the form of "
+ '"incomingwebhooks/{webhook.id}?token={webhook.token}".'
+ )
+
+ return endpoint
+
+ def _build_chime_payload(self) -> str:
+ """Builds payload for Chime and ensures messages do not exceed max length allowed."""
+ payload: dict[str, Any] = {}
+ # We need to make sure that the message does not exceed the max length for Chime
+ if len(self.message) <= 4096:
+ payload["Content"] = self.message
+ else:
+ raise AirflowException("Chime message must be 4096 characters or less.")
+
+ return json.dumps(payload)
+
+ def execute(self) -> None:
Review Comment:
agree with Niko better to find another name for this function as execute may cause confusion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a diff in pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #31939:
URL: https://github.com/apache/airflow/pull/31939#discussion_r1231443594
##########
airflow/providers/amazon/aws/hooks/chime.py:
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module contains a web hook for Chime."""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from airflow.exceptions import AirflowException
+from airflow.providers.http.hooks.http import HttpHook
+
+
+class ChimeWebhookHook(HttpHook):
+ """Interact with Chime Web Hooks to create notifications.
+
+ .. warning:: This hook is only designed to work with Web Hooks and not Chatbots.
+
+ :param http_conn_id: Http connection ID with host as "https://hooks.chime.aws" and
+ default webhook endpoint in the extra field in the form of
+ {"webhook_endpoint": "incomingwebhooks/{webhook.id}?token{webhook.token}"}
+ :param webhook_endpoint: Discord webhook endpoint in the form of
+ "incomingwebhooks/{webhook.id}?token={webhook.token}"
+ :param message: The message you want to send to your Discord channel
+ (max 4096 characters)
+ """
+
+ conn_name_attr = "http_conn_id"
+ default_conn_name = "chime_default"
+ conn_type = "chime"
+ hook_name = "Chime Web Hook"
Review Comment:
Looking at the other hooks it seems like they all have `Amazon _service_` format
for example:
https://github.com/apache/airflow/blob/cda83c226bf231f3a7e1b78c00bf13f9defb5d6e/airflow/providers/amazon/aws/hooks/redshift_sql.py#L59
https://github.com/apache/airflow/blob/cda83c226bf231f3a7e1b78c00bf13f9defb5d6e/airflow/providers/amazon/aws/hooks/emr.py#L51
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] cjames23 commented on pull request #31939: Create ChimeWebhook Hook to send messages to chime room from Airflow.
Posted by "cjames23 (via GitHub)" <gi...@apache.org>.
cjames23 commented on PR #31939:
URL: https://github.com/apache/airflow/pull/31939#issuecomment-1607951691
@potiuk is there a way to run selective checks locally with breeze? If I try to run it with pytest inside the breeze container I get an import error.
```
ImportError while importing test module '/opt/airflow/dev/breeze/tests/test_selective_checks.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.8/site-packages/_pytest/python.py:617: in _importtestmodule
mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
/usr/local/lib/python3.8/site-packages/_pytest/pathlib.py:565: in import_path
importlib.import_module(module_name)
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1014: in _gcd_import
???
<frozen importlib._bootstrap>:991: in _find_and_load
???
<frozen importlib._bootstrap>:975: in _find_and_load_unlocked
???
<frozen importlib._bootstrap>:671: in _load_unlocked
???
/usr/local/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:178: in exec_module
exec(co, module.__dict__)
test_selective_checks.py:24: in <module>
from airflow_breeze.global_constants import COMMITTERS, GithubEvents
E ModuleNotFoundError: No module named 'airflow_breeze'
```
I want to test if for http do I need to set run-amazon-tests to true rather than false based on the latest error I am seeing from the last attempted run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal merged pull request #31939: Add `ChimeWebhookHook`
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal merged PR #31939:
URL: https://github.com/apache/airflow/pull/31939
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org