You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "Vitor-Avila (via GitHub)" <gi...@apache.org> on 2025/02/20 22:13:14 UTC
[PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Vitor-Avila opened a new pull request, #32336:
URL: https://github.com/apache/superset/pull/32336
### SUMMARY
This PR fixes a few things with the Slack method migration from V1 to V2 (required as Slack is fully deprecating `files.upload()` in March):
* `get_channels_with_search` was returning a list of dictionaries (one per channel found) and we were dumping this directly in the `target` configuration for a report. The `target` field expects a comma-separated list of channels IDs for the V2 approach.
* I've added a schema that drops any additional fields from each channels to reduce information (some organizations could have thousands of channels).
* Enforcing `exact_match` search during the migration to avoid for example migrating a report from `other-channel` to `another-channel`.
* Stripping channel names before the search to account for `one-channel, second-channel`.
Fixes https://github.com/apache/superset/issues/31936
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
No UI changes.
### TESTING INSTRUCTIONS
Added testing.
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [x] Has associated issue: https://github.com/apache/superset/issues/31936
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982313667
##########
superset/commands/report/execute.py:
##########
@@ -139,22 +139,37 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ channels_list = get_recipients_list(slack_recipients["target"])
+ channels_list = [channel.lstrip("#") for channel in channels_list]
+ channels = get_channels_with_search(
+ search_string=channels_list,
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(channels_list) != len(channels):
+ missing_channels = set(channels_list) - {
+ channel["name"] for channel in channels
+ }
+ msg = (
+ "Could not find the following channels: "
+ f"{', '.join(missing_channels)}"
+ )
+ raise UpdateFailedError(msg)
Review Comment:
Ok, this makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982287169
##########
superset/utils/core.py:
##########
@@ -811,7 +811,13 @@ def send_mime_email(
smtp.quit()
-def get_email_address_list(address_string: str) -> list[str]:
+def get_recipients_list(address_string: str | None) -> list[str]:
Review Comment:
Since this isn't really fetching anything, can we change it to sound more like a utility, like "recipient_string_to_list"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "korbit-ai[bot] (via GitHub)" <gi...@apache.org>.
korbit-ai[bot] commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1976102049
##########
superset/reports/api.py:
##########
@@ -574,7 +575,7 @@ def slack_channels(self, **kwargs: Any) -> Response:
"""
try:
params = kwargs.get("rison", {})
- search_string = params.get("search_string")
+ search_string = get_recipients_list(params.get("search_string"))
Review Comment:
### Invalid Search String Processing <sub></sub>
<details>
<summary>Tell me more</summary>
###### What is the issue?
The get_recipients_list function is being misused in the slack_channels endpoint. This function is designed to parse recipient lists but is being used to process a search string, which could lead to incorrect channel searching.
###### Why this matters
Using a recipient list parser for search string processing could split or modify the search term unexpectedly, potentially causing the Slack channel search to fail or return incorrect results.
###### Suggested change ∙ *Feature Preview*
Return to using the original search string directly:
```python
search_string = params.get("search_string")
```
</details>
<sub>
[](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/4ec87b51-24cb-48ac-b55d-1557de7e5112?suggestedFixEnabled=true)
💬 Chat with Korbit by mentioning @korbit-ai.
</sub>
<!--- korbi internal id:e2435be5-4116-4e0d-acf4-e2239b3a8496 -->
##########
superset/commands/report/execute.py:
##########
@@ -139,15 +139,21 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ channels_list = get_recipients_list(slack_recipients["target"])
+ channels = get_channels_with_search(
+ search_string=channels_list,
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(channels_list) != len(channels):
+ raise UpdateFailedError("Not all channels could be found")
Review Comment:
### Slack Channel Validation Fails with Duplicate Inputs <sub></sub>
<details>
<summary>Tell me more</summary>
###### What is the issue?
The channel validation in update_report_schedule_slack_v2 fails to handle duplicates in the input list, which could cause false negatives.
###### Why this matters
If the input channels_list contains duplicate channel names, the length comparison will incorrectly raise an error even if all channels are actually found.
###### Suggested change ∙ *Feature Preview*
Update the validation to compare unique channel names against found channels:
```python
if len(set(channels_list)) != len(channels):
raise UpdateFailedError("Not all channels could be found")
```
</details>
<sub>
[](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/250d1f8f-a9ec-4fd8-b7aa-1090ede64476?suggestedFixEnabled=true)
💬 Chat with Korbit by mentioning @korbit-ai.
</sub>
<!--- korbi internal id:c31de323-3687-4543-9519-48682782138d -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982281457
##########
superset/commands/report/execute.py:
##########
@@ -571,10 +586,13 @@ def _send(
recipient.type = ReportRecipientType.SLACKV2
notification = create_notification(recipient, notification_content)
notification.send()
- except (UpdateFailedError, NotificationParamException) as err:
- # log the error but keep processing the report with SlackV1
- logger.warning(
- "Failed to update slack recipients to v2: %s", str(err)
+ except (UpdateFailedError, NotificationParamException) as mig_err:
Review Comment:
can you catch these errors below on line 597?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982301199
##########
superset/commands/report/execute.py:
##########
@@ -139,22 +139,37 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ channels_list = get_recipients_list(slack_recipients["target"])
+ channels_list = [channel.lstrip("#") for channel in channels_list]
+ channels = get_channels_with_search(
+ search_string=channels_list,
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(channels_list) != len(channels):
+ missing_channels = set(channels_list) - {
+ channel["name"] for channel in channels
+ }
+ msg = (
+ "Could not find the following channels: "
+ f"{', '.join(missing_channels)}"
+ )
+ raise UpdateFailedError(msg)
+ channel_ids = ", ".join(channel["id"] for channel in channels)
recipient.recipient_config_json = json.dumps(
{
- "target": get_channels_with_search(
- slack_recipients["target"],
- types=[
- SlackChannelTypes.PRIVATE,
- SlackChannelTypes.PUBLIC,
- ],
- )
+ "target": channel_ids,
}
)
except Exception as ex:
- logger.warning(
- "Failed to update slack recipients to v2: %s", str(ex), exc_info=True
- )
- raise UpdateFailedError from ex
+ # Revert to v1 to preserve configuration (requires manual fix)
+ recipient.type = ReportRecipientType.SLACK
+ msg = f"Failed to update slack recipients to v2: {str(ex)}"
+ logger.error(msg, exc_info=True)
Review Comment:
great catch! I'll do that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1965870394
##########
superset/commands/report/execute.py:
##########
@@ -139,15 +139,21 @@
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ current_target_count = slack_recipients["target"].split(",")
Review Comment:
In general a valid thought, but this is post `validate()` when the report/alert is created so the `target` configuration should have been properly handled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982300748
##########
superset/reports/api.py:
##########
@@ -574,7 +574,7 @@ def slack_channels(self, **kwargs: Any) -> Response:
"""
try:
params = kwargs.get("rison", {})
- search_string = params.get("search_string")
+ search_string = [params.get("search_string", "").strip()]
Review Comment:
I think that's a valid concern. I can probably move this back there but still rely on the `util` we already have.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1983152654
##########
superset/reports/api.py:
##########
@@ -574,7 +574,7 @@ def slack_channels(self, **kwargs: Any) -> Response:
"""
try:
params = kwargs.get("rison", {})
- search_string = params.get("search_string")
+ search_string = [params.get("search_string", "").strip()]
Review Comment:
Ohh, I know why I moved this outside. It's because during the v1 -> v2 migration, I need to know the list length before calling `get_channels_with_search` to confirm if all channels were found. Doing it outside and then again in the method felt repetitive, but considering this is specific to the migration, and this code will be removed soon as only V2 is going to be supported going forward I think that's ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982288291
##########
superset/reports/api.py:
##########
@@ -574,7 +574,7 @@ def slack_channels(self, **kwargs: Any) -> Response:
"""
try:
params = kwargs.get("rison", {})
- search_string = params.get("search_string")
+ search_string = [params.get("search_string", "").strip()]
Review Comment:
My main concern here is really because we have this utility that converts a recipient string to a list, and so if we are going to pass around this search_string as a list, we should probably rename it as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "korbit-ai[bot] (via GitHub)" <gi...@apache.org>.
korbit-ai[bot] commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1964413618
##########
superset/commands/report/execute.py:
##########
@@ -139,15 +139,21 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ current_target_count = slack_recipients["target"].split(",")
+ channels = get_channels_with_search(
+ slack_recipients["target"],
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(current_target_count) != len(channels):
+ raise UpdateFailedError("Not all channels could be found")
Review Comment:
### Non-Specific Error Message <sub></sub>
<details>
<summary>Tell me more</summary>
###### What is the issue?
The error message doesn't provide specifics about which channels were not found, making it difficult to diagnose issues.
###### Why this matters
Without detailed information about missing channels, administrators would have difficulty troubleshooting failed migrations and fixing the underlying channel configuration issues.
###### Suggested change ∙ *Feature Preview*
Enhance error message with specific channel information:
```python
if len(current_target_count) != len(channels):
missing_channels = set(current_target_count) - {channel["name"] for channel in channels}
raise UpdateFailedError(f"Could not find the following channels: {', '.join(missing_channels)}")
```
</details>
<sub>
[](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/cb6bd7a0-ad45-4673-b4cd-1caa8c9de67b?suggestedFixEnabled=true)
💬 Chat with Korbit by mentioning @korbit-ai.
</sub>
<!--- korbi internal id:6d27d7bb-96bc-41ff-b6e1-8fcf5eb25a4c -->
##########
superset/commands/report/execute.py:
##########
@@ -139,15 +139,21 @@
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ current_target_count = slack_recipients["target"].split(",")
Review Comment:
### Unsafe String Split Operation <sub></sub>
<details>
<summary>Tell me more</summary>
###### What is the issue?
The split operation assumes the target string contains only comma-separated values, but could fail if the target string contains malformed data or is empty.
###### Why this matters
This could cause the migration to fail with an unhandled exception if the target string is malformed or empty, preventing successful migration of Slack recipients.
###### Suggested change ∙ *Feature Preview*
Add input validation and safe handling of the target string:
```python
current_target_count = slack_recipients.get("target", "").split(",") if slack_recipients.get("target") else []
```
</details>
<sub>
[](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/7f89236c-59fc-4530-b7b6-826e0e116496?suggestedFixEnabled=true)
💬 Chat with Korbit by mentioning @korbit-ai.
</sub>
<!--- korbi internal id:29157ecb-3038-4c52-80e0-d5c104d6e303 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "korbit-ai[bot] (via GitHub)" <gi...@apache.org>.
korbit-ai[bot] commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1980144308
##########
superset/reports/api.py:
##########
@@ -574,7 +575,7 @@ def slack_channels(self, **kwargs: Any) -> Response:
"""
try:
params = kwargs.get("rison", {})
- search_string = params.get("search_string")
+ search_string = get_recipients_list(params.get("search_string"))
Review Comment:
Thanks for the clarification, @Vitor-Avila. It seems reasonable to use another method for handling the string processing. However, it would still be better to handle this functionality without involving the `get_recipients_list` function, as the search string should not be a list of recipients. Reformulating the code to handle the search string without inadvertent splitting or altering would be beneficial for avoiding future errors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982278692
##########
superset/commands/report/execute.py:
##########
@@ -139,22 +139,37 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ channels_list = get_recipients_list(slack_recipients["target"])
+ channels_list = [channel.lstrip("#") for channel in channels_list]
+ channels = get_channels_with_search(
+ search_string=channels_list,
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(channels_list) != len(channels):
+ missing_channels = set(channels_list) - {
+ channel["name"] for channel in channels
+ }
+ msg = (
+ "Could not find the following channels: "
+ f"{', '.join(missing_channels)}"
+ )
+ raise UpdateFailedError(msg)
+ channel_ids = ", ".join(channel["id"] for channel in channels)
recipient.recipient_config_json = json.dumps(
{
- "target": get_channels_with_search(
- slack_recipients["target"],
- types=[
- SlackChannelTypes.PRIVATE,
- SlackChannelTypes.PUBLIC,
- ],
- )
+ "target": channel_ids,
}
)
except Exception as ex:
- logger.warning(
- "Failed to update slack recipients to v2: %s", str(ex), exc_info=True
- )
- raise UpdateFailedError from ex
+ # Revert to v1 to preserve configuration (requires manual fix)
+ recipient.type = ReportRecipientType.SLACK
+ msg = f"Failed to update slack recipients to v2: {str(ex)}"
+ logger.error(msg, exc_info=True)
Review Comment:
You can change this to logger.exception and remove the exc_info
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila merged PR #32336:
URL: https://github.com/apache/superset/pull/32336
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982301813
##########
superset/utils/core.py:
##########
@@ -811,7 +811,13 @@ def send_mime_email(
smtp.quit()
-def get_email_address_list(address_string: str) -> list[str]:
+def get_recipients_list(address_string: str | None) -> list[str]:
Review Comment:
Sure! I'll do that in another commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982306551
##########
superset/commands/report/execute.py:
##########
@@ -139,22 +139,37 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ channels_list = get_recipients_list(slack_recipients["target"])
+ channels_list = [channel.lstrip("#") for channel in channels_list]
+ channels = get_channels_with_search(
+ search_string=channels_list,
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(channels_list) != len(channels):
+ missing_channels = set(channels_list) - {
+ channel["name"] for channel in channels
+ }
+ msg = (
+ "Could not find the following channels: "
+ f"{', '.join(missing_channels)}"
+ )
+ raise UpdateFailedError(msg)
Review Comment:
@eschutho from my tests currently (before this PR) what's happening is that the execution itself is listed as successful in the execution logs, but nothing is sent to Slack. I thought about calling `send()` with the type set to `V1` again, but that would cause an infinite loop as we would validate again if there are scopes to migrate, attempt the migration, fail again, etc. Calling it with `V2` would also fail as we don't have all IDs.
Considering that the Slack deprecation is next Tuesday, I think it makes sense to consider any issues to migrate to V2 an error that should be fixed asap. Let me know your thoughts -- happy to think of other alternatives.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982267371
##########
superset/reports/api.py:
##########
@@ -574,7 +574,7 @@ def slack_channels(self, **kwargs: Any) -> Response:
"""
try:
params = kwargs.get("rison", {})
- search_string = params.get("search_string")
+ search_string = [params.get("search_string", "").strip()]
Review Comment:
Any benefit for doing the search string cleanup here (and other place where you pass it to `get_channels_with_search`) as opposed to letting the `get_channels_with_search` be responsible for it in one place?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982587425
##########
superset/commands/report/execute.py:
##########
@@ -139,22 +139,37 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ channels_list = get_recipients_list(slack_recipients["target"])
+ channels_list = [channel.lstrip("#") for channel in channels_list]
+ channels = get_channels_with_search(
+ search_string=channels_list,
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(channels_list) != len(channels):
+ missing_channels = set(channels_list) - {
+ channel["name"] for channel in channels
+ }
+ msg = (
+ "Could not find the following channels: "
+ f"{', '.join(missing_channels)}"
+ )
+ raise UpdateFailedError(msg)
Review Comment:
Thanks! Won't resolve this as I think it's good context for someone looking at the changes in the future
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982590984
##########
superset/commands/report/execute.py:
##########
@@ -139,22 +139,37 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ channels_list = get_recipients_list(slack_recipients["target"])
+ channels_list = [channel.lstrip("#") for channel in channels_list]
+ channels = get_channels_with_search(
+ search_string=channels_list,
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(channels_list) != len(channels):
+ missing_channels = set(channels_list) - {
+ channel["name"] for channel in channels
+ }
+ msg = (
+ "Could not find the following channels: "
+ f"{', '.join(missing_channels)}"
+ )
+ raise UpdateFailedError(msg)
+ channel_ids = ", ".join(channel["id"] for channel in channels)
Review Comment:
Oh, we don't. Removing this now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982314011
##########
superset/commands/report/execute.py:
##########
@@ -139,22 +139,37 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ channels_list = get_recipients_list(slack_recipients["target"])
+ channels_list = [channel.lstrip("#") for channel in channels_list]
+ channels = get_channels_with_search(
+ search_string=channels_list,
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(channels_list) != len(channels):
+ missing_channels = set(channels_list) - {
+ channel["name"] for channel in channels
+ }
+ msg = (
+ "Could not find the following channels: "
+ f"{', '.join(missing_channels)}"
+ )
+ raise UpdateFailedError(msg)
+ channel_ids = ", ".join(channel["id"] for channel in channels)
Review Comment:
nit, do we need the space after the comma here?
```suggestion
channel_ids = ",".join(channel["id"] for channel in channels)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982315751
##########
superset/commands/report/execute.py:
##########
@@ -139,22 +139,37 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ channels_list = get_recipients_list(slack_recipients["target"])
+ channels_list = [channel.lstrip("#") for channel in channels_list]
+ channels = get_channels_with_search(
+ search_string=channels_list,
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(channels_list) != len(channels):
+ missing_channels = set(channels_list) - {
+ channel["name"] for channel in channels
+ }
+ msg = (
+ "Could not find the following channels: "
+ f"{', '.join(missing_channels)}"
+ )
+ raise UpdateFailedError(msg)
+ channel_ids = ", ".join(channel["id"] for channel in channels)
recipient.recipient_config_json = json.dumps(
{
- "target": get_channels_with_search(
- slack_recipients["target"],
- types=[
- SlackChannelTypes.PRIVATE,
- SlackChannelTypes.PUBLIC,
- ],
- )
+ "target": channel_ids,
Review Comment:
Thanks for catching this error/missing step, btw!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1980143853
##########
superset/reports/api.py:
##########
@@ -574,7 +575,7 @@ def slack_channels(self, **kwargs: Any) -> Response:
"""
try:
params = kwargs.get("rison", {})
- search_string = params.get("search_string")
+ search_string = get_recipients_list(params.get("search_string"))
Review Comment:
I was using `get_recipients_list` here just to `strip()` and return a `list`. I'll remove it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1976101198
##########
superset/commands/report/execute.py:
##########
@@ -139,15 +139,21 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ current_target_count = slack_recipients["target"].split(",")
+ channels = get_channels_with_search(
+ slack_recipients["target"],
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(current_target_count) != len(channels):
+ raise UpdateFailedError("Not all channels could be found")
Review Comment:
This is good feedback! I'll add this in a following commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1965870394
##########
superset/commands/report/execute.py:
##########
@@ -139,15 +139,21 @@
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ current_target_count = slack_recipients["target"].split(",")
Review Comment:
In general a valid thought, but this is post `validate()` when the report/alert is created/updated so the `target` configuration should have been properly handled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982290139
##########
superset/commands/report/execute.py:
##########
@@ -139,22 +139,37 @@ def update_report_schedule_slack_v2(self) -> None:
slack_recipients = json.loads(recipient.recipient_config_json)
# we need to ensure that existing reports can also fetch
# ids from private channels
+ channels_list = get_recipients_list(slack_recipients["target"])
+ channels_list = [channel.lstrip("#") for channel in channels_list]
+ channels = get_channels_with_search(
+ search_string=channels_list,
+ types=[
+ SlackChannelTypes.PRIVATE,
+ SlackChannelTypes.PUBLIC,
+ ],
+ exact_match=True,
+ )
+ if len(channels_list) != len(channels):
+ missing_channels = set(channels_list) - {
+ channel["name"] for channel in channels
+ }
+ msg = (
+ "Could not find the following channels: "
+ f"{', '.join(missing_channels)}"
+ )
+ raise UpdateFailedError(msg)
Review Comment:
Do we want to block the conversion to v2 if channel ids can't be found or just log them and continue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(Slack): Fix Slack recipients migration to V2 [superset]
Posted by "Vitor-Avila (via GitHub)" <gi...@apache.org>.
Vitor-Avila commented on code in PR #32336:
URL: https://github.com/apache/superset/pull/32336#discussion_r1982607943
##########
superset/commands/report/execute.py:
##########
@@ -571,10 +586,13 @@ def _send(
recipient.type = ReportRecipientType.SLACKV2
notification = create_notification(recipient, notification_content)
notification.send()
- except (UpdateFailedError, NotificationParamException) as err:
- # log the error but keep processing the report with SlackV1
- logger.warning(
- "Failed to update slack recipients to v2: %s", str(err)
+ except (UpdateFailedError, NotificationParamException) as mig_err:
Review Comment:
This is actually inside an except block that's parallel to 597. But I can move that line to an "outer" try/except and have this specific except nested and I think that would remove this duplicated syntax. I'll try that now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org