You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "shivaam (via GitHub)" <gi...@apache.org> on 2026/10/13 05:28:20 UTC

[PR] Add legacy SMTP delivery compatibility to the provider [airflow]

shivaam opened a new pull request, #74623:
URL: https://github.com/apache/airflow/pull/74623

   Core's email wrapper and the SMTP provider currently own different SMTP delivery policies. Sending existing core callers through `SmtpHook` would combine a code-ownership migration with changes to configuration, authentication, retries and failure handling. This PR prepares the provider capability first so that a later core migration can preserve users' existing settings and behavior.
   
   Add `airflow.providers.smtp.utils.legacy.send_mime_email`, which accepts an already-built MIME message, resolved SMTP settings/credentials, and a lazy SSL-context factory. It does not read Airflow configuration or Connections, and it adds no dependency. New Dag code should continue using the existing hook, operator or notifier.
   
   This is an enabling slice, not the complete core-email migration. Core callers, message construction, templates, sender precedence, the configured backend, and the modern SMTP hook/operator/notifier are unchanged. Core adoption must follow a released provider containing this API, coordinated with the release managers, and be validated against installed artifacts rather than only a source checkout.
   
   Compatibility intentionally preserves constructor-only disconnection retries, ignored partial-recipient refusal, pre-send logging, negative-retry no-op behavior, and existing post-construction failure/cleanup semantics. A delivery or QUIT failure can occur after server acceptance; a caller retry can duplicate mail. This PR does not claim exactly-once delivery or improve these policies. Policy improvements require separate review rather than silently changing the ownership move.
   
   The compatibility API is public because its intended consumer is in another distribution. Approval of that cross-distribution contract is part of this review. Future notes about private core patch targets belong in the core follow-up, not this provider release.
   
   related: #30530
   
   ## Verification
   
   Verified locally on Python 3.11 at `5ff60a1b9f454486cc2715bf21dbf520c949bfd3`, rebased onto `9fc4143efc5eba3b0436122ff7505bc516e8f80a`:
   
   - 229 regressions: 99 SMTP, 34 core/processor email, 16 Task SDK email, 77 SES and 3 SendGrid.
   - 1,012 compatibility/Kubernetes tests passed; 29 expected skips. This selection excludes explicitly database-marked tests.
   - New transport: 100% statement/branch coverage (24 statements, 10 branches). An in-memory negative control that discards the TLS context caused three expected test failures; tracked sources were not mutated.
   - Real Mailpit: 22 checks across plain SMTP, STARTTLS, implicit TLS, rejected credentials/certificates, outage/recovery, unchanged EmailOperator execution and repeated notifier calls. Parsed captures verify Unicode, binary attachment and To/Cc/Bcc envelope data.
   - Source distribution and wheel build; installation outside the checkout and one real SMTP send from the installed wheel. The import path was asserted to belong to the wheel target, not a mounted source package.
   - SMTP-package mypy, full-provider manual type gate, regular/commit hooks, documentation builds and spelling passed. Two fresh independent Claude Code reviews were reconciled; no remaining P1/P2 findings.
   
   The real-Mailpit checks prove SMTP acceptance and stored MIME/envelope data, not delivery to Gmail, spam placement, DNS/MX/SPF/DKIM/DMARC, or a scheduler-triggered alert. This is not a full multi-Python or database CI matrix. No newsfragment is included, following provider release guidance.
   
   @eladkal could you review this provider-first compatibility boundary as an initial step toward the core email migration? In particular, feedback on the explicit-input transport API and release ordering would help before preparing the separate core-delegation PR.
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes — Codex and Claude Code (independent review)
   
   Generated-by: Codex with Claude Code review, following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   
   ---
   
   Drafted-by: Codex (no human review of this description before posting)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add emailer utilities to the SMTP provider [airflow]

Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala merged PR #74623:
URL: https://github.com/apache/airflow/pull/74623


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add emailer utilities to the SMTP provider [airflow]

Posted by "shivaam (via GitHub)" <gi...@apache.org>.
shivaam commented on code in PR #74623:
URL: https://github.com/apache/airflow/pull/74623#discussion_r4327479803


##########
providers/smtp/src/airflow/providers/smtp/utils/emailer.py:
##########
@@ -0,0 +1,340 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Send email using the ``[email]`` and ``[smtp]`` Airflow configuration sections."""
+
+from __future__ import annotations
+
+import collections.abc
+import logging
+import os
+import re
+import smtplib
+import ssl
+from collections.abc import Iterable
+from email.mime.application import MIMEApplication
+from email.mime.multipart import MIMEMultipart
+from email.mime.text import MIMEText
+from email.utils import formatdate
+from typing import Any
+
+from airflow.providers.common.compat.sdk import AirflowException, BaseHook, conf
+
+log = logging.getLogger(__name__)
+
+
+def send_email(
+    to: list[str] | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    dryrun: bool = False,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    conn_id: str | None = None,
+    custom_headers: dict[str, Any] | None = None,
+    **kwargs,
+) -> None:
+    """
+    Send an email using the backend specified in the *EMAIL_BACKEND* configuration option.
+
+    :param to: A list or iterable of email addresses to send the email to.
+    :param subject: The subject of the email.
+    :param html_content: The content of the email in HTML format.
+    :param files: A list of paths to files to attach to the email.
+    :param dryrun: If *True*, the email will not actually be sent. Default: *False*.
+    :param cc: A string or iterable of strings containing email addresses to send a copy of the email to.
+    :param bcc: A string or iterable of strings containing email addresses to send a
+        blind carbon copy of the email to.
+    :param mime_subtype: The subtype of the MIME message. Default: "mixed".
+    :param mime_charset: The charset of the email. Default: "utf-8".
+    :param conn_id: The connection ID to use for the backend. If not provided, the default connection
+        specified in the *EMAIL_CONN_ID* configuration option will be used.
+    :param custom_headers: A dictionary of additional headers to add to the MIME message.
+        No validations are run on these values, and they should be able to be encoded.
+    :param kwargs: Additional keyword arguments to pass to the backend.
+    """
+    backend = conf.getimport("email", "EMAIL_BACKEND")
+    backend_conn_id = conn_id or conf.get("email", "EMAIL_CONN_ID")
+    from_email = conf.get("email", "from_email", fallback=None)
+
+    to_list = get_email_address_list(to)
+    to_comma_separated = ", ".join(to_list)
+
+    return backend(
+        to_comma_separated,
+        subject,
+        html_content,
+        files=files,
+        dryrun=dryrun,
+        cc=cc,
+        bcc=bcc,
+        mime_subtype=mime_subtype,
+        mime_charset=mime_charset,
+        conn_id=backend_conn_id,
+        from_email=from_email,
+        custom_headers=custom_headers,
+        **kwargs,
+    )
+
+
+def send_email_smtp(
+    to: str | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    dryrun: bool = False,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    conn_id: str = "smtp_default",
+    from_email: str | None = None,
+    custom_headers: dict[str, Any] | None = None,
+    **kwargs,
+) -> None:
+    """
+    Send an email with html content.
+
+    :param to: Recipient email address or list of addresses.
+    :param subject: Email subject.
+    :param html_content: Email body in HTML format.
+    :param files: List of file paths to attach to the email.
+    :param dryrun: If True, the email will not be sent, but all other actions will be performed.
+    :param cc: Carbon copy recipient email address or list of addresses.
+    :param bcc: Blind carbon copy recipient email address or list of addresses.
+    :param mime_subtype: MIME subtype of the email.
+    :param mime_charset: MIME charset of the email.
+    :param conn_id: Connection ID of the SMTP server.
+    :param from_email: Sender email address.
+    :param custom_headers: Dictionary of custom headers to include in the email.
+    :param kwargs: Additional keyword arguments.
+
+    >>> send_email("test@example.com", "foo", "<b>Foo</b> bar", ["/dev/null"], dryrun=True)
+    """
+    smtp_mail_from = conf.get("smtp", "SMTP_MAIL_FROM")
+
+    if smtp_mail_from is not None:
+        mail_from = smtp_mail_from
+    else:
+        if from_email is None:
+            raise ValueError(
+                "You should set from email - either by smtp/smtp_mail_from config or `from_email` parameter"
+            )
+        mail_from = from_email
+
+    msg, recipients = build_mime_message(
+        mail_from=mail_from,
+        to=to,
+        subject=subject,
+        html_content=html_content,
+        files=files,
+        cc=cc,
+        bcc=bcc,
+        mime_subtype=mime_subtype,
+        mime_charset=mime_charset,
+        custom_headers=custom_headers,
+    )
+
+    send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)
+
+
+def build_mime_message(
+    mail_from: str | None,
+    to: str | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    custom_headers: dict[str, Any] | None = None,
+) -> tuple[MIMEMultipart, list[str]]:
+    """
+    Build a MIME message that can be used to send an email and returns a full list of recipients.
+
+    :param mail_from: Email address to set as the email's "From" field.
+    :param to: A string or iterable of strings containing email addresses to set as the email's "To" field.
+    :param subject: The subject of the email.
+    :param html_content: The content of the email in HTML format.
+    :param files: A list of paths to files to be attached to the email.
+    :param cc: A string or iterable of strings containing email addresses to set as the email's "CC" field.
+    :param bcc: A string or iterable of strings containing email addresses to set as the email's "BCC" field.
+    :param mime_subtype: The subtype of the MIME message. Default: "mixed".
+    :param mime_charset: The charset of the email. Default: "utf-8".
+    :param custom_headers: Additional headers to add to the MIME message. No validations are run on these
+        values, and they should be able to be encoded.
+    :return: A tuple containing the email as a MIMEMultipart object and a list of recipient email addresses.
+    """
+    to = get_email_address_list(to)
+
+    msg = MIMEMultipart(mime_subtype)
+    msg["Subject"] = subject
+    if mail_from:
+        msg["From"] = mail_from
+    msg["To"] = ", ".join(to)
+    recipients = to
+    if cc:
+        cc = get_email_address_list(cc)
+        msg["CC"] = ", ".join(cc)
+        recipients += cc
+
+    if bcc:
+        # don't add bcc in header
+        bcc = get_email_address_list(bcc)
+        recipients += bcc
+
+    msg["Date"] = formatdate(localtime=True)
+    mime_text = MIMEText(html_content, "html", mime_charset)
+    msg.attach(mime_text)
+
+    for fname in files or []:
+        basename = os.path.basename(fname)
+        with open(fname, "rb") as file:
+            part = MIMEApplication(file.read(), Name=basename)
+            part.add_header("Content-Disposition", "attachment", filename=basename)
+            part["Content-ID"] = f"<{basename}>"
+            msg.attach(part)
+
+    if custom_headers:
+        for header_key, header_value in custom_headers.items():
+            msg[header_key] = header_value
+
+    return msg, recipients
+
+
+def send_mime_email(
+    e_from: str,
+    e_to: str | list[str],
+    mime_msg: MIMEMultipart,
+    conn_id: str = "smtp_default",
+    dryrun: bool = False,
+) -> None:
+    """
+    Send a MIME email.
+
+    :param e_from: The email address of the sender.
+    :param e_to: The email address or a list of email addresses of the recipient(s).
+    :param mime_msg: The MIME message to send.
+    :param conn_id: The ID of the SMTP connection to use.
+    :param dryrun: If True, the email will not be sent, but a log message will be generated.
+    """
+    smtp_host = conf.get_mandatory_value("smtp", "SMTP_HOST")
+    smtp_port = conf.getint("smtp", "SMTP_PORT")
+    smtp_starttls = conf.getboolean("smtp", "SMTP_STARTTLS")
+    smtp_ssl = conf.getboolean("smtp", "SMTP_SSL")
+    smtp_retry_limit = conf.getint("smtp", "SMTP_RETRY_LIMIT")
+    smtp_timeout = conf.getint("smtp", "SMTP_TIMEOUT")
+    smtp_user = None
+    smtp_password = None
+
+    if conn_id is not None:
+        try:
+            airflow_conn = BaseHook.get_connection(conn_id)

Review Comment:
   Thanks, I tested this on 3.1.8 and main with the SMTP login stored only in the metadata DB. Marking a task failed through the API runs the listener in the API server, which sets `_AIRFLOW_PROCESS_CONTEXT=server`, so `BaseHook.get_connection` reads the DB and the email is sent, same as core.
   
   The gap I could reproduce is a plain script with no Airflow process context: core finds the DB login, this module doesn't. I'll cover that in the core PR with a regression test and a decision on how to handle it.
   
   ---
   Drafted-by: Claude Code (Opus 5.5); reviewed by @shivaam before posting
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add legacy SMTP delivery compatibility to the provider [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #74623:
URL: https://github.com/apache/airflow/pull/74623#discussion_r4255668036


##########
providers/smtp/docs/utils/legacy.rst:
##########


Review Comment:
   I don't think we need that.
   The plan should be simple - copy all related items to provider
   after release deprecate all items in core and set them to read from provider while verifying min version of the provider in core.



##########
providers/smtp/src/airflow/providers/smtp/utils/legacy.py:
##########
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transport compatibility for callers that resolve legacy email settings."""
+
+from __future__ import annotations
+
+import logging
+import smtplib
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    import ssl
+    from collections.abc import Callable
+    from email.mime.multipart import MIMEMultipart
+
+log = logging.getLogger(__name__)
+
+
+def send_mime_email(

Review Comment:
   We already have that in smtp provider utils don't we?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add emailer utilities to the SMTP provider [airflow]

Posted by "shivaam (via GitHub)" <gi...@apache.org>.
shivaam commented on code in PR #74623:
URL: https://github.com/apache/airflow/pull/74623#discussion_r4327479803


##########
providers/smtp/src/airflow/providers/smtp/utils/emailer.py:
##########
@@ -0,0 +1,340 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Send email using the ``[email]`` and ``[smtp]`` Airflow configuration sections."""
+
+from __future__ import annotations
+
+import collections.abc
+import logging
+import os
+import re
+import smtplib
+import ssl
+from collections.abc import Iterable
+from email.mime.application import MIMEApplication
+from email.mime.multipart import MIMEMultipart
+from email.mime.text import MIMEText
+from email.utils import formatdate
+from typing import Any
+
+from airflow.providers.common.compat.sdk import AirflowException, BaseHook, conf
+
+log = logging.getLogger(__name__)
+
+
+def send_email(
+    to: list[str] | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    dryrun: bool = False,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    conn_id: str | None = None,
+    custom_headers: dict[str, Any] | None = None,
+    **kwargs,
+) -> None:
+    """
+    Send an email using the backend specified in the *EMAIL_BACKEND* configuration option.
+
+    :param to: A list or iterable of email addresses to send the email to.
+    :param subject: The subject of the email.
+    :param html_content: The content of the email in HTML format.
+    :param files: A list of paths to files to attach to the email.
+    :param dryrun: If *True*, the email will not actually be sent. Default: *False*.
+    :param cc: A string or iterable of strings containing email addresses to send a copy of the email to.
+    :param bcc: A string or iterable of strings containing email addresses to send a
+        blind carbon copy of the email to.
+    :param mime_subtype: The subtype of the MIME message. Default: "mixed".
+    :param mime_charset: The charset of the email. Default: "utf-8".
+    :param conn_id: The connection ID to use for the backend. If not provided, the default connection
+        specified in the *EMAIL_CONN_ID* configuration option will be used.
+    :param custom_headers: A dictionary of additional headers to add to the MIME message.
+        No validations are run on these values, and they should be able to be encoded.
+    :param kwargs: Additional keyword arguments to pass to the backend.
+    """
+    backend = conf.getimport("email", "EMAIL_BACKEND")
+    backend_conn_id = conn_id or conf.get("email", "EMAIL_CONN_ID")
+    from_email = conf.get("email", "from_email", fallback=None)
+
+    to_list = get_email_address_list(to)
+    to_comma_separated = ", ".join(to_list)
+
+    return backend(
+        to_comma_separated,
+        subject,
+        html_content,
+        files=files,
+        dryrun=dryrun,
+        cc=cc,
+        bcc=bcc,
+        mime_subtype=mime_subtype,
+        mime_charset=mime_charset,
+        conn_id=backend_conn_id,
+        from_email=from_email,
+        custom_headers=custom_headers,
+        **kwargs,
+    )
+
+
+def send_email_smtp(
+    to: str | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    dryrun: bool = False,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    conn_id: str = "smtp_default",
+    from_email: str | None = None,
+    custom_headers: dict[str, Any] | None = None,
+    **kwargs,
+) -> None:
+    """
+    Send an email with html content.
+
+    :param to: Recipient email address or list of addresses.
+    :param subject: Email subject.
+    :param html_content: Email body in HTML format.
+    :param files: List of file paths to attach to the email.
+    :param dryrun: If True, the email will not be sent, but all other actions will be performed.
+    :param cc: Carbon copy recipient email address or list of addresses.
+    :param bcc: Blind carbon copy recipient email address or list of addresses.
+    :param mime_subtype: MIME subtype of the email.
+    :param mime_charset: MIME charset of the email.
+    :param conn_id: Connection ID of the SMTP server.
+    :param from_email: Sender email address.
+    :param custom_headers: Dictionary of custom headers to include in the email.
+    :param kwargs: Additional keyword arguments.
+
+    >>> send_email("test@example.com", "foo", "<b>Foo</b> bar", ["/dev/null"], dryrun=True)
+    """
+    smtp_mail_from = conf.get("smtp", "SMTP_MAIL_FROM")
+
+    if smtp_mail_from is not None:
+        mail_from = smtp_mail_from
+    else:
+        if from_email is None:
+            raise ValueError(
+                "You should set from email - either by smtp/smtp_mail_from config or `from_email` parameter"
+            )
+        mail_from = from_email
+
+    msg, recipients = build_mime_message(
+        mail_from=mail_from,
+        to=to,
+        subject=subject,
+        html_content=html_content,
+        files=files,
+        cc=cc,
+        bcc=bcc,
+        mime_subtype=mime_subtype,
+        mime_charset=mime_charset,
+        custom_headers=custom_headers,
+    )
+
+    send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)
+
+
+def build_mime_message(
+    mail_from: str | None,
+    to: str | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    custom_headers: dict[str, Any] | None = None,
+) -> tuple[MIMEMultipart, list[str]]:
+    """
+    Build a MIME message that can be used to send an email and returns a full list of recipients.
+
+    :param mail_from: Email address to set as the email's "From" field.
+    :param to: A string or iterable of strings containing email addresses to set as the email's "To" field.
+    :param subject: The subject of the email.
+    :param html_content: The content of the email in HTML format.
+    :param files: A list of paths to files to be attached to the email.
+    :param cc: A string or iterable of strings containing email addresses to set as the email's "CC" field.
+    :param bcc: A string or iterable of strings containing email addresses to set as the email's "BCC" field.
+    :param mime_subtype: The subtype of the MIME message. Default: "mixed".
+    :param mime_charset: The charset of the email. Default: "utf-8".
+    :param custom_headers: Additional headers to add to the MIME message. No validations are run on these
+        values, and they should be able to be encoded.
+    :return: A tuple containing the email as a MIMEMultipart object and a list of recipient email addresses.
+    """
+    to = get_email_address_list(to)
+
+    msg = MIMEMultipart(mime_subtype)
+    msg["Subject"] = subject
+    if mail_from:
+        msg["From"] = mail_from
+    msg["To"] = ", ".join(to)
+    recipients = to
+    if cc:
+        cc = get_email_address_list(cc)
+        msg["CC"] = ", ".join(cc)
+        recipients += cc
+
+    if bcc:
+        # don't add bcc in header
+        bcc = get_email_address_list(bcc)
+        recipients += bcc
+
+    msg["Date"] = formatdate(localtime=True)
+    mime_text = MIMEText(html_content, "html", mime_charset)
+    msg.attach(mime_text)
+
+    for fname in files or []:
+        basename = os.path.basename(fname)
+        with open(fname, "rb") as file:
+            part = MIMEApplication(file.read(), Name=basename)
+            part.add_header("Content-Disposition", "attachment", filename=basename)
+            part["Content-ID"] = f"<{basename}>"
+            msg.attach(part)
+
+    if custom_headers:
+        for header_key, header_value in custom_headers.items():
+            msg[header_key] = header_value
+
+    return msg, recipients
+
+
+def send_mime_email(
+    e_from: str,
+    e_to: str | list[str],
+    mime_msg: MIMEMultipart,
+    conn_id: str = "smtp_default",
+    dryrun: bool = False,
+) -> None:
+    """
+    Send a MIME email.
+
+    :param e_from: The email address of the sender.
+    :param e_to: The email address or a list of email addresses of the recipient(s).
+    :param mime_msg: The MIME message to send.
+    :param conn_id: The ID of the SMTP connection to use.
+    :param dryrun: If True, the email will not be sent, but a log message will be generated.
+    """
+    smtp_host = conf.get_mandatory_value("smtp", "SMTP_HOST")
+    smtp_port = conf.getint("smtp", "SMTP_PORT")
+    smtp_starttls = conf.getboolean("smtp", "SMTP_STARTTLS")
+    smtp_ssl = conf.getboolean("smtp", "SMTP_SSL")
+    smtp_retry_limit = conf.getint("smtp", "SMTP_RETRY_LIMIT")
+    smtp_timeout = conf.getint("smtp", "SMTP_TIMEOUT")
+    smtp_user = None
+    smtp_password = None
+
+    if conn_id is not None:
+        try:
+            airflow_conn = BaseHook.get_connection(conn_id)

Review Comment:
   Thanks, I tested this on 3.1.8 and main with the SMTP login stored only in the metadata DB. Marking a task failed through the API runs the listener in the API server, which sets `_AIRFLOW_PROCESS_CONTEXT=server`, so `BaseHook.get_connection` reads the DB and the email is sent, same as core.
   
   The gap I could reproduce is a plain script with no Airflow process context: core finds the DB login, this module doesn't. I'll cover that in the core PR with a regression test and a decision on how to handle it.
   
   ---
   Drafted-by: Claude Code (Opus 5.5) (no human review before posting)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add emailer utilities to the SMTP provider [airflow]

Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on code in PR #74623:
URL: https://github.com/apache/airflow/pull/74623#discussion_r4327769217


##########
providers/smtp/src/airflow/providers/smtp/utils/emailer.py:
##########
@@ -0,0 +1,340 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Send email using the ``[email]`` and ``[smtp]`` Airflow configuration sections."""
+
+from __future__ import annotations
+
+import collections.abc
+import logging
+import os
+import re
+import smtplib
+import ssl
+from collections.abc import Iterable
+from email.mime.application import MIMEApplication
+from email.mime.multipart import MIMEMultipart
+from email.mime.text import MIMEText
+from email.utils import formatdate
+from typing import Any
+
+from airflow.providers.common.compat.sdk import AirflowException, BaseHook, conf
+
+log = logging.getLogger(__name__)
+
+
+def send_email(
+    to: list[str] | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    dryrun: bool = False,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    conn_id: str | None = None,
+    custom_headers: dict[str, Any] | None = None,
+    **kwargs,
+) -> None:
+    """
+    Send an email using the backend specified in the *EMAIL_BACKEND* configuration option.
+
+    :param to: A list or iterable of email addresses to send the email to.
+    :param subject: The subject of the email.
+    :param html_content: The content of the email in HTML format.
+    :param files: A list of paths to files to attach to the email.
+    :param dryrun: If *True*, the email will not actually be sent. Default: *False*.
+    :param cc: A string or iterable of strings containing email addresses to send a copy of the email to.
+    :param bcc: A string or iterable of strings containing email addresses to send a
+        blind carbon copy of the email to.
+    :param mime_subtype: The subtype of the MIME message. Default: "mixed".
+    :param mime_charset: The charset of the email. Default: "utf-8".
+    :param conn_id: The connection ID to use for the backend. If not provided, the default connection
+        specified in the *EMAIL_CONN_ID* configuration option will be used.
+    :param custom_headers: A dictionary of additional headers to add to the MIME message.
+        No validations are run on these values, and they should be able to be encoded.
+    :param kwargs: Additional keyword arguments to pass to the backend.
+    """
+    backend = conf.getimport("email", "EMAIL_BACKEND")
+    backend_conn_id = conn_id or conf.get("email", "EMAIL_CONN_ID")
+    from_email = conf.get("email", "from_email", fallback=None)
+
+    to_list = get_email_address_list(to)
+    to_comma_separated = ", ".join(to_list)
+
+    return backend(
+        to_comma_separated,
+        subject,
+        html_content,
+        files=files,
+        dryrun=dryrun,
+        cc=cc,
+        bcc=bcc,
+        mime_subtype=mime_subtype,
+        mime_charset=mime_charset,
+        conn_id=backend_conn_id,
+        from_email=from_email,
+        custom_headers=custom_headers,
+        **kwargs,
+    )
+
+
+def send_email_smtp(
+    to: str | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    dryrun: bool = False,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    conn_id: str = "smtp_default",
+    from_email: str | None = None,
+    custom_headers: dict[str, Any] | None = None,
+    **kwargs,
+) -> None:
+    """
+    Send an email with html content.
+
+    :param to: Recipient email address or list of addresses.
+    :param subject: Email subject.
+    :param html_content: Email body in HTML format.
+    :param files: List of file paths to attach to the email.
+    :param dryrun: If True, the email will not be sent, but all other actions will be performed.
+    :param cc: Carbon copy recipient email address or list of addresses.
+    :param bcc: Blind carbon copy recipient email address or list of addresses.
+    :param mime_subtype: MIME subtype of the email.
+    :param mime_charset: MIME charset of the email.
+    :param conn_id: Connection ID of the SMTP server.
+    :param from_email: Sender email address.
+    :param custom_headers: Dictionary of custom headers to include in the email.
+    :param kwargs: Additional keyword arguments.
+
+    >>> send_email("test@example.com", "foo", "<b>Foo</b> bar", ["/dev/null"], dryrun=True)
+    """
+    smtp_mail_from = conf.get("smtp", "SMTP_MAIL_FROM")
+
+    if smtp_mail_from is not None:
+        mail_from = smtp_mail_from
+    else:
+        if from_email is None:
+            raise ValueError(
+                "You should set from email - either by smtp/smtp_mail_from config or `from_email` parameter"
+            )
+        mail_from = from_email
+
+    msg, recipients = build_mime_message(
+        mail_from=mail_from,
+        to=to,
+        subject=subject,
+        html_content=html_content,
+        files=files,
+        cc=cc,
+        bcc=bcc,
+        mime_subtype=mime_subtype,
+        mime_charset=mime_charset,
+        custom_headers=custom_headers,
+    )
+
+    send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)
+
+
+def build_mime_message(
+    mail_from: str | None,
+    to: str | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    custom_headers: dict[str, Any] | None = None,
+) -> tuple[MIMEMultipart, list[str]]:
+    """
+    Build a MIME message that can be used to send an email and returns a full list of recipients.
+
+    :param mail_from: Email address to set as the email's "From" field.
+    :param to: A string or iterable of strings containing email addresses to set as the email's "To" field.
+    :param subject: The subject of the email.
+    :param html_content: The content of the email in HTML format.
+    :param files: A list of paths to files to be attached to the email.
+    :param cc: A string or iterable of strings containing email addresses to set as the email's "CC" field.
+    :param bcc: A string or iterable of strings containing email addresses to set as the email's "BCC" field.
+    :param mime_subtype: The subtype of the MIME message. Default: "mixed".
+    :param mime_charset: The charset of the email. Default: "utf-8".
+    :param custom_headers: Additional headers to add to the MIME message. No validations are run on these
+        values, and they should be able to be encoded.
+    :return: A tuple containing the email as a MIMEMultipart object and a list of recipient email addresses.
+    """
+    to = get_email_address_list(to)
+
+    msg = MIMEMultipart(mime_subtype)
+    msg["Subject"] = subject
+    if mail_from:
+        msg["From"] = mail_from
+    msg["To"] = ", ".join(to)
+    recipients = to
+    if cc:
+        cc = get_email_address_list(cc)
+        msg["CC"] = ", ".join(cc)
+        recipients += cc
+
+    if bcc:
+        # don't add bcc in header
+        bcc = get_email_address_list(bcc)
+        recipients += bcc
+
+    msg["Date"] = formatdate(localtime=True)
+    mime_text = MIMEText(html_content, "html", mime_charset)
+    msg.attach(mime_text)
+
+    for fname in files or []:
+        basename = os.path.basename(fname)
+        with open(fname, "rb") as file:
+            part = MIMEApplication(file.read(), Name=basename)
+            part.add_header("Content-Disposition", "attachment", filename=basename)
+            part["Content-ID"] = f"<{basename}>"
+            msg.attach(part)
+
+    if custom_headers:
+        for header_key, header_value in custom_headers.items():
+            msg[header_key] = header_value
+
+    return msg, recipients
+
+
+def send_mime_email(
+    e_from: str,
+    e_to: str | list[str],
+    mime_msg: MIMEMultipart,
+    conn_id: str = "smtp_default",
+    dryrun: bool = False,
+) -> None:
+    """
+    Send a MIME email.
+
+    :param e_from: The email address of the sender.
+    :param e_to: The email address or a list of email addresses of the recipient(s).
+    :param mime_msg: The MIME message to send.
+    :param conn_id: The ID of the SMTP connection to use.
+    :param dryrun: If True, the email will not be sent, but a log message will be generated.
+    """
+    smtp_host = conf.get_mandatory_value("smtp", "SMTP_HOST")
+    smtp_port = conf.getint("smtp", "SMTP_PORT")
+    smtp_starttls = conf.getboolean("smtp", "SMTP_STARTTLS")
+    smtp_ssl = conf.getboolean("smtp", "SMTP_SSL")
+    smtp_retry_limit = conf.getint("smtp", "SMTP_RETRY_LIMIT")
+    smtp_timeout = conf.getint("smtp", "SMTP_TIMEOUT")
+    smtp_user = None
+    smtp_password = None
+
+    if conn_id is not None:
+        try:
+            airflow_conn = BaseHook.get_connection(conn_id)

Review Comment:
   Sounds good! thanks for testing it



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add legacy SMTP delivery compatibility to the provider [airflow]

Posted by "shivaam (via GitHub)" <gi...@apache.org>.
shivaam commented on PR #74623:
URL: https://github.com/apache/airflow/pull/74623#issuecomment-6139091194

   @eladkal — could you review the provider-first approach, compatibility API, and release ordering?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Copy core email utilities to the SMTP provider [airflow]

Posted by "shivaam (via GitHub)" <gi...@apache.org>.
shivaam commented on code in PR #74623:
URL: https://github.com/apache/airflow/pull/74623#discussion_r4326398249


##########
providers/smtp/docs/utils/emailer.rst:
##########
@@ -0,0 +1,36 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+Email backend
+=============
+
+:mod:`airflow.providers.smtp.utils.emailer` provides the email utilities previously
+available in core as ``airflow.utils.email``, with the same behavior. SMTP server

Review Comment:
   Done, the docs page no longer mentions core. It now only describes what the module does and how to configure it.
   
   ---
   Drafted-by: Claude Code (Opus 5.5); reviewed by @shivaam before posting
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add emailer utilities to the SMTP provider [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #74623:
URL: https://github.com/apache/airflow/pull/74623#issuecomment-6433161173

   ### Backport successfully created: v3-4-test
   
   Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
   the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
   
   In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
   
   <table>
           <tr>
           <th>Status</th>
           <th>Branch</th>
           <th>Result</th>
       </tr>
       <tr>
           <td>✅</td>
           <td>v3-4-test</td>
           <td><a href="https://github.com/apache/airflow/pull/75704"><img src="https://img.shields.io/badge/PR-75704-blue" alt="PR Link"></a></td>
       </tr>
   </table>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Copy core email utilities to the SMTP provider [airflow]

Posted by "shivaam (via GitHub)" <gi...@apache.org>.
shivaam commented on code in PR #74623:
URL: https://github.com/apache/airflow/pull/74623#discussion_r4304450331


##########
providers/smtp/docs/utils/legacy.rst:
##########


Review Comment:
   Agreed, reworked it that way. The PR now copies `airflow/utils/email.py` into `airflow.providers.smtp.utils.emailer` and drops the transport-only `legacy` module. The only code differences from core are the imports (through `common.compat`) and the Connection lookup, which uses `BaseHook.get_connection` instead of importing `airflow.models`. Once a provider release includes it, I'll open the core PR that deprecates `airflow.utils.email` and delegates to the provider, with the minimum SMTP provider version checked in core.
   
   
   ---
   Drafted-by: Claude Code (Opus 5.5); reviewed by @shivaam before posting
   



##########
providers/smtp/src/airflow/providers/smtp/utils/legacy.py:
##########
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transport compatibility for callers that resolve legacy email settings."""
+
+from __future__ import annotations
+
+import logging
+import smtplib
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    import ssl
+    from collections.abc import Callable
+    from email.mime.multipart import MIMEMultipart
+
+log = logging.getLogger(__name__)
+
+
+def send_mime_email(

Review Comment:
   Not today: the SMTP provider has no `utils` package on main. `SmtpHook` has similar private helpers, but it reads host, port and TLS from the Connection and retries failed sends, while the core functions read the `[smtp]` config section and only take the login and password from the Connection. Pointing core at the hook would change behavior for existing setups, so this copies the core module as is. Sharing the MIME helpers between the hook and this module can be a follow-up.
   
   
   ---
   Drafted-by: Claude Code (Opus 5.5); reviewed by @shivaam before posting
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Copy core email utilities to the SMTP provider [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on code in PR #74623:
URL: https://github.com/apache/airflow/pull/74623#discussion_r4326031818


##########
providers/smtp/docs/utils/emailer.rst:
##########
@@ -0,0 +1,36 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+Email backend
+=============
+
+:mod:`airflow.providers.smtp.utils.emailer` provides the email utilities previously
+available in core as ``airflow.utils.email``, with the same behavior. SMTP server

Review Comment:
   Lets avoid mentioning anything related to core.
   User who use this model don't need to know about the migration.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add emailer utilities to the SMTP provider [airflow]

Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on code in PR #74623:
URL: https://github.com/apache/airflow/pull/74623#discussion_r4326988390


##########
providers/smtp/src/airflow/providers/smtp/utils/emailer.py:
##########
@@ -0,0 +1,340 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Send email using the ``[email]`` and ``[smtp]`` Airflow configuration sections."""
+
+from __future__ import annotations
+
+import collections.abc
+import logging
+import os
+import re
+import smtplib
+import ssl
+from collections.abc import Iterable
+from email.mime.application import MIMEApplication
+from email.mime.multipart import MIMEMultipart
+from email.mime.text import MIMEText
+from email.utils import formatdate
+from typing import Any
+
+from airflow.providers.common.compat.sdk import AirflowException, BaseHook, conf
+
+log = logging.getLogger(__name__)
+
+
+def send_email(
+    to: list[str] | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    dryrun: bool = False,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    conn_id: str | None = None,
+    custom_headers: dict[str, Any] | None = None,
+    **kwargs,
+) -> None:
+    """
+    Send an email using the backend specified in the *EMAIL_BACKEND* configuration option.
+
+    :param to: A list or iterable of email addresses to send the email to.
+    :param subject: The subject of the email.
+    :param html_content: The content of the email in HTML format.
+    :param files: A list of paths to files to attach to the email.
+    :param dryrun: If *True*, the email will not actually be sent. Default: *False*.
+    :param cc: A string or iterable of strings containing email addresses to send a copy of the email to.
+    :param bcc: A string or iterable of strings containing email addresses to send a
+        blind carbon copy of the email to.
+    :param mime_subtype: The subtype of the MIME message. Default: "mixed".
+    :param mime_charset: The charset of the email. Default: "utf-8".
+    :param conn_id: The connection ID to use for the backend. If not provided, the default connection
+        specified in the *EMAIL_CONN_ID* configuration option will be used.
+    :param custom_headers: A dictionary of additional headers to add to the MIME message.
+        No validations are run on these values, and they should be able to be encoded.
+    :param kwargs: Additional keyword arguments to pass to the backend.
+    """
+    backend = conf.getimport("email", "EMAIL_BACKEND")
+    backend_conn_id = conn_id or conf.get("email", "EMAIL_CONN_ID")
+    from_email = conf.get("email", "from_email", fallback=None)
+
+    to_list = get_email_address_list(to)
+    to_comma_separated = ", ".join(to_list)
+
+    return backend(
+        to_comma_separated,
+        subject,
+        html_content,
+        files=files,
+        dryrun=dryrun,
+        cc=cc,
+        bcc=bcc,
+        mime_subtype=mime_subtype,
+        mime_charset=mime_charset,
+        conn_id=backend_conn_id,
+        from_email=from_email,
+        custom_headers=custom_headers,
+        **kwargs,
+    )
+
+
+def send_email_smtp(
+    to: str | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    dryrun: bool = False,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    conn_id: str = "smtp_default",
+    from_email: str | None = None,
+    custom_headers: dict[str, Any] | None = None,
+    **kwargs,
+) -> None:
+    """
+    Send an email with html content.
+
+    :param to: Recipient email address or list of addresses.
+    :param subject: Email subject.
+    :param html_content: Email body in HTML format.
+    :param files: List of file paths to attach to the email.
+    :param dryrun: If True, the email will not be sent, but all other actions will be performed.
+    :param cc: Carbon copy recipient email address or list of addresses.
+    :param bcc: Blind carbon copy recipient email address or list of addresses.
+    :param mime_subtype: MIME subtype of the email.
+    :param mime_charset: MIME charset of the email.
+    :param conn_id: Connection ID of the SMTP server.
+    :param from_email: Sender email address.
+    :param custom_headers: Dictionary of custom headers to include in the email.
+    :param kwargs: Additional keyword arguments.
+
+    >>> send_email("test@example.com", "foo", "<b>Foo</b> bar", ["/dev/null"], dryrun=True)
+    """
+    smtp_mail_from = conf.get("smtp", "SMTP_MAIL_FROM")
+
+    if smtp_mail_from is not None:
+        mail_from = smtp_mail_from
+    else:
+        if from_email is None:
+            raise ValueError(
+                "You should set from email - either by smtp/smtp_mail_from config or `from_email` parameter"
+            )
+        mail_from = from_email
+
+    msg, recipients = build_mime_message(
+        mail_from=mail_from,
+        to=to,
+        subject=subject,
+        html_content=html_content,
+        files=files,
+        cc=cc,
+        bcc=bcc,
+        mime_subtype=mime_subtype,
+        mime_charset=mime_charset,
+        custom_headers=custom_headers,
+    )
+
+    send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)
+
+
+def build_mime_message(
+    mail_from: str | None,
+    to: str | Iterable[str],
+    subject: str,
+    html_content: str,
+    files: list[str] | None = None,
+    cc: str | Iterable[str] | None = None,
+    bcc: str | Iterable[str] | None = None,
+    mime_subtype: str = "mixed",
+    mime_charset: str = "utf-8",
+    custom_headers: dict[str, Any] | None = None,
+) -> tuple[MIMEMultipart, list[str]]:
+    """
+    Build a MIME message that can be used to send an email and returns a full list of recipients.
+
+    :param mail_from: Email address to set as the email's "From" field.
+    :param to: A string or iterable of strings containing email addresses to set as the email's "To" field.
+    :param subject: The subject of the email.
+    :param html_content: The content of the email in HTML format.
+    :param files: A list of paths to files to be attached to the email.
+    :param cc: A string or iterable of strings containing email addresses to set as the email's "CC" field.
+    :param bcc: A string or iterable of strings containing email addresses to set as the email's "BCC" field.
+    :param mime_subtype: The subtype of the MIME message. Default: "mixed".
+    :param mime_charset: The charset of the email. Default: "utf-8".
+    :param custom_headers: Additional headers to add to the MIME message. No validations are run on these
+        values, and they should be able to be encoded.
+    :return: A tuple containing the email as a MIMEMultipart object and a list of recipient email addresses.
+    """
+    to = get_email_address_list(to)
+
+    msg = MIMEMultipart(mime_subtype)
+    msg["Subject"] = subject
+    if mail_from:
+        msg["From"] = mail_from
+    msg["To"] = ", ".join(to)
+    recipients = to
+    if cc:
+        cc = get_email_address_list(cc)
+        msg["CC"] = ", ".join(cc)
+        recipients += cc
+
+    if bcc:
+        # don't add bcc in header
+        bcc = get_email_address_list(bcc)
+        recipients += bcc
+
+    msg["Date"] = formatdate(localtime=True)
+    mime_text = MIMEText(html_content, "html", mime_charset)
+    msg.attach(mime_text)
+
+    for fname in files or []:
+        basename = os.path.basename(fname)
+        with open(fname, "rb") as file:
+            part = MIMEApplication(file.read(), Name=basename)
+            part.add_header("Content-Disposition", "attachment", filename=basename)
+            part["Content-ID"] = f"<{basename}>"
+            msg.attach(part)
+
+    if custom_headers:
+        for header_key, header_value in custom_headers.items():
+            msg[header_key] = header_value
+
+    return msg, recipients
+
+
+def send_mime_email(
+    e_from: str,
+    e_to: str | list[str],
+    mime_msg: MIMEMultipart,
+    conn_id: str = "smtp_default",
+    dryrun: bool = False,
+) -> None:
+    """
+    Send a MIME email.
+
+    :param e_from: The email address of the sender.
+    :param e_to: The email address or a list of email addresses of the recipient(s).
+    :param mime_msg: The MIME message to send.
+    :param conn_id: The ID of the SMTP connection to use.
+    :param dryrun: If True, the email will not be sent, but a log message will be generated.
+    """
+    smtp_host = conf.get_mandatory_value("smtp", "SMTP_HOST")
+    smtp_port = conf.getint("smtp", "SMTP_PORT")
+    smtp_starttls = conf.getboolean("smtp", "SMTP_STARTTLS")
+    smtp_ssl = conf.getboolean("smtp", "SMTP_SSL")
+    smtp_retry_limit = conf.getint("smtp", "SMTP_RETRY_LIMIT")
+    smtp_timeout = conf.getint("smtp", "SMTP_TIMEOUT")
+    smtp_user = None
+    smtp_password = None
+
+    if conn_id is not None:
+        try:
+            airflow_conn = BaseHook.get_connection(conn_id)

Review Comment:
   One non-blocking compatibility concern on Airflow 3.1: this lookup can fail when email is sent from a server-side listener.
   
   For example, marking a task failed through the UI/API invokes on_task_instance_failed in the API server. If that listener sends email through this backend and the SMTP connection is stored in the metadata database, the SDK lookup can raise an ImportError because it expects a task’s SUPERVISOR_COMMS channel.
   
   Core’s current email helper handles that case. Please add a regression test and preserve this behavior before switching core to this implementation. This can be a follow-up since this PR leaves core unchanged.
   This is based on source inspection, I haven’t reproduced it at runtime.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Copy core email utilities to the SMTP provider [airflow]

Posted by "shivaam (via GitHub)" <gi...@apache.org>.
shivaam commented on code in PR #74623:
URL: https://github.com/apache/airflow/pull/74623#discussion_r4304450331


##########
providers/smtp/docs/utils/legacy.rst:
##########


Review Comment:
   Agreed, reworked it that way. The PR now copies `airflow/utils/email.py` into `airflow.providers.smtp.utils.emailer` and drops the transport-only `legacy` module. The only code differences from core are the imports (through `common.compat`) and the Connection lookup, which uses `BaseHook.get_connection` instead of importing `airflow.models`. Once a provider release includes it, I'll open the core PR that deprecates `airflow.utils.email` and delegates to the provider, with the minimum SMTP provider version checked in core.



##########
providers/smtp/src/airflow/providers/smtp/utils/legacy.py:
##########
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transport compatibility for callers that resolve legacy email settings."""
+
+from __future__ import annotations
+
+import logging
+import smtplib
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    import ssl
+    from collections.abc import Callable
+    from email.mime.multipart import MIMEMultipart
+
+log = logging.getLogger(__name__)
+
+
+def send_mime_email(

Review Comment:
   Not today: the SMTP provider has no `utils` package on main. `SmtpHook` has similar private helpers, but it reads host, port and TLS from the Connection and retries failed sends, while the core functions read the `[smtp]` config section and only take the login and password from the Connection. Pointing core at the hook would change behavior for existing setups, so this copies the core module as is. Sharing the MIME helpers between the hook and this module can be a follow-up.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org