You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Feudjo (via GitHub)" <gi...@apache.org> on 2026/05/20 14:20:09 UTC

[PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Feudjo opened a new pull request, #67244:
URL: https://github.com/apache/airflow/pull/67244

    <!-- SPDX-License-Identifier: Apache-2.0
         https://www.apache.org/licenses/LICENSE-2.0 -->
   
   <!--
   Thank you for contributing!
   
   Please provide above a brief description of the changes made in this pull request.
   Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
   
   Please make sure that your code changes are covered with tests.
   And in case of new features or big changes remember to adjust the documentation.
   
   Feel free to ping (in general) for the review if you do not see reaction for a few days
   (72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
   
   In case of an existing issue, reference it using one of the following:
   
   * closes: #ISSUE
   * related: #ISSUE
   -->
   
   ---
   
   # fix: remove trigger event payload from INFO log in `run_trigger()`
    
   Closes #<ISSUE_NUMBER>
    
   ## What and Why
    
   In `TriggerRunner.run_trigger()`, the full `TriggerEvent` payload is logged at INFO:
    
   ```python
   # Before
   await self.log.ainfo(
       "Trigger fired event", name=self.triggers[trigger_id]["name"], result=event
   )
   ```
    
   `result=event` contains the complete serialised trigger response. For data-heavy
   triggers this can be megabytes of data logged on every execution, causing:
    
   - Memory pressure in the triggerer process
   - Sensitive API response data written to plain-text log files
   ## Change
    
   ```python
   # After
   await self.log.ainfo(
       "Trigger fired event", name=self.triggers[trigger_id]["name"]
   )
   ```
    The payload is not logged.
    
   
   ##### Was generative AI tooling used to co-author this PR?
   
   <!--
   If generative AI tooling has been used in the process of authoring this PR, please
   change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
   -->
   
   - [X ] Yes (please specify the tool below)
   
   
   Generated-by: [Claude] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   
   
   ---
   
   * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
   * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "Feudjo (via GitHub)" <gi...@apache.org>.
Feudjo commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-5670016874

   > Welcome to Apache Airflow, well done!
   
   Thanks. Happy to be contributing πŸ™πŸΌ.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "Feudjo (via GitHub)" <gi...@apache.org>.
Feudjo commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-4714901475

   Hi @potiuk 
   I'm seeing failures in airflow_local_settings.py in the CI run, but this file isn't part of my changes. I tried reproducing locally and I'm not sure if this is pre-existing on main or something my change triggered indirectly. Would appreciate your guidance on how to proceed β€” happy to investigate further if you can point me in the right direction.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #67244:
URL: https://github.com/apache/airflow/pull/67244#discussion_r3678107532


##########
airflow-core/tests/unit/jobs/test_triggerer_job.py:
##########
@@ -2589,3 +2589,87 @@ async def _drive():
     trigger_id, _event, seq = events[0]
     assert trigger_id == 1
     assert seq is None
+
+
+@pytest.mark.asyncio
+async def test_trigger_event_payload_not_logged_at_info(cap_structlog):
+    """Ensure the full event payload is not logged at INFO level."""
+    runner = TriggerRunner()
+    runner.triggers = {
+        1: {
+            "task": MagicMock(spec=asyncio.Task),
+            "is_watcher": False,
+            "name": "test_dag/run_id/test_task/0/1",
+            "events": 0,
+        }
+    }
+
+    mock_trigger = MagicMock(spec=BaseTrigger)
+    mock_trigger.task_instance = MagicMock()
+    mock_trigger.task_instance.map_index = -1
+
+    payload = {"api_response": {"token": "s3cr3t-api-k3y", "user_id": 42}}
+
+    async def fake_run():
+        yield TriggerEvent(payload)
+
+    mock_trigger.run = fake_run
+
+    mock_trigger.cleanup = AsyncMock()
+
+    task = asyncio.create_task(runner.run_trigger(1, mock_trigger))
+    await task
+
+    assert any(log["event"] == "Trigger fired event" for log in cap_structlog), (
+        "Expected a 'Trigger fired event' log entry"
+    )
+    info_logs = [log for log in cap_structlog if log.get("log_level") == "info"]
+
+    for _key, value in payload.items():
+        assert not any(str(value) in str(log) for log in info_logs), (
+            "payload value must not appear in INFO-level logs"
+        )
+
+
+@pytest.mark.asyncio
+async def test_trigger_event_payload_available_at_debug(cap_structlog):
+    runner = TriggerRunner()
+    runner.triggers = {
+        1: {
+            "task": MagicMock(spec=asyncio.Task),
+            "is_watcher": False,
+            "name": "test_dag/run_id/test_task/0/1",
+            "events": 0,
+        }
+    }
+
+    mock_trigger = MagicMock(spec=BaseTrigger)
+    mock_trigger.task_instance = MagicMock()
+    mock_trigger.task_instance.map_index = -1
+
+    payload = {"api_response": {"token": "s3cr3t-api-k3y", "user_id": 42}}
+
+    async def fake_run():
+        yield TriggerEvent(payload)
+
+    mock_trigger.run = fake_run
+
+    mock_log = AsyncMock()
+    mock_log.isEnabledFor = MagicMock(return_value=True)

Review Comment:
   This sets `isEnabledFor`, but the production code calls `is_enabled_for` (the structlog spelling β€” `isEnabledFor` exists only on `structlog.stdlib.BoundLogger`, not on the `FilteringBoundLogger` protocol used here).
   
   Since `runner.log` is an `AsyncMock`, `self.log.is_enabled_for(logging.DEBUG)` returns an auto-created child mock rather than your `True` β€” truthy, so the branch is taken regardless. The test therefore passes for the wrong reason and would keep passing if the `is_enabled_for` guard were removed entirely. It also leaves an un-awaited coroutine behind, since `AsyncMock` children are async by default.
   
   Setting `is_enabled_for` as a plain `MagicMock(return_value=True)` fixes both, and a companion case with it returning `False` would pin the guard down properly.
   
   ---
   Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting



##########
airflow-core/tests/unit/jobs/test_triggerer_job.py:
##########
@@ -2589,3 +2589,87 @@ async def _drive():
     trigger_id, _event, seq = events[0]
     assert trigger_id == 1
     assert seq is None
+
+
+@pytest.mark.asyncio
+async def test_trigger_event_payload_not_logged_at_info(cap_structlog):
+    """Ensure the full event payload is not logged at INFO level."""
+    runner = TriggerRunner()
+    runner.triggers = {
+        1: {
+            "task": MagicMock(spec=asyncio.Task),
+            "is_watcher": False,
+            "name": "test_dag/run_id/test_task/0/1",
+            "events": 0,
+        }
+    }
+
+    mock_trigger = MagicMock(spec=BaseTrigger)
+    mock_trigger.task_instance = MagicMock()
+    mock_trigger.task_instance.map_index = -1
+
+    payload = {"api_response": {"token": "s3cr3t-api-k3y", "user_id": 42}}
+
+    async def fake_run():
+        yield TriggerEvent(payload)
+
+    mock_trigger.run = fake_run
+
+    mock_trigger.cleanup = AsyncMock()
+
+    task = asyncio.create_task(runner.run_trigger(1, mock_trigger))
+    await task
+
+    assert any(log["event"] == "Trigger fired event" for log in cap_structlog), (
+        "Expected a 'Trigger fired event' log entry"
+    )
+    info_logs = [log for log in cap_structlog if log.get("log_level") == "info"]
+
+    for _key, value in payload.items():
+        assert not any(str(value) in str(log) for log in info_logs), (
+            "payload value must not appear in INFO-level logs"
+        )
+
+
+@pytest.mark.asyncio
+async def test_trigger_event_payload_available_at_debug(cap_structlog):
+    runner = TriggerRunner()
+    runner.triggers = {
+        1: {
+            "task": MagicMock(spec=asyncio.Task),
+            "is_watcher": False,
+            "name": "test_dag/run_id/test_task/0/1",
+            "events": 0,
+        }
+    }
+
+    mock_trigger = MagicMock(spec=BaseTrigger)
+    mock_trigger.task_instance = MagicMock()
+    mock_trigger.task_instance.map_index = -1
+
+    payload = {"api_response": {"token": "s3cr3t-api-k3y", "user_id": 42}}
+
+    async def fake_run():
+        yield TriggerEvent(payload)
+
+    mock_trigger.run = fake_run
+
+    mock_log = AsyncMock()
+    mock_log.isEnabledFor = MagicMock(return_value=True)
+    runner.log = mock_log
+
+    mock_trigger.cleanup = AsyncMock()
+
+    task = asyncio.create_task(runner.run_trigger(1, mock_trigger))
+    await task
+
+    # ainfo must not be called with result
+    for call in mock_log.ainfo.call_args_list:
+        assert "result" not in call.kwargs, "Payload must not appear in ainfo call"
+
+    # adebug must be called with full payload as 'result'
+    print(mock_log.adebug.call_args.kwargs.get("result"))

Review Comment:
   Leftover debug `print` β€” please drop.
   
   ---
   Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting



##########
airflow-core/src/airflow/jobs/triggerer_job_runner.py:
##########
@@ -1554,9 +1554,15 @@ async def run_trigger(
                     event_stream = trigger.run()
 
                 async for event in event_stream:
-                    await self.log.ainfo(
-                        "Trigger fired event", name=self.triggers[trigger_id]["name"], result=event
-                    )
+                    # Avoid logging the full payload at INFO β€” it may contain sensitive data and
+                    # inflate log storage on every execution. DEBUG is used instead so developers
+                    # can still inspect the payload when needed without.

Review Comment:
   The comment ends mid-sentence: "…can still inspect the payload when needed without."
   
   While you're here: `adebug` already no-ops when DEBUG isn't enabled, so the `is_enabled_for` guard only saves building the kwargs. If that isn't a measured win, calling `adebug` unconditionally would be simpler and would remove the branch the test above is struggling to cover.
   
   ---
   Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 merged PR #67244:
URL: https://github.com/apache/airflow/pull/67244


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-5656201254

   Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-5656199844

   Welcome to Apache Airflow, well done!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-4529640464

   @Feudjo A few things need addressing before review β€” see our [Pull Request quality criteria](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-quality-criteria).
   
   - :x: **CI fails**: `CI image checks / Static checks` (and possibly other checks β€” see the [Checks tab](https://github.com/apache/airflow/pull/67244/checks) for the full list).
   
   No rush.
   
   ---
   
   _Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer β€” a real person β€” will take the next look at your PR. We use this [two-stage triage process](https://github.com/apache/airflow/blob/main/contributing-docs/25_maintainer_pr_triage.md#why-the-first-pass-is-automated) so that our maintainers' limited time is spent where it matters most: the conversation with you._
   
   ---
   Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "Feudjo (via GitHub)" <gi...@apache.org>.
Feudjo commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-5384327093

   Hi @potiuk,
   just following up, I addressed the requested changes.
   Could you take another look when you get a chance?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-4582718081

   Can you add test and enable opportunity to print the event in debug mode (can  be protected with ifs to avoid unnecessary serialization). Ideally also add some tests mocking the behaviour to avoid regression and make a comment in the code explaining why we are doing it 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "Feudjo (via GitHub)" <gi...@apache.org>.
Feudjo commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-4644224904

   > Can you add test and enable opportunity to print the event in debug mode (can be protected with ifs to avoid unnecessary serialization). Ideally also add some tests mocking the behaviour to avoid regression and make a comment in the code explaining why we are doing it
   
   @potiuk 
   Thanks for the review. I've addressed the comments:
   
   - Printing log in debug mode
   - Added unit test.
   
   Could you please take another look when you have time?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-4499333123

   Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our [Contributors' Guide](https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (ruff, mypy and type annotations). Our [prek-hooks]( https://github.com/apache/airflow/blob/main/contributing-docs/08_static_code_checks.rst#prerequisites-for-prek-hooks) will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/airflow-core/docs/howto/custom-operator.rst) Consider adding an example Dag that shows how users should use it.
   - Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
   - Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
   - Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#coding-style-and-best-practices).
   - Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
   Apache Airflow is a community-driven project and together we are making it better πŸš€.
   In case of doubts contact the developers at:
   Mailing List: dev@airflow.apache.org
   Slack: https://s.apache.org/airflow-slack
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "Feudjo (via GitHub)" <gi...@apache.org>.
Feudjo commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-4582426089

   @potiuk ready for review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "Feudjo (via GitHub)" <gi...@apache.org>.
Feudjo commented on code in PR #67244:
URL: https://github.com/apache/airflow/pull/67244#discussion_r3711175991


##########
airflow-core/src/airflow/jobs/triggerer_job_runner.py:
##########
@@ -1554,9 +1554,15 @@ async def run_trigger(
                     event_stream = trigger.run()
 
                 async for event in event_stream:
-                    await self.log.ainfo(
-                        "Trigger fired event", name=self.triggers[trigger_id]["name"], result=event
-                    )
+                    # Avoid logging the full payload at INFO β€” it may contain sensitive data and
+                    # inflate log storage on every execution. DEBUG is used instead so developers
+                    # can still inspect the payload when needed without.

Review Comment:
   Good point β€” I'll remove the is_enabled_for guard entirely. Since result=event is just passing an existing variable with no expensive computation. Removing the guard  makes both the production code and the test much simpler. Updated accordingly.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: remove trigger event payload from INFO log in run_trigger() [airflow]

Posted by "Feudjo (via GitHub)" <gi...@apache.org>.
Feudjo commented on PR #67244:
URL: https://github.com/apache/airflow/pull/67244#issuecomment-4827244108

   Hi @potiuk,
   All CI checks are passing. Could you please take another look when you have a moment? Thank you!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org