You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "shahar1 (via GitHub)" <gi...@apache.org> on 2026/09/12 17:53:50 UTC
[PR] Allow template-field copies into StartTriggerArgs in operator __init__ [airflow]
shahar1 opened a new pull request, #73040:
URL: https://github.com/apache/airflow/pull/73040
`validate-operators-init` flags every non-sanctioned read of a template field in `__init__`, on the premise that such a read acts on the un-rendered Jinja expression and belongs in `execute()`. `DataprocSubmitJobOperator` is flagged for six reads that only copy template fields verbatim into `start_trigger_args.trigger_kwargs`. With `start_from_trigger`, the scheduler hands those kwargs straight to the triggerer, which renders the operator template fields itself (`BaseTrigger.task_instance` setter in `airflow-core/src/airflow/triggers/base.py`), so copying them un-rendered at construct time is the intended data flow and there is nothing to move. The entry cannot be cleared by an operator change without hiding the reads from the hook (see the discussion on #70296 from 30 July and the approach in #72083).
This teaches the hook that verbatim reads inside `StartTriggerArgs(...)` or `dataclasses.replace(self.start_trigger_args, ...)` are sanctioned (one level into `{...}` / `dict(...)`; a transformation inside them is still flagged), and removes the `DataprocSubmitJobOperator` exemption. The operator itself is unchanged. The same narrowing approach was used for argument-provision checks in #70505.
Verified locally: `prek run validate-operators-init --all-files` passes with the change and fails on the six Dataproc reads with the hook change stashed and the exemption removed; `uv run --project scripts pytest scripts/tests/ci/prek/test_validate_operators_init.py` passes (53 tests, 4 new cases).
related: #70296
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Fable 5.1)
Generated-by: Claude Code (Fable 5.1) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Allow template-field copies into StartTriggerArgs in operator __init__ [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 merged PR #73040:
URL: https://github.com/apache/airflow/pull/73040
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Allow template-field copies into StartTriggerArgs in operator __init__ [airflow]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on code in PR #73040:
URL: https://github.com/apache/airflow/pull/73040#discussion_r4002487225
##########
scripts/ci/prek/validate_operators_init.py:
##########
@@ -345,6 +352,55 @@ def mark(value: ast.expr | None, field: str) -> None:
return sanctioned
+def _is_start_trigger_args_assignment(target: ast.expr, value: ast.Call) -> bool:
+ """
+ Check whether an assignment rebuilds ``self.start_trigger_args``.
+
+ Matches ``self.start_trigger_args = StartTriggerArgs(...)`` and
+ ``self.start_trigger_args = dataclasses.replace(self.start_trigger_args, ...)``. Aliased
+ imports and positional ``StartTriggerArgs`` arguments are deliberately not matched.
+
+ :param target: The assignment target.
+ :param value: The assigned call.
+ :return: True if the assignment constructs or copies ``StartTriggerArgs``.
+ """
+ if not (isinstance(target, ast.Attribute) and _target_name(target) == "start_trigger_args"):
+ return False
+ name = _resolve_base_name(value.func)
Review Comment:
_resolve_base_name returns base.attr. factory.StartTriggerArgs(...) is treated as the constructor. helper.replace(self.start_trigger_args, ...) is treated as dataclasses.replace. Line 399: helper.dict(...) is treated as builtin dict.
Those callables can consume or transform an unrendered template value, which is the hole this hook exists to close. Match the precise forms: a Name StartTriggerArgs, dataclasses.replace / replace imported from dataclasses, builtin dict. Add rejection tests for the three spoofs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Allow template-field copies into StartTriggerArgs in operator __init__ [airflow]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on code in PR #73040:
URL: https://github.com/apache/airflow/pull/73040#discussion_r4002487225
##########
scripts/ci/prek/validate_operators_init.py:
##########
@@ -345,6 +352,55 @@ def mark(value: ast.expr | None, field: str) -> None:
return sanctioned
+def _is_start_trigger_args_assignment(target: ast.expr, value: ast.Call) -> bool:
+ """
+ Check whether an assignment rebuilds ``self.start_trigger_args``.
+
+ Matches ``self.start_trigger_args = StartTriggerArgs(...)`` and
+ ``self.start_trigger_args = dataclasses.replace(self.start_trigger_args, ...)``. Aliased
+ imports and positional ``StartTriggerArgs`` arguments are deliberately not matched.
+
+ :param target: The assignment target.
+ :param value: The assigned call.
+ :return: True if the assignment constructs or copies ``StartTriggerArgs``.
+ """
+ if not (isinstance(target, ast.Attribute) and _target_name(target) == "start_trigger_args"):
+ return False
+ name = _resolve_base_name(value.func)
Review Comment:
_resolve_base_name returns base.attr. factory.StartTriggerArgs(...) is treated as the constructor. helper.replace(self.start_trigger_args, ...) is treated as dataclasses.replace. Line 399: helper.dict(...) is treated as builtin dict.
Those callables can consume or transform an unrendered template value, which is the hole this hook exists to close. Match the precise forms: a Name StartTriggerArgs, dataclasses.replace / replace imported from dataclasses, builtin dict. Add rejection tests for the three spoofs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Allow template-field copies into StartTriggerArgs in operator __init__ [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on code in PR #73040:
URL: https://github.com/apache/airflow/pull/73040#discussion_r4006703249
##########
scripts/ci/prek/validate_operators_init.py:
##########
@@ -345,6 +352,55 @@ def mark(value: ast.expr | None, field: str) -> None:
return sanctioned
+def _is_start_trigger_args_assignment(target: ast.expr, value: ast.Call) -> bool:
+ """
+ Check whether an assignment rebuilds ``self.start_trigger_args``.
+
+ Matches ``self.start_trigger_args = StartTriggerArgs(...)`` and
+ ``self.start_trigger_args = dataclasses.replace(self.start_trigger_args, ...)``. Aliased
+ imports and positional ``StartTriggerArgs`` arguments are deliberately not matched.
+
+ :param target: The assignment target.
+ :param value: The assigned call.
+ :return: True if the assignment constructs or copies ``StartTriggerArgs``.
+ """
+ if not (isinstance(target, ast.Attribute) and _target_name(target) == "start_trigger_args"):
+ return False
+ name = _resolve_base_name(value.func)
Review Comment:
Handled, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org