You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jason810496 (via GitHub)" <gi...@apache.org> on 2026/10/07 16:13:32 UTC
[PR] Go SDK: run TriggerDagRun tasks in the Go runtime [airflow]
jason810496 opened a new pull request, #74417:
URL: https://github.com/apache/airflow/pull/74417
- **Depends on https://github.com/apache/airflow/pull/74342 get merged first** (its commits are cherry-picked here; only the last 5 commits are new).
- **Diff for early review**: https://github.com/jason810496/airflow/compare/ae8503d15e6e8a5daa52d1a2d9a5a3096cd6d7ae...feature/go-sdk/trigger-dag-run-runtime
- related: #74411 (e2e), #74414 (the same options for the TypeScript SDK)
## Why
A Go `airflow.TriggerDagRun` task was meant to run as `TriggerDagRunOperator` on a Python worker, but a Python worker refuses a task whose Dag file is a Go bundle, so the task could never run. As ADR-0008 decides, each Lang SDK runtime runs the trigger task itself, as the TypeScript SDK already does.
## How
- The Go runtime finds a trigger task of a registered Dag and runs it the way `TriggerDagRunOperator` does:
- checks whether the target Dag is paused, when asked;
- pushes the run link, triggers the run, and pushes `trigger_run_id`;
- skips or fails without retry when the run already exists;
- waits for the run by polling, or defers to the standard provider's `DagStateTrigger` in the Python triggerer when `Deferrable` is set, then resumes in the Go runtime.
- A trigger task takes `DagSpec.Queue` unless its `TaskSpec` sets one, so it is routed to the Go runtime.
- Like the TypeScript SDK, the arguments are sent as written, with no Jinja rendering.
```go
dag := airflow.Dag("go_native_trigger", airflow.DagSpec{Queue: "golang"})
dag.Task(airflow.TriggerDagRun(airflow.TriggerDagRunSpec{
DagID: "report",
LogicalDate: time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC), // was a string; zero means now
Conf: map[string]any{"triggered_by": "go_native_trigger"},
WaitForCompletion: true,
Deferrable: &deferrable,
}), airflow.TaskSpec{TaskID: "trigger_report"})
```
## What
- `pkg/execution`: the trigger runner, and the `TriggerDagRun`, `GetDagRunState` and `GetDag` calls it makes.
- `TriggerDagRunSpec.LogicalDate` is a `time.Time`; the shared conformance case and `go_native.json` follow.
- The trigger task takes the Dag's queue; `serialize.go` and the generated spec docs follow.
- Docs on `TriggerDagRun`: who runs it, its queue, `Deferrable`, and no templating.
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes, with help of Claude Code Opus 5.5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Go SDK: run TriggerDagRun tasks in the Go runtime [airflow]
Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on code in PR #74417:
URL: https://github.com/apache/airflow/pull/74417#discussion_r4218114487
##########
go-sdk/airflow/trigger_dag_run.go:
##########
@@ -36,21 +38,23 @@ import (
// leaves its parameter at the Python default. PokeInterval and Deferrable are pointers so that a
// pointer to 0 or false can set that value instead of leaving the default.
//
-// DagID, RunID, LogicalDate and the values in Conf are templated. They can hold Jinja such as
-// "{{ ds }}", which Airflow renders when the task runs.
+// The task does not render templates. DagID, RunID, Note and the values in Conf are sent as they
+// are, so a string such as "{{ ds }}" reaches the new Dag run unchanged. The task does not offer
+// openlineage_inject_parent_info, so it adds nothing to Conf.
type TriggerDagRunSpec struct {
// DagID is the dag_id of the Dag to trigger. It is required.
DagID string
- // RunID is the run_id of the new Dag run. When RunID is empty, Airflow generates one.
+ // RunID is the run_id of the new Dag run. When RunID is empty, the task names the run after
Review Comment:
```go
if spec.DagID == "" {
return TriggerDagRunSpec{}, errors.New("airflow.TriggerDagRunSpec has no DagID")
}
for field, value := range map[string]string{"DagID": spec.DagID, "RunID": spec.RunID} {
if strings.Contains(value, "{{") {
return TriggerDagRunSpec{}, fmt.Errorf(
"airflow.TriggerDagRunSpec.%s is %q; the task does not render templates", field, value,
)
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Go SDK: run TriggerDagRun tasks in the Go runtime [airflow]
Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on code in PR #74417:
URL: https://github.com/apache/airflow/pull/74417#discussion_r4218105133
##########
go-sdk/airflow/trigger_dag_run.go:
##########
@@ -36,21 +38,23 @@ import (
// leaves its parameter at the Python default. PokeInterval and Deferrable are pointers so that a
// pointer to 0 or false can set that value instead of leaving the default.
//
-// DagID, RunID, LogicalDate and the values in Conf are templated. They can hold Jinja such as
-// "{{ ds }}", which Airflow renders when the task runs.
+// The task does not render templates. DagID, RunID, Note and the values in Conf are sent as they
+// are, so a string such as "{{ ds }}" reaches the new Dag run unchanged. The task does not offer
+// openlineage_inject_parent_info, so it adds nothing to Conf.
type TriggerDagRunSpec struct {
// DagID is the dag_id of the Dag to trigger. It is required.
DagID string
- // RunID is the run_id of the new Dag run. When RunID is empty, Airflow generates one.
+ // RunID is the run_id of the new Dag run. When RunID is empty, the task names the run after
Review Comment:
**Replacement for L125–L127** (apply manually — lines not in the diff):
Optional: since nothing gets rendered, RunID: "triggered_{{ run_id }}" (an easy habit to bring over from Python) only fails at runtime: the run_id regex check returns a 400 on the first run, and a templated DagID gets a
404. {{ can never be valid in a dag_id, and the default allowed_run_id_pattern rejects it in a run_id, so rejecting it here would surface the mistake when the Dag is parsed. Note and Conf should stay unguarded, since
{{ is legitimate there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Go SDK: run TriggerDagRun tasks in the Go runtime [airflow]
Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 merged PR #74417:
URL: https://github.com/apache/airflow/pull/74417
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org