You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Shivam Rastogi <sh...@gmail.com> on 2026/05/06 22:45:41 UTC
Introducing the TypeScript Task SDK (initial PR - 66289)
Hi everyone,
Following the Go SDK pattern adapted for Node's runtime, I've opened an
initial PR for a Node.js / TypeScript Task SDK.
*PR Link:* https://github.com/apache/airflow/pull/66289
I would appreciate feedback from reviewers with node experience on the
package layout, public APIs, and runtime design.
*The scope of this PR is minimal:*
- Two public APIs: registerTask + startWorker.
-
Typed Edge and Execution API clients with retry, heartbeat circuit
breaker, and SIGTERM drain.
-
Workload→execution token swap (recently introduced in PR #60108).
-
Unit and local integration tests included.
*Deferred to follow-ups:* XCom / Variables / Connections, log forwarding,
and Coordinator integration.
I'm committed to maintaining this SDK long term and shipping the follow-up
scope.
Regards,
Shivam Rastogi
Re: Introducing the TypeScript Task SDK (initial PR - 66289)
Posted by "Zhe-You(Jason) Liu" <ja...@apache.org>.
Hi Shivam,
Thank you for your excellent work on the TypeScript SDK supporting the
Coordinator interface.
I will take a closer look once you finish.
I really appreciate your effort!
Best,
Jason
On Sun, May 10, 2026 at 1:27 PM Shivam Rastogi <sh...@gmail.com>
wrote:
> Yes, I've been in touch with Tzu-ping and Jason about the Coordinator work.
> The current PR targets the Edge path (same interface as the existing go
> SDK), and I have already started working on the Coordinator integration. My
> goal is to support both models. I will demo both on the dev call soon.
>
> Regards,
> Shivam Rastogi
>
> On Thu, 7 May 2026 at 21:43, Rahul Vats <ra...@gmail.com> wrote:
>
> > Thanks Shivam for taking this up. Yeah totally agree with what Jens
> > suggested we should use same interface for all coming SDK's if possible.
> >
> > Regards,
> > Rahul Vats
> >
> > On Fri, 8 May 2026 at 02:15, Jens Scheffler <js...@apache.org> wrote:
> >
> > > Hi Shivam,
> > >
> > > not that I'd be a TS user or expert but sounds promising!
> > >
> > > As there is in parallel a discussion with Java SDK which changes the
> > > structures to use a Coordinator interface compared to the existing Go
> > > SDK.... which probably would be the future general interface being
> > > independent from Edge Worker API. Have you seen the draft PRs and would
> > > you consider the approach for Java SDK would be also suitable for TS?
> > > That opens more doors to execute TS tasks and even potentially let TS
> > > being a language to define Dags. But of course with increased
> complexity.
> > >
> > > Jens
> > >
> > > On 07.05.26 00:45, Shivam Rastogi wrote:
> > > > Hi everyone,
> > > >
> > > > Following the Go SDK pattern adapted for Node's runtime, I've opened
> an
> > > > initial PR for a Node.js / TypeScript Task SDK.
> > > >
> > > > *PR Link:* https://github.com/apache/airflow/pull/66289
> > > >
> > > > I would appreciate feedback from reviewers with node experience on
> the
> > > > package layout, public APIs, and runtime design.
> > > >
> > > >
> > > > *The scope of this PR is minimal:*
> > > >
> > > > - Two public APIs: registerTask + startWorker.
> > > > -
> > > >
> > > > Typed Edge and Execution API clients with retry, heartbeat
> circuit
> > > > breaker, and SIGTERM drain.
> > > > -
> > > >
> > > > Workload→execution token swap (recently introduced in PR #60108).
> > > > -
> > > >
> > > > Unit and local integration tests included.
> > > >
> > > > *Deferred to follow-ups:* XCom / Variables / Connections, log
> > forwarding,
> > > > and Coordinator integration.
> > > >
> > > > I'm committed to maintaining this SDK long term and shipping the
> > > follow-up
> > > > scope.
> > > >
> > > > Regards,
> > > >
> > > > Shivam Rastogi
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > > For additional commands, e-mail: dev-help@airflow.apache.org
> > >
> > >
> >
>
Re: Introducing the TypeScript Task SDK (initial PR - 66289)
Posted by Shivam Rastogi <sh...@gmail.com>.
Yes, I've been in touch with Tzu-ping and Jason about the Coordinator work.
The current PR targets the Edge path (same interface as the existing go
SDK), and I have already started working on the Coordinator integration. My
goal is to support both models. I will demo both on the dev call soon.
Regards,
Shivam Rastogi
On Thu, 7 May 2026 at 21:43, Rahul Vats <ra...@gmail.com> wrote:
> Thanks Shivam for taking this up. Yeah totally agree with what Jens
> suggested we should use same interface for all coming SDK's if possible.
>
> Regards,
> Rahul Vats
>
> On Fri, 8 May 2026 at 02:15, Jens Scheffler <js...@apache.org> wrote:
>
> > Hi Shivam,
> >
> > not that I'd be a TS user or expert but sounds promising!
> >
> > As there is in parallel a discussion with Java SDK which changes the
> > structures to use a Coordinator interface compared to the existing Go
> > SDK.... which probably would be the future general interface being
> > independent from Edge Worker API. Have you seen the draft PRs and would
> > you consider the approach for Java SDK would be also suitable for TS?
> > That opens more doors to execute TS tasks and even potentially let TS
> > being a language to define Dags. But of course with increased complexity.
> >
> > Jens
> >
> > On 07.05.26 00:45, Shivam Rastogi wrote:
> > > Hi everyone,
> > >
> > > Following the Go SDK pattern adapted for Node's runtime, I've opened an
> > > initial PR for a Node.js / TypeScript Task SDK.
> > >
> > > *PR Link:* https://github.com/apache/airflow/pull/66289
> > >
> > > I would appreciate feedback from reviewers with node experience on the
> > > package layout, public APIs, and runtime design.
> > >
> > >
> > > *The scope of this PR is minimal:*
> > >
> > > - Two public APIs: registerTask + startWorker.
> > > -
> > >
> > > Typed Edge and Execution API clients with retry, heartbeat circuit
> > > breaker, and SIGTERM drain.
> > > -
> > >
> > > Workload→execution token swap (recently introduced in PR #60108).
> > > -
> > >
> > > Unit and local integration tests included.
> > >
> > > *Deferred to follow-ups:* XCom / Variables / Connections, log
> forwarding,
> > > and Coordinator integration.
> > >
> > > I'm committed to maintaining this SDK long term and shipping the
> > follow-up
> > > scope.
> > >
> > > Regards,
> > >
> > > Shivam Rastogi
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> > For additional commands, e-mail: dev-help@airflow.apache.org
> >
> >
>
Re: Introducing the TypeScript Task SDK (initial PR - 66289)
Posted by Rahul Vats <ra...@gmail.com>.
Thanks Shivam for taking this up. Yeah totally agree with what Jens
suggested we should use same interface for all coming SDK's if possible.
Regards,
Rahul Vats
On Fri, 8 May 2026 at 02:15, Jens Scheffler <js...@apache.org> wrote:
> Hi Shivam,
>
> not that I'd be a TS user or expert but sounds promising!
>
> As there is in parallel a discussion with Java SDK which changes the
> structures to use a Coordinator interface compared to the existing Go
> SDK.... which probably would be the future general interface being
> independent from Edge Worker API. Have you seen the draft PRs and would
> you consider the approach for Java SDK would be also suitable for TS?
> That opens more doors to execute TS tasks and even potentially let TS
> being a language to define Dags. But of course with increased complexity.
>
> Jens
>
> On 07.05.26 00:45, Shivam Rastogi wrote:
> > Hi everyone,
> >
> > Following the Go SDK pattern adapted for Node's runtime, I've opened an
> > initial PR for a Node.js / TypeScript Task SDK.
> >
> > *PR Link:* https://github.com/apache/airflow/pull/66289
> >
> > I would appreciate feedback from reviewers with node experience on the
> > package layout, public APIs, and runtime design.
> >
> >
> > *The scope of this PR is minimal:*
> >
> > - Two public APIs: registerTask + startWorker.
> > -
> >
> > Typed Edge and Execution API clients with retry, heartbeat circuit
> > breaker, and SIGTERM drain.
> > -
> >
> > Workload→execution token swap (recently introduced in PR #60108).
> > -
> >
> > Unit and local integration tests included.
> >
> > *Deferred to follow-ups:* XCom / Variables / Connections, log forwarding,
> > and Coordinator integration.
> >
> > I'm committed to maintaining this SDK long term and shipping the
> follow-up
> > scope.
> >
> > Regards,
> >
> > Shivam Rastogi
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
> For additional commands, e-mail: dev-help@airflow.apache.org
>
>
Re: Introducing the TypeScript Task SDK (initial PR - 66289)
Posted by Jens Scheffler <js...@apache.org>.
Hi Shivam,
not that I'd be a TS user or expert but sounds promising!
As there is in parallel a discussion with Java SDK which changes the
structures to use a Coordinator interface compared to the existing Go
SDK.... which probably would be the future general interface being
independent from Edge Worker API. Have you seen the draft PRs and would
you consider the approach for Java SDK would be also suitable for TS?
That opens more doors to execute TS tasks and even potentially let TS
being a language to define Dags. But of course with increased complexity.
Jens
On 07.05.26 00:45, Shivam Rastogi wrote:
> Hi everyone,
>
> Following the Go SDK pattern adapted for Node's runtime, I've opened an
> initial PR for a Node.js / TypeScript Task SDK.
>
> *PR Link:* https://github.com/apache/airflow/pull/66289
>
> I would appreciate feedback from reviewers with node experience on the
> package layout, public APIs, and runtime design.
>
>
> *The scope of this PR is minimal:*
>
> - Two public APIs: registerTask + startWorker.
> -
>
> Typed Edge and Execution API clients with retry, heartbeat circuit
> breaker, and SIGTERM drain.
> -
>
> Workload→execution token swap (recently introduced in PR #60108).
> -
>
> Unit and local integration tests included.
>
> *Deferred to follow-ups:* XCom / Variables / Connections, log forwarding,
> and Coordinator integration.
>
> I'm committed to maintaining this SDK long term and shipping the follow-up
> scope.
>
> Regards,
>
> Shivam Rastogi
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@airflow.apache.org
For additional commands, e-mail: dev-help@airflow.apache.org