Re: [D] Proposal: Add Separate Amber Worker Container in the Computing-Unit Pod and Route All UDFs There [texera]

["mengw15 (via GitHub)" <gi...@apache.org>]

GitHub user mengw15 edited a discussion: Proposal: Add Separate Amber Worker Container in the Computing-Unit Pod and Route All UDFs There

## Summary

Currently, a computing unit is a single Pod with one Container running the Amber Coordinator (master). We propose adding a separate Container in the same Pod that runs an Amber Driver (worker). Operator scheduling will be updated so all UDF operators execute on the Driver container.

## Motivation

This design leverages Akka’s distribution features 

## Design (high level)

Separate Driver Container in the same Pod
Extend the Pod spec to start one new container for an Amber Driver.

Scheduling change
Update the scheduler so that all UDF operators are placed on the Driver container.
Non-UDF operators keep current placement (initially unchanged).

<img width="4555" height="3380" alt="computing-unit-pod-proposal drawio" src="https://github.com/user-attachments/assets/9475c647-4c46-4a0c-9f97-081c3f501c9e" />


GitHub link: https://github.com/apache/texera/discussions/4021

----
This is an automatically sent email for notifications@texera.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@texera.apache.org