You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@texera.apache.org by gi...@apache.org on 2026/06/28 09:07:47 UTC
(texera) 01/01: feat(infra): add `bin/local-dev.sh` dev-stack manager + TUI (#5961)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-5961-01b9254bea697b936724c46c20cdd353f00ba226
in repository https://gitbox.apache.org/repos/asf/texera.git
commit de6c31ac48ddf829b00c78b1d4f45b826236ff9d
Author: Yicong Huang <17...@users.noreply.github.com>
AuthorDate: Sun Jun 28 02:07:25 2026 -0700
feat(infra): add `bin/local-dev.sh` dev-stack manager + TUI (#5961)
### What changes were proposed in this PR?
- Introduce `bin/local-dev.sh` to build/start/stop/status/logs for
docker infra + JVM services + agent-service + frontend.
- Add `bin/local-dev-tui.py` (Textual) to provide a live dashboard,
command prompt, and log tailing by shelling out to bin/local-dev.sh.
- Add tests and CI plumbing (labeler + required-checks stack + build
workflow job) to run shell smoke tests and pytest unit tests for the
tooling.
#### Demo 1: Starting Texera Local Dev
https://github.com/user-attachments/assets/beff0dd7-5afa-4de7-8c5c-72f8ed58bfe7
#### Demo 2: Auto-updating Changed Services
https://github.com/user-attachments/assets/c84cfe12-2b9b-4e80-8651-b1375a6a82fb
#### Demo 3: Tailing Service Logs
https://github.com/user-attachments/assets/d53b35f4-c821-4fcc-bd11-5dd696946895
#### What you can do
| Command | What it does |
| --------------------------------------------- |
-----------------------------------------------------------------------
|
| `bin/local-dev.sh` / `… status` | One-shot dashboard (state · uptime ·
CPU% · mem · PID per service). |
| `bin/local-dev.sh -i` | Live Textual TUI of the same dashboard, with a
command prompt. |
| `bin/local-dev.sh up` | Build dirty JVMs, install yarn/bun deps if
locks moved, bring all 14 services up. Bootstraps the postgres schema on
stale volumes. |
| `bin/local-dev.sh down` | Stop everything, reverse order. |
| `bin/local-dev.sh auto` | Detect which services' source changed (`★`)
and rebuild + bounce only those. |
| `bin/local-dev.sh <svc>` | Rebuild that one service and bounce it. |
| `bin/local-dev.sh logs <svc>` | Tail a service's log. |
#### Key behaviours
- **Dirty detection** drives `auto` / the `★` indicator: SHA-1 of source
bytes against a per-service stamp, walking the per-service transitive
sbt dependency graph parsed from `build.sbt`. Editing
`common/workflow-operator/src` doesn't trigger services that don't
depend on it.
- **Dynamic everything** — texera version, sbt graph, host LAN IP (for
the Iceberg/MinIO endpoint that Lakekeeper hands back to host JVMs), JDK
17 location, Python interpreter, Node toolchain — all auto-detected at
startup with clear "what we saw / what to install" diagnostics on miss.
- **Bash 3.2 compatible** — runs on Apple's pinned `/bin/bash` 3.2 with
no re-exec.
### Any related issues, documentation, discussions?
Closes #5960.
### How was this PR tested?
- 36 pytest + 10 shell smoke under bash 3.2 / bash 5 / zsh.
- Manual cold-start on macOS: `up` brings all 14 services to healthy
from an empty state; `down` tears them down. Editing `.scala` files
marks only the dependent services dirty; `auto` rebuilds + bounces just
those (amber's two siblings always paired). `up --build / --fresh /
--skip=…` all exercised. Toolchain-missing paths print the diagnostic
env + install hint.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Anthropic, Claude Opus 4.7).
---------
Co-authored-by: Claude Opus 4.7 (1M context) <no...@anthropic.com>
---
.github/labeler.yml | 11 +
.github/workflows/build.yml | 42 +
.github/workflows/required-checks.yml | 9 +-
amber/dev-requirements.txt | 3 +
bin/local-dev-tui.py | 1651 +++++++++++++
bin/local-dev.sh | 2536 ++++++++++++++++++++
bin/local-dev/docker-compose.override.yml | 56 +
.../local-dev/tests/__init__.py | 18 -
bin/local-dev/tests/conftest.py | 44 +
bin/local-dev/tests/test_local_dev_sh.sh | 167 ++
bin/local-dev/tests/test_local_dev_tui.py | 376 +++
11 files changed, 4894 insertions(+), 19 deletions(-)
diff --git a/.github/labeler.yml b/.github/labeler.yml
index 75f20d2c33..e520c6a6df 100644
--- a/.github/labeler.yml
+++ b/.github/labeler.yml
@@ -113,6 +113,17 @@ dev:
- any-glob-to-any-file:
- 'bin/**'
+infra:
+ # Dev-loop tooling for bringing up / tailing the local stack lives under
+ # bin/local-dev*. Tracked on its own label so the matching CI stack runs
+ # on changes here without dragging the broader `dev` label (which is a
+ # no-op stack-mapping-wise) into the equation.
+ - changed-files:
+ - any-glob-to-any-file:
+ - 'bin/local-dev.sh'
+ - 'bin/local-dev-tui.py'
+ - 'bin/local-dev/**'
+
dependencies:
- changed-files:
- any-glob-to-any-file:
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index addb2d3236..b44f035552 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -60,6 +60,10 @@ on:
required: false
type: boolean
default: true
+ run_infra:
+ required: false
+ type: boolean
+ default: true
mode:
# PR (default) | nightly | release. Only "PR" passes
# --ignore-transitive-version to check_binary_deps.py.
@@ -917,3 +921,41 @@ jobs:
report_type: test_results
disable_search: true
fail_ci_if_error: false
+
+ infra:
+ # Tests for the dev-loop tooling under bin/local-dev*.
+ # Light job: bash syntax + smoke tests, plus pytest unit tests for the
+ # pure helpers in local-dev-tui.py. No docker / sbt / live stack — those
+ # belong in an e2e job we'll add when the tool is exercised more widely.
+ if: ${{ inputs.run_infra }}
+ name: ${{ format('infra{0} ({1})', inputs.job_name_suffix, matrix.os) }}
+ runs-on: ${{ matrix.os }}
+ strategy:
+ fail-fast: false
+ matrix:
+ os: [ubuntu-latest, macos-latest]
+ python-version: ["3.12"]
+ steps:
+ - name: Checkout Texera
+ uses: actions/checkout@v5
+ with:
+ ref: ${{ inputs.checkout_ref || github.sha }}
+ fetch-depth: 0
+ - name: Prepare backport workspace
+ if: ${{ inputs.backport_target_branch != '' }}
+ run: bash ./.github/scripts/prepare-backport-checkout.sh "${{ inputs.backport_target_branch }}" "${{ inputs.backport_commit_range }}"
+ - name: Setup Python
+ uses: actions/setup-python@v6
+ with:
+ python-version: ${{ matrix.python-version }}
+ - name: Install Python test deps
+ # amber/dev-requirements.txt pins both `pytest` and `textual` — the
+ # latter so a fresh dev who follows the AGENTS.md venv-setup
+ # instructions ends up with the TUI working.
+ run: |
+ python -m pip install --upgrade pip
+ python -m pip install -r amber/dev-requirements.txt
+ - name: Run shell smoke tests (bash)
+ run: bash bin/local-dev/tests/test_local_dev_sh.sh
+ - name: Run Python unit tests
+ run: python -m pytest bin/local-dev/tests/ -v --tb=short
diff --git a/.github/workflows/required-checks.yml b/.github/workflows/required-checks.yml
index 2d7012fe33..d854be3f17 100644
--- a/.github/workflows/required-checks.yml
+++ b/.github/workflows/required-checks.yml
@@ -65,6 +65,7 @@ jobs:
run_platform: ${{ steps.decide.outputs.run_platform }}
run_pyamber: ${{ steps.decide.outputs.run_pyamber }}
run_agent_service: ${{ steps.decide.outputs.run_agent_service }}
+ run_infra: ${{ steps.decide.outputs.run_infra }}
backport_targets: ${{ steps.decide.outputs.backport_targets }}
steps:
- name: Wait for Pull Request Labeler
@@ -153,7 +154,8 @@ jobs:
"agent-service": ["agent-service"],
common: ["amber", "amber-integration", "platform"],
"ddl-change": ["amber", "amber-integration", "platform"],
- ci: ["frontend", "amber", "amber-integration", "platform", "pyamber", "agent-service"],
+ infra: ["infra"],
+ ci: ["frontend", "amber", "amber-integration", "platform", "pyamber", "agent-service", "infra"],
};
let runFrontend = true;
@@ -162,6 +164,7 @@ jobs:
let runPlatform = true;
let runPyamber = true;
let runAgentService = true;
+ let runInfra = true;
if (eventName === "pull_request") {
const stacks = new Set();
@@ -176,6 +179,7 @@ jobs:
runPlatform = stacks.has("platform");
runPyamber = stacks.has("pyamber");
runAgentService = stacks.has("agent-service");
+ runInfra = stacks.has("infra");
core.info(
`Stacks selected by label union: ${[...stacks].sort().join(", ") || "(none)"}`
);
@@ -187,6 +191,7 @@ jobs:
core.setOutput("run_platform", runPlatform ? "true" : "false");
core.setOutput("run_pyamber", runPyamber ? "true" : "false");
core.setOutput("run_agent_service", runAgentService ? "true" : "false");
+ core.setOutput("run_infra", runInfra ? "true" : "false");
// Backport targets: all current release/* labels on the PR.
const targets = [...new Set(labels.filter((n) => /^release\/.+$/.test(n)))].sort();
@@ -242,6 +247,7 @@ jobs:
run_platform: ${{ needs.precheck.outputs.run_platform == 'true' }}
run_pyamber: ${{ needs.precheck.outputs.run_pyamber == 'true' }}
run_agent_service: ${{ needs.precheck.outputs.run_agent_service == 'true' }}
+ run_infra: ${{ needs.precheck.outputs.run_infra == 'true' }}
secrets: inherit
backport:
@@ -263,6 +269,7 @@ jobs:
run_platform: ${{ needs.precheck.outputs.run_platform == 'true' }}
run_pyamber: ${{ needs.precheck.outputs.run_pyamber == 'true' }}
run_agent_service: ${{ needs.precheck.outputs.run_agent_service == 'true' }}
+ run_infra: ${{ needs.precheck.outputs.run_infra == 'true' }}
secrets: inherit
required-checks:
diff --git a/amber/dev-requirements.txt b/amber/dev-requirements.txt
index 4ab14960cb..3ac549283d 100644
--- a/amber/dev-requirements.txt
+++ b/amber/dev-requirements.txt
@@ -32,3 +32,6 @@ pytest-cov==7.1.0
# protoc plugin for bin/python-proto-gen.sh; runtime needs only the
# base `betterproto` in requirements.txt.
betterproto[compiler]==2.0.0b7
+
+# Required by `bin/local-dev.sh -i` (the interactive Textual TUI).
+textual==8.2.7
diff --git a/bin/local-dev-tui.py b/bin/local-dev-tui.py
new file mode 100644
index 0000000000..3b294491f7
--- /dev/null
+++ b/bin/local-dev-tui.py
@@ -0,0 +1,1651 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""bin/local-dev-tui.py — Textual dashboard for the Texera local dev stack.
+
+Lives next to bin/local-dev.sh; the shell script remains the canonical engine
+(build, start, stop, status) and this TUI shells out to it for every action.
+The dashboard itself owns state polling, dirty-source detection, and the
+prompt loop. Textual handles diff rendering so the screen doesn't accrete in
+scrollback the way the old zsh `\\e[H` redraw did.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import contextlib
+import hashlib
+import os
+import re
+import shlex
+import shutil
+import subprocess
+import time
+from dataclasses import dataclass, field
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Optional
+
+from rich.text import Text
+from textual import events, work
+from textual.app import App, ComposeResult
+from textual.binding import Binding
+from textual.containers import Vertical
+from textual.reactive import reactive
+from textual.widgets import DataTable, Input, RichLog, Static
+
+# ─────────────────── Constants ───────────────────
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+STATE_DIR = Path(os.environ.get("TEXERA_LOCAL_DEV_DIR", "/tmp/texera-local-dev"))
+LOG_DIR = STATE_DIR / "logs"
+BUILD_STAMP_DIR = STATE_DIR / "build-stamps"
+# Per-service phase markers written by the shell during stop/build/start.
+# Each file holds `<phase>\t<epoch>` — we read it back in _tick_state and
+# render an animated transitional STATE if it's recent (<90s).
+PHASE_DIR = STATE_DIR / "svc-phase"
+PHASE_STALE_S = 90.0
+REPL_LOG = LOG_DIR / "repl.log"
+LOG_DIR.mkdir(parents=True, exist_ok=True)
+BUILD_STAMP_DIR.mkdir(parents=True, exist_ok=True)
+
+LOCAL_DEV_SH = REPO_ROOT / "bin" / "local-dev.sh"
+DOCKER_PROJECT = "texera-local-dev"
+
+HISTORY_FILE = STATE_DIR / "tui-history"
+MAX_HISTORY = 500
+
+SOURCE_SUFFIXES = {".scala", ".java", ".proto"}
+
+# Single source of truth for per-JVM-service source dirs is the SBT build
+# graph in build.sbt. We parse it instead of hardcoding `common/*` so that
+# adding a `lazy val NewCommon = (project in file("common/new"))` and a
+# corresponding `.dependsOn(NewCommon)` automatically flows through to
+# dirty-detection without anyone touching this file.
+
+# Matches test-scope dependsOn args we want to drop from the runtime
+# graph. `X % "test->test"`, `X % "test"`, `X % Test`. Tighter than the
+# previous "any %" check, which would have silently dropped a future
+# `.dependsOn(X % "compile->compile")` and broken dirty-detection on X.
+# No trailing `\b` — `\b` after a literal `"` doesn't match because both
+# sides are non-word chars; we anchor the end of the quoted form on the
+# closing `"` itself, and the `Test` form on (?:\b|$).
+_TEST_SCOPE_RE = re.compile(r'%\s*(?:"test(?:->[^"]*)?"|Test(?:\b|$))')
+
+
+def _parse_sbt_deps() -> dict[str, dict]:
+ """Walk `build.sbt` and return a dependency graph keyed by sbt project
+ name. Each entry is `{"path": <repo-relative dir>, "deps": [..main-scope
+ deps..]}`. Test-only `% "test->test"` deps are filtered out — they
+ can't affect a service's runtime artifact, so they don't drive dirty.
+ """
+ bs = REPO_ROOT / "build.sbt"
+ try:
+ text = bs.read_text(encoding="utf-8", errors="replace")
+ except OSError:
+ return {}
+
+ proj_re = re.compile(
+ r'^lazy\s+val\s+([A-Z][A-Za-z0-9]*)\s*=\s*\(project\s+in\s+file\("([^"]+)"\)\)',
+ re.MULTILINE,
+ )
+ matches = list(proj_re.finditer(text))
+ if not matches:
+ return {}
+
+ graph: dict[str, dict] = {}
+ for i, m in enumerate(matches):
+ name = m.group(1)
+ path = m.group(2)
+ # The "block" for this project spans from this match to the next
+ # `lazy val ... = (project in file(...))` (or end of file). All the
+ # chained `.dependsOn(...)` calls live in there.
+ end = matches[i + 1].start() if i + 1 < len(matches) else len(text)
+ block = text[m.start():end]
+
+ deps: list[str] = []
+ for arg_list in re.findall(r"\.dependsOn\(([^)]*)\)", block):
+ for arg in arg_list.split(","):
+ arg = arg.strip()
+ # Skip ONLY test-scope deps — match `% "test->test"` and
+ # `% Test` exactly. Filtering all `%`-scoped args was too
+ # broad: a future `.dependsOn(X % "compile->compile")` or
+ # `% Provided` would silently disappear from the dep
+ # graph and break dirty-detection on `X`.
+ if _TEST_SCOPE_RE.search(arg):
+ continue
+ # Strip any trailing whitespace / comments. Project refs
+ # are bare identifiers.
+ ref = arg.split()[0] if arg else ""
+ if ref and ref[0].isupper():
+ deps.append(ref)
+ graph[name] = {"path": path, "deps": deps}
+ return graph
+
+
+def _transitive_src_dirs(sbt_project: Optional[str], graph: dict[str, dict]) -> list[str]:
+ """Return source dirs for sbt_project AND every project it transitively
+ depends on. Each dir is `<project_path>/src` (the canonical sbt source
+ location). Order is stable; duplicates removed. Raises if the graph
+ is empty or `sbt_project` isn't in it — silently falling back to a
+ hardcoded common/* list would mask a real build.sbt drift."""
+ if not graph:
+ raise RuntimeError("build.sbt dependency graph is empty — parse failed")
+ if not sbt_project or sbt_project not in graph:
+ raise RuntimeError(
+ f"sbt project {sbt_project!r} not found in build.sbt graph; "
+ f"known projects: {sorted(graph.keys())}"
+ )
+ visited: set[str] = set()
+ out: list[str] = []
+
+ def walk(name: str) -> None:
+ if name in visited or name not in graph:
+ return
+ visited.add(name)
+ path = graph[name]["path"]
+ src = f"{path}/src"
+ if src not in out:
+ out.append(src)
+ for d in graph[name]["deps"]:
+ walk(d)
+ walk(sbt_project)
+ return out
+
+
+_SBT_GRAPH = _parse_sbt_deps()
+
+
+# Per-sbt-project transitive closure, memoised. Computed once on first
+# touch; subsequent dirty checks are O(1) lookup vs the prior O(deps)
+# BFS per call.
+_SBT_TRANSITIVE_CACHE: dict[str, list[str]] = {}
+
+
+def _transitive_src_dirs_cached(sbt_project: Optional[str]) -> list[str]:
+ if sbt_project is None:
+ return _transitive_src_dirs(None, _SBT_GRAPH)
+ if sbt_project not in _SBT_TRANSITIVE_CACHE:
+ _SBT_TRANSITIVE_CACHE[sbt_project] = _transitive_src_dirs(sbt_project, _SBT_GRAPH)
+ return _SBT_TRANSITIVE_CACHE[sbt_project]
+
+POLL_INTERVAL_S = 1.0 # how often to refresh service state
+DIRTY_INTERVAL_S = 2.0 # how often to recompute dirty indicators
+
+# Services whose own runtime watches the filesystem and rebuilds on source
+# change: `yarn start` runs `ng serve` (Angular dev server, hot-reload) and
+# `bun run --watch` reloads the agent-service on change. The dashboard
+# surfaces this in the SRC column so the user doesn't try to bounce them
+# unnecessarily; the ★ "dirty" indicator only flashes for them when the lock
+# file changes (i.e. an actual dep refresh is needed).
+WATCH_TYPES = {"yarn", "bun"}
+
+
+# ─────────────────── Texera version (dynamic) ───────────────────
+
+_VERSION_RE = re.compile(
+ r'^\s*ThisBuild\s*/\s*version\s*:=\s*"([^"]+)"', re.MULTILINE
+)
+
+
+def texera_version() -> str:
+ """Parse the project version from build.sbt so artifact paths track
+ whatever branch the developer is on (it's 1.3.0-incubating-SNAPSHOT on
+ main today, was 1.2.0-incubating on release/v1.2, will differ again).
+ Override with `TEXERA_VERSION` env var to bypass parsing."""
+ env = os.environ.get("TEXERA_VERSION")
+ if env:
+ return env
+ bs = REPO_ROOT / "build.sbt"
+ if not bs.exists():
+ raise RuntimeError(
+ f"build.sbt not found at {bs} — set TEXERA_VERSION to bypass"
+ )
+ m = _VERSION_RE.search(bs.read_text(errors="replace"))
+ if not m:
+ raise RuntimeError(
+ f"could not find `ThisBuild / version := \"…\"` in {bs} — "
+ f"set TEXERA_VERSION to bypass"
+ )
+ return m.group(1)
+
+
+TEXERA_VERSION = texera_version()
+
+
+# ─────────────────── Service catalog ───────────────────
+
+@dataclass
+class Service:
+ name: str
+ type: str # "docker" | "jvm" | "yarn" | "bun"
+ port: int
+ sbt_project: Optional[str] = None # for jvm
+ own_src: Optional[str] = None # for jvm
+ artifact_jar: Optional[str] = None # for jvm
+
+
+def _jvm(name: str, port: int, project: Optional[str], own_src: str) -> Service:
+ """sbt-native-packager lays the dist out as
+ `target/<artifact>-<VERSION>/lib/org.apache.texera.<artifact>-<VERSION>.jar`
+ for every subproject. amber is the exception: its sbt subproject is
+ named `amber` (not `texera-web`) and the dist goes under `amber/target/`
+ rather than the repo-level `target/`. computing-unit-master rides that
+ same amber dist as a sibling launcher — its `project` is None because
+ no separate sbt invocation produces it."""
+ is_amber_svc = name in ("texera-web", "computing-unit-master")
+ artifact = "amber" if is_amber_svc else name
+ target_prefix = "amber/" if is_amber_svc else ""
+ jar = (
+ f"{target_prefix}target/{artifact}-{TEXERA_VERSION}/lib/"
+ f"org.apache.texera.{artifact}-{TEXERA_VERSION}.jar"
+ )
+ return Service(name, "jvm", port, sbt_project=project,
+ own_src=own_src, artifact_jar=jar)
+
+
+SERVICES: list[Service] = [
+ Service("postgres", "docker", 5432),
+ Service("minio", "docker", 9000),
+ Service("lakefs", "docker", 8000),
+ Service("lakekeeper", "docker", 8181),
+ Service("litellm", "docker", 4000),
+ _jvm("config-service", 9094, "ConfigService",
+ "config-service/src"),
+ _jvm("access-control-service", 9096, "AccessControlService",
+ "access-control-service/src"),
+ _jvm("file-service", 9092, "FileService",
+ "file-service/src"),
+ _jvm("workflow-compiling-service", 9090, "WorkflowCompilingService",
+ "workflow-compiling-service/src"),
+ _jvm("computing-unit-master", 8085, None,
+ "amber/src"),
+ _jvm("computing-unit-managing-service", 8082, "ComputingUnitManagingService",
+ "computing-unit-managing-service/src"),
+ _jvm("texera-web", 8080, "WorkflowExecutionService",
+ "amber/src"),
+ Service("agent-service", "bun", 3001),
+ Service("frontend", "yarn", 4200),
+]
+
+SERVICES_BY_NAME = {s.name: s for s in SERVICES}
+
+
+# ─────────────────── Live state model ───────────────────
+
+def read_svc_phase(svc_name: str) -> Optional[str]:
+ """Return the shell-written phase (`stopping` / `building` / `starting`)
+ or None if absent / stale. Stale-after-90s guards against a crashed
+ `bin/local-dev.sh` leaving an orphan file."""
+ f = PHASE_DIR / svc_name
+ try:
+ text = f.read_text().strip()
+ except OSError:
+ return None
+ if not text:
+ return None
+ parts = text.split("\t", 1)
+ phase = parts[0]
+ try:
+ ts = float(parts[1]) if len(parts) > 1 else 0.0
+ except ValueError:
+ ts = 0.0
+ if ts and (time.time() - ts) > PHASE_STALE_S:
+ return None
+ return phase
+
+
+@dataclass
+class LiveState:
+ """Snapshot of the world this tick — what the dashboard renders."""
+ docker: dict[str, tuple[str, str]] = field(default_factory=dict) # name -> (state, status)
+ pids: dict[str, Optional[str]] = field(default_factory=dict) # name -> pid or None
+ dirty: dict[str, bool] = field(default_factory=dict)
+ mtimes: dict[str, Optional[str]] = field(default_factory=dict)
+ # Per-service resource usage (computed every poll for native services
+ # via cheap `ps`; refreshed less often for docker via `docker stats`
+ # since that call costs ~2s).
+ uptimes: dict[str, str] = field(default_factory=dict) # name -> "12m" / "—"
+ cpu_pct: dict[str, str] = field(default_factory=dict) # name -> "8.2%" / "—"
+ mem_use: dict[str, str] = field(default_factory=dict) # name -> "85M" / "—"
+ phases: dict[str, str] = field(default_factory=dict) # name -> "stopping"/"building"/"starting" or ""
+
+
+# ─────────────────── Helpers ───────────────────
+
+async def _run_capture(*argv: str) -> str:
+ proc = await asyncio.create_subprocess_exec(
+ *argv,
+ stdout=asyncio.subprocess.PIPE,
+ stderr=asyncio.subprocess.DEVNULL,
+ )
+ stdout, _ = await proc.communicate()
+ return stdout.decode(errors="replace")
+
+
+async def lsof_port_pid(port: int) -> Optional[str]:
+ out = await _run_capture("lsof", "-nP", f"-iTCP:{port}", "-sTCP:LISTEN", "-t")
+ out = out.strip()
+ return out.split("\n", 1)[0] if out else None
+
+
+async def docker_ps_all() -> dict[str, tuple[str, str]]:
+ out = await _run_capture(
+ "docker", "compose", "-p", DOCKER_PROJECT, "ps", "-a",
+ "--format", "{{.Service}}|{{.State}}|{{.Status}}",
+ )
+ result: dict[str, tuple[str, str]] = {}
+ for line in out.splitlines():
+ parts = line.split("|", 2)
+ if len(parts) == 3:
+ result[parts[0]] = (parts[1], parts[2])
+ return result
+
+
+def docker_state(svc_state: str, svc_status: str) -> str:
+ """Map docker's raw state/status to the small palette the dashboard renders."""
+ if svc_state == "running":
+ if "(healthy)" in svc_status:
+ return "running"
+ if "(health: starting)" in svc_status:
+ return "starting"
+ if "(unhealthy)" in svc_status:
+ return "unhealthy"
+ return "running"
+ if svc_state == "exited":
+ return "exited" if svc_status.startswith("Exited (0)") else "failed"
+ if svc_state in ("created", "restarting", "paused", "removing"):
+ return "starting"
+ return "stopped"
+
+
+# ─────────────────── Dirty-source detection (content hash) ───────────────────
+
+# ─────────────────── Uptime / resource helpers ───────────────────
+
+def _format_uptime(secs: int) -> str:
+ """Compact duration: `12s`, `5m 23s`, `2h 14m`, `3d 4h`."""
+ if secs < 0:
+ return "—"
+ if secs < 60:
+ return f"{secs}s"
+ if secs < 3600:
+ return f"{secs // 60}m {secs % 60}s"
+ if secs < 86400:
+ return f"{secs // 3600}h {(secs % 3600) // 60}m"
+ return f"{secs // 86400}d {(secs % 86400) // 3600}h"
+
+
+def _format_bytes(n: int) -> str:
+ """Bytes → short human string (`85M`, `1.2G`, `467K`)."""
+ if n < 1024:
+ return f"{n}B"
+ if n < 1024 ** 2:
+ return f"{n // 1024}K"
+ if n < 1024 ** 3:
+ return f"{n // (1024 ** 2)}M"
+ return f"{n / (1024 ** 3):.1f}G"
+
+
+def _parse_etime(et: str) -> int:
+ """`ps -o etime` format `[[DD-]hh:]mm:ss` → seconds."""
+ et = et.strip()
+ if not et:
+ return -1
+ days = 0
+ if "-" in et:
+ d, et = et.split("-", 1)
+ days = int(d or 0)
+ parts = et.split(":")
+ try:
+ if len(parts) == 3:
+ h, m, s = (int(p or 0) for p in parts)
+ elif len(parts) == 2:
+ h = 0
+ m, s = (int(p or 0) for p in parts)
+ else:
+ h = m = 0
+ s = int(parts[0] or 0)
+ except ValueError:
+ return -1
+ return days * 86400 + h * 3600 + m * 60 + s
+
+
+def _docker_container_for(svc_name: str) -> str:
+ """Container names: most are `texera-<svc>`; `litellm` is unprefixed."""
+ if svc_name == "litellm":
+ return "litellm"
+ return f"texera-{svc_name}"
+
+
+_DOCKER_STATS_CACHE: dict[str, tuple[str, str]] = {}
+_DOCKER_STATS_TS: float = 0.0
+_DOCKER_STATS_TTL = 5.0 # seconds — docker stats --no-stream costs ~2s, don't run it more than this
+
+
+def _refresh_docker_stats() -> None:
+ """Single `docker stats --no-stream` pass; refreshes the cached
+ container → (cpu%, mem) map. Called from a background worker so the
+ main poll tick stays cheap."""
+ global _DOCKER_STATS_CACHE, _DOCKER_STATS_TS
+ try:
+ out = subprocess.run(
+ ["docker", "stats", "--no-stream",
+ "--format", "{{.Name}}|{{.CPUPerc}}|{{.MemUsage}}"],
+ capture_output=True, text=True, timeout=8,
+ ).stdout
+ except (subprocess.TimeoutExpired, FileNotFoundError, OSError):
+ return
+ fresh: dict[str, tuple[str, str]] = {}
+ for line in out.splitlines():
+ parts = line.split("|")
+ if len(parts) != 3:
+ continue
+ name, cpu, memu = parts
+ # MemUsage is "85.2MiB / 1.95GiB" — strip the "/ total" half.
+ used = memu.split(" /")[0].strip()
+ # Normalise to our own _format_bytes shape: parse the IEC unit suffix.
+ m = re.match(r"^([0-9.]+)\s*([KMGTPE]?i?B)?$", used)
+ if m:
+ val = float(m.group(1))
+ unit = (m.group(2) or "B").lower()
+ mult = {
+ "b": 1,
+ "kb": 1024, "kib": 1024,
+ "mb": 1024**2, "mib": 1024**2,
+ "gb": 1024**3, "gib": 1024**3,
+ "tb": 1024**4, "tib": 1024**4,
+ }.get(unit, 1)
+ mem_str = _format_bytes(int(val * mult))
+ else:
+ mem_str = used or "—"
+ fresh[name] = (cpu, mem_str)
+ _DOCKER_STATS_CACHE = fresh
+ _DOCKER_STATS_TS = time.time()
+
+
+def proc_uptime_cpu_mem(pid: str) -> tuple[str, str, str]:
+ """For a native PID return (uptime, cpu%, mem) — `—` if not running."""
+ try:
+ out = subprocess.run(
+ ["ps", "-p", pid, "-o", "etime=,pcpu=,rss="],
+ capture_output=True, text=True, timeout=2,
+ ).stdout.strip()
+ except (subprocess.TimeoutExpired, FileNotFoundError, OSError):
+ return ("—", "—", "—")
+ if not out:
+ return ("—", "—", "—")
+ parts = out.split()
+ if len(parts) < 3:
+ return ("—", "—", "—")
+ etime, cpu, rss_kb = parts[0], parts[1], parts[2]
+ secs = _parse_etime(etime)
+ try:
+ mem = _format_bytes(int(rss_kb) * 1024)
+ except ValueError:
+ mem = "—"
+ return (_format_uptime(secs), f"{cpu}%", mem)
+
+
+def container_uptime(svc_name: str) -> str:
+ """ISO StartedAt → human duration. Empty if not started."""
+ container = _docker_container_for(svc_name)
+ try:
+ out = subprocess.run(
+ ["docker", "inspect", "-f", "{{.State.StartedAt}}", container],
+ capture_output=True, text=True, timeout=2,
+ ).stdout.strip()
+ except (subprocess.TimeoutExpired, FileNotFoundError, OSError):
+ return "—"
+ if not out or out.startswith("0001-01-01"):
+ return "—"
+ try:
+ # ISO 8601 with fractional + Z. Trim to the second.
+ trimmed = out.split(".", 1)[0].rstrip("Z")
+ started = datetime.strptime(trimmed, "%Y-%m-%dT%H:%M:%S").replace(tzinfo=timezone.utc)
+ except ValueError:
+ return "—"
+ elapsed = (datetime.now(timezone.utc) - started).total_seconds()
+ return _format_uptime(max(0, int(elapsed)))
+
+
+def _jvm_src_dirs(svc: Service) -> list[Path]:
+ # Derive the source-dir set from the SBT dependency graph in build.sbt.
+ # Services that share another's sbt project (e.g. computing-unit-master
+ # rides amber's dist) enter the graph at the *producing* project via
+ # SHARED_SBT_PROJECT.
+ SHARED_SBT_PROJECT = {
+ "computing-unit-master": "WorkflowExecutionService",
+ }
+ project = svc.sbt_project or SHARED_SBT_PROJECT.get(svc.name)
+ if not project:
+ raise RuntimeError(
+ f"service {svc.name!r} has no sbt_project and no SHARED_SBT_PROJECT "
+ f"mapping — can't compute its source-dir closure"
+ )
+ src_dirs = _transitive_src_dirs_cached(project)
+ return [REPO_ROOT / d for d in src_dirs if (REPO_ROOT / d).exists()]
+
+
+def _jvm_source_files(svc: Service) -> list[Path]:
+ files: list[Path] = []
+ for d in _jvm_src_dirs(svc):
+ for f in d.rglob("*"):
+ if f.is_file() and f.suffix in SOURCE_SUFFIXES:
+ files.append(f)
+ # Sort by raw bytes of the string representation so the order matches
+ # GNU `sort -z` exactly (the shell-side `svc_source_hash` uses that).
+ # Python's default `Path.sort()` is component-wise and could disagree
+ # at e.g. `Foo.scala` vs `Foo/` sibling-name collisions, which would
+ # silently break stamp agreement between the two implementations.
+ files.sort(key=lambda p: str(p).encode())
+ return files
+
+
+def source_hash(svc: Service, files: Optional[list[Path]] = None) -> str:
+ """SHA-1 of every source byte under `svc`'s transitive sbt deps.
+ Pass a pre-computed `files` list to skip the rglob walk — the
+ dirty-check fast path already needs the file list to compare mtimes
+ against the stamp, no point rglob'ing twice."""
+ if files is None:
+ files = _jvm_source_files(svc)
+ h = hashlib.sha1()
+ for f in files:
+ try:
+ h.update(f.read_bytes())
+ except OSError:
+ pass
+ return h.hexdigest()
+
+
+def _newest_mtime_after(files: list[Path], stamp_mtime: float) -> bool:
+ for f in files:
+ try:
+ if f.stat().st_mtime > stamp_mtime:
+ return True
+ except OSError:
+ continue
+ return False
+
+
+def is_dirty(svc: Service) -> bool:
+ """Did the service's relevant source change since the last build?
+
+ JVM: SHA-1 of all .scala/.java/.proto bytes vs. the hash we wrote at the
+ last build, with an mtime fast filter so 99% of ticks are O(stat).
+ yarn/bun: lock vs. node_modules dir mtime — cheap.
+ docker: always clean.
+ """
+ if svc.type == "docker":
+ return False
+ if svc.type == "jvm":
+ return _jvm_is_dirty(svc)
+ if svc.type == "yarn":
+ nm = REPO_ROOT / "frontend" / "node_modules" / ".yarn-state.yml"
+ lock = REPO_ROOT / "frontend" / "yarn.lock"
+ if not nm.exists() or not lock.exists():
+ return True
+ return lock.stat().st_mtime > nm.stat().st_mtime
+ if svc.type == "bun":
+ nm = REPO_ROOT / "agent-service" / "node_modules"
+ lock = REPO_ROOT / "agent-service" / "bun.lock"
+ if not nm.exists() or not lock.exists():
+ return True
+ return lock.stat().st_mtime > nm.stat().st_mtime
+ return False
+
+
+def _jvm_is_dirty(svc: Service) -> bool:
+ stamp = BUILD_STAMP_DIR / svc.name
+ if not stamp.exists() or stamp.stat().st_size == 0:
+ # Lazy seed: if a jar is present, assume it matches current source and
+ # write the hash. First REPL after a fresh checkout pays this once.
+ jar = REPO_ROOT / svc.artifact_jar if svc.artifact_jar else None
+ if jar is None or not jar.exists():
+ return True
+ stamp.write_text(source_hash(svc))
+ return False
+
+ files = _jvm_source_files(svc)
+ stamp_mtime = stamp.stat().st_mtime
+
+ # Fast filter: any source newer than the stamp? If not, definitely clean.
+ if not _newest_mtime_after(files, stamp_mtime):
+ return False
+
+ # Slow path — did the content actually move? Reuse the file list above
+ # instead of having `source_hash` rglob the tree a second time.
+ stored = stamp.read_text().strip()
+ current = source_hash(svc, files)
+ if current == stored:
+ # Same content, only mtimes moved (git checkout / touch). Refresh
+ # the stamp's mtime so the fast filter passes next tick.
+ os.utime(stamp, None)
+ return False
+ return True
+
+
+def artifact_mtime_str(svc: Service) -> Optional[str]:
+ if svc.type == "jvm" and svc.artifact_jar:
+ jar = REPO_ROOT / svc.artifact_jar
+ if jar.exists():
+ return datetime.fromtimestamp(jar.stat().st_mtime).strftime("%m-%d %H:%M")
+ return None
+ if svc.type == "bun":
+ f = REPO_ROOT / "agent-service" / "bun.lock"
+ elif svc.type == "yarn":
+ f = REPO_ROOT / "frontend" / "yarn.lock"
+ else:
+ return None
+ if f.exists():
+ return datetime.fromtimestamp(f.stat().st_mtime).strftime("%m-%d %H:%M")
+ return None
+
+
+# ─────────────────── Banner state (cheap) ───────────────────
+
+def git_head() -> tuple[str, str]:
+ branch = subprocess_run("git", "-C", str(REPO_ROOT), "rev-parse", "--abbrev-ref", "HEAD") or "?"
+ sha = subprocess_run("git", "-C", str(REPO_ROOT), "rev-parse", "--short", "HEAD") or "?"
+ return branch, sha
+
+
+def worktree_info() -> tuple[str, bool]:
+ """Return (label, is_worktree). Label is the leaf directory name of the
+ checkout — for the canonical clone this is `texera`, for a worktree it
+ matches the worktree's directory name (which by our convention reflects
+ the branch). is_worktree distinguishes the main checkout from extras so
+ the banner can flag it."""
+ name = REPO_ROOT.name
+ git_dir = subprocess_run("git", "-C", str(REPO_ROOT), "rev-parse", "--git-dir")
+ common_dir = subprocess_run("git", "-C", str(REPO_ROOT), "rev-parse", "--git-common-dir")
+ is_worktree = False
+ try:
+ if git_dir and common_dir:
+ g = Path(git_dir) if Path(git_dir).is_absolute() else (REPO_ROOT / git_dir)
+ c = Path(common_dir) if Path(common_dir).is_absolute() else (REPO_ROOT / common_dir)
+ is_worktree = g.resolve() != c.resolve()
+ except Exception:
+ pass
+ return name, is_worktree
+
+
+def subprocess_run(*argv: str) -> str:
+ import subprocess as sp
+ try:
+ return sp.check_output(argv, stderr=sp.DEVNULL, text=True).strip()
+ except Exception:
+ return ""
+
+
+# ─────────────────── Input with shell-style history ───────────────────
+
+class CommandHistory:
+ """Pure-Python state machine for command history navigation.
+
+ Kept separate from `HistoricInput` (which subclasses Textual's `Input`)
+ so the navigation logic can be unit-tested without a running app —
+ Textual's reactive setters need an active App context, so they can't
+ be exercised from a bare pytest. `HistoricInput` is a thin wrapper that
+ delegates here and forwards the resulting value to its `Input.value`.
+
+ Conventions match bash/zsh: ↑ walks back from newest to oldest, the
+ in-progress draft is saved when stepping off it the first time, ↓
+ walks forward and restores the draft once you step past the newest
+ entry. Consecutive duplicates are coalesced on `push`."""
+
+ def __init__(self, history_file: Optional[Path] = None, max_size: int = MAX_HISTORY) -> None:
+ self._file = history_file
+ self._max = max_size
+ self._history: list[str] = self._load()
+ self._idx: int = -1 # -1 = at the live draft; 0+ = back in history
+ self._draft: str = ""
+
+ def _load(self) -> list[str]:
+ if not self._file or not self._file.exists():
+ return []
+ try:
+ lines = self._file.read_text(errors="replace").splitlines()
+ return [s for s in (l.strip() for l in lines) if s][-self._max:]
+ except OSError:
+ return []
+
+ def _save(self) -> None:
+ if not self._file:
+ return
+ try:
+ self._file.write_text("\n".join(self._history[-self._max:]) + "\n")
+ except OSError:
+ pass
+
+ def push(self, cmd: str) -> None:
+ cmd = cmd.strip()
+ if not cmd:
+ return
+ if self._history and self._history[-1] == cmd:
+ self._reset()
+ return
+ self._history.append(cmd)
+ self._save()
+ self._reset()
+
+ def _reset(self) -> None:
+ self._idx = -1
+ self._draft = ""
+
+ def back(self, current_value: str) -> Optional[str]:
+ """Step one entry back in history. Returns the new value to display,
+ or None if we're already at the oldest entry (caller should leave
+ the input alone)."""
+ if not self._history:
+ return None
+ if self._idx == -1:
+ self._draft = current_value
+ if self._idx + 1 >= len(self._history):
+ return None
+ self._idx += 1
+ return self._history[-1 - self._idx]
+
+ def forward(self) -> Optional[str]:
+ """Step one entry forward. Returns the draft when you cross the
+ newest entry. Returns None if we weren't browsing history."""
+ if self._idx == -1:
+ return None
+ self._idx -= 1
+ if self._idx == -1:
+ return self._draft
+ return self._history[-1 - self._idx]
+
+
+class HistoricInput(Input):
+ """Textual Input wired up to `CommandHistory` for shell-style ↑/↓.
+
+ History is persisted to `HISTORY_FILE` under STATE_DIR so it survives
+ across REPL sessions."""
+
+ BINDINGS = [
+ Binding("up", "history_back", "history back", show=False),
+ Binding("down", "history_forward", "history forward", show=False),
+ ]
+
+ def __init__(self, *args, **kwargs) -> None:
+ super().__init__(*args, **kwargs)
+ self._hist = CommandHistory(HISTORY_FILE)
+
+ def push(self, cmd: str) -> None:
+ self._hist.push(cmd)
+
+ def _set_value(self, v: str) -> None:
+ self.value = v
+ with contextlib.suppress(Exception):
+ self.cursor_position = len(v)
+
+ def action_history_back(self) -> None:
+ v = self._hist.back(self.value)
+ if v is not None:
+ self._set_value(v)
+
+ def action_history_forward(self) -> None:
+ v = self._hist.forward()
+ if v is not None:
+ self._set_value(v)
+
+
+# ─────────────────── Textual app ───────────────────
+
+class LogResizeHandle(Static):
+ """The one-row grip above the log pane. Drag it up/down with the mouse
+ to resize the log; the height is clamped by the RichLog CSS min/max.
+ Doubles as the static "drag to resize" label when idle."""
+
+ DEFAULT_LABEL = " ⇅ log (drag this row to resize)"
+
+ def __init__(self, **kwargs) -> None:
+ super().__init__(self.DEFAULT_LABEL, **kwargs)
+ self._dragging = False
+ self._drag_origin_y = 0
+ self._drag_origin_h = 0
+
+ def on_mouse_down(self, event: events.MouseDown) -> None:
+ # We only care about the primary button.
+ if event.button != 1:
+ return
+ log = self.app.query_one("#log", RichLog)
+ # `region.height` is the current rendered height in cells. Lock it
+ # in as the baseline so we can compute deltas without races against
+ # auto-layout while the user drags.
+ self._drag_origin_y = event.screen_y
+ self._drag_origin_h = log.region.height
+ self._dragging = True
+ self.capture_mouse(True)
+ # Replace the label so users know they grabbed it.
+ self.update(f" ⇅ resizing… (height {self._drag_origin_h})")
+
+ def on_mouse_move(self, event: events.MouseMove) -> None:
+ if not self._dragging:
+ return
+ # Dragging UP (smaller screen_y) should GROW the log since the log
+ # sits below this handle. Invert the delta and clamp; CSS max/min
+ # are a fallback but the explicit clamp keeps the label honest.
+ delta = self._drag_origin_y - event.screen_y
+ new_h = max(4, min(60, int(self._drag_origin_h + delta)))
+ self.app.query_one("#log", RichLog).styles.height = new_h
+ self.update(f" ⇅ resizing… (height {new_h})")
+
+ def on_mouse_up(self, event: events.MouseUp) -> None:
+ if not self._dragging:
+ return
+ self._dragging = False
+ self.capture_mouse(False)
+ self.update(self.DEFAULT_LABEL)
+
+
+# "Apache Texera" wordmark in box-drawing block characters. Single line,
+# 6 rows × ~104 cols. Coloured teal to match the brand.
+LOGO_TEXERA = (
+ " █████╗ ██████╗ █████╗ ██████╗██╗ ██╗███████╗ ████████╗███████╗██╗ ██╗███████╗██████╗ █████╗ \n"
+ "██╔══██╗██╔══██╗██╔══██╗██╔════╝██║ ██║██╔════╝ ╚══██╔══╝██╔════╝╚██╗██╔╝██╔════╝██╔══██╗██╔══██╗\n"
+ "███████║██████╔╝███████║██║ ███████║█████╗ ██║ █████╗ ╚███╔╝ █████╗ ██████╔╝███████║\n"
+ "██╔══██║██╔═══╝ ██╔══██║██║ ██╔══██║██╔══╝ ██║ ██╔══╝ ██╔██╗ ██╔══╝ ██╔══██╗██╔══██║\n"
+ "██║ ██║██║ ██║ ██║╚██████╗██║ ██║███████╗ ██║ ███████╗██╔╝ ██╗███████╗██║ ██║██║ ██║\n"
+ "╚═╝ ╚═╝╚═╝ ╚═╝ ╚═╝ ╚═════╝╚═╝ ╚═╝╚══════╝ ╚═╝ ╚══════╝╚═╝ ╚═╝╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝"
+)
+
+
+STATE_STYLE = {
+ "running": ("●", "green"),
+ "starting": ("⚠", "yellow"),
+ "unhealthy": ("✗", "red"),
+ "failed": ("✗", "red"),
+ "exited": ("✓", "grey50"),
+ "stopped": ("○", "grey50"),
+}
+
+
+def state_cell(state: str) -> Text:
+ sym, style = STATE_STYLE.get(state, ("○", "grey50"))
+ return Text(f"{sym} {state}", style=style)
+
+
+class LocalDevApp(App):
+ """Live dashboard + REPL for the texera local dev stack."""
+
+ CSS = """
+ Screen { layout: vertical; }
+ #banner {
+ height: 11;
+ background: $boost;
+ color: $text;
+ padding: 0 2;
+ border-bottom: heavy $primary;
+ }
+ /* Collapsed mode: hide the wordmark, leave the title/sub two-liner.
+ Use `Ctrl-B` to toggle — useful on short terminals where the dashboard
+ gets cramped by the logo. */
+ #banner.-collapsed {
+ height: 3;
+ }
+ #banner.-collapsed #banner-logo {
+ display: none;
+ }
+ #banner-logo {
+ color: #20b2aa;
+ text-style: bold;
+ height: 7;
+ }
+ #banner-title { text-style: bold; }
+ #banner-sub { color: $text-muted; }
+ DataTable {
+ height: 1fr;
+ border-bottom: solid $primary-darken-1;
+ }
+ #log-header {
+ height: 1;
+ color: $text-muted;
+ background: $primary-darken-3;
+ padding: 0 2;
+ }
+ #log-header.-hidden { display: none; }
+ #log-header:hover { color: $accent; background: $primary-darken-2; }
+ RichLog {
+ height: 18;
+ min-height: 4;
+ max-height: 60;
+ background: $surface;
+ scrollbar-size: 1 1;
+ padding: 0 1;
+ }
+ RichLog.-hidden { display: none; }
+ #status-bar {
+ height: 1;
+ background: $primary-darken-2;
+ color: $text;
+ padding: 0 2;
+ }
+ Input { background: $surface; }
+ """
+
+ BINDINGS = [
+ # Ctrl-C: first press cancels the current command (or log tail); if
+ # nothing's active, requires a second press within 2 s to actually
+ # quit. Matches the way shells & many TUIs treat it.
+ Binding("ctrl+c", "soft_quit", "Cancel / Quit", priority=True, show=False),
+ Binding("escape", "escape_view", "Exit log view", priority=True, show=False),
+ Binding("ctrl+l", "clear_log", "Clear log", show=False),
+ Binding("ctrl+r", "manual_refresh", "Refresh", show=False),
+ Binding("ctrl+b", "toggle_banner", "Toggle banner", show=False),
+ ]
+
+ # Reactive state — Textual diffs widget content when these change.
+ live: reactive[LiveState] = reactive(LiveState, recompose=False)
+ active_cmd: reactive[Optional[str]] = reactive(None)
+ cmd_started_at: reactive[float] = reactive(0.0)
+ log_log_position: reactive[int] = reactive(0) # byte offset we've already read
+
+ # Non-reactive bookkeeping
+ _branch: str = "?"
+ _sha: str = "?"
+ _last_dirty_check: float = 0.0
+ _cached_dirty: dict[str, bool]
+ _cached_mtimes: dict[str, Optional[str]]
+ _cmd_proc: Optional[asyncio.subprocess.Process] = None
+
+ def __init__(self) -> None:
+ super().__init__()
+ self._cached_dirty = {s.name: False for s in SERVICES}
+ self._cached_mtimes = {s.name: None for s in SERVICES}
+ self._branch, self._sha = git_head()
+ self._worktree_name, self._is_worktree = worktree_info()
+ self._log_visible = False
+ self._log_auto_hide_handle = None # type: ignore
+ self._last_ctrl_c_ts: float = 0.0
+
+ # ── Log visibility ──
+ def _set_log_visible(self, show: bool) -> None:
+ self._log_visible = show
+ log = self.query_one("#log", RichLog)
+ header = self.query_one("#log-header", LogResizeHandle)
+ if show:
+ log.remove_class("-hidden")
+ header.remove_class("-hidden")
+ else:
+ log.add_class("-hidden")
+ header.add_class("-hidden")
+
+ # Column keys we keep so update_cell() can address cells reliably. The
+ # string labels passed to add_columns() are NOT the keys (Textual hands
+ # back auto-generated ColumnKey objects), so doing
+ # `update_cell(row, "STATE", ...)` silently fails.
+ _COL_LABELS = ("●", "SERVICE", "PORT", "PID", "UPTIME", "CPU%", "MEM", "ARTIFACT", "BUILD", "STATE")
+ _COL_KEYS = ("sym", "svc", "port", "pid", "uptime", "cpu", "mem", "mtime", "src", "state")
+ # Min width per column key. Without these, Textual sizes the column to
+ # its header (e.g. PID = 3 chars), then truncates wider cell values
+ # like "76348" → "763", which made every JVM look like it shared a
+ # PID. Numeric/timestamp columns deserve enough room for the
+ # widest plausible value.
+ _COL_MIN_WIDTH = {
+ "svc": 32,
+ "port": 6,
+ "pid": 7, # up to 7-digit PIDs (Linux default max is 4194304)
+ "uptime": 10,
+ "cpu": 7,
+ "mem": 7,
+ "mtime": 18,
+ "src": 5,
+ "state": 12,
+ }
+
+ # ── Layout ──
+ def compose(self) -> ComposeResult:
+ yield Vertical(
+ Static(LOGO_TEXERA, id="banner-logo"),
+ Static("", id="banner-title"),
+ Static("", id="banner-sub"),
+ id="banner",
+ )
+ table = DataTable(zebra_stripes=False, cursor_type="row")
+ for label, key in zip(self._COL_LABELS, self._COL_KEYS):
+ table.add_column(label, key=key, width=self._COL_MIN_WIDTH.get(key))
+ for s in SERVICES:
+ table.add_row("○", s.name, f":{s.port}", "—", "—", "—", "—", "—", " ", "stopped", key=s.name)
+ yield table
+ yield LogResizeHandle(id="log-header", classes="-hidden")
+ yield RichLog(id="log", highlight=False, markup=False, wrap=False,
+ auto_scroll=True, classes="-hidden")
+ yield Static("", id="status-bar")
+ yield HistoricInput(placeholder="type a command (h for help · ↑/↓ history · q to quit)", id="prompt")
+
+ def on_mount(self) -> None:
+ self.title = "Texera Local Dev"
+ self._update_banner()
+ self._update_status_bar()
+ self.query_one("#prompt", Input).focus()
+ # Background polling tasks
+ self.set_interval(POLL_INTERVAL_S, self._tick_state)
+ self.set_interval(0.2, self._tick_log)
+ self.set_interval(0.5, self._tick_banner)
+ # 3 Hz lightweight tick so the `stopping.`/`building..` etc dots
+ # animate smoothly between full state polls. Skips work entirely
+ # when no phase markers are active, so the steady-state cost is a
+ # 14-row dict lookup.
+ self.set_interval(0.33, self._tick_phase_animation)
+ # Kick the first poll immediately so the table populates fast.
+ self.call_later(self._tick_state)
+ self.call_later(self._tick_log)
+
+ # ── Updates ──
+ def _update_banner(self) -> None:
+ now = datetime.now().strftime("%H:%M:%S")
+ wt_tag = f"worktree: {self._worktree_name}" if self._is_worktree else f"checkout: {self._worktree_name}"
+ sub = f"{wt_tag} · branch: {self._branch} @ {self._sha} · {now}"
+ self.query_one("#banner-title", Static).update("Apache Texera — Local Dev")
+ self.query_one("#banner-sub", Static).update(sub)
+
+ def _update_status_bar(self) -> None:
+ running = sum(
+ 1 for s in SERVICES
+ if (s.type == "docker" and docker_state(*self.live.docker.get(s.name, ("", ""))) == "running")
+ or (s.type != "docker" and self.live.pids.get(s.name))
+ )
+ total = len(SERVICES)
+ dirty = sum(1 for d in self.live.dirty.values() if d)
+ active = self.active_cmd or "idle"
+ elapsed = ""
+ if self.active_cmd and self.cmd_started_at:
+ elapsed = f" ({int(time.monotonic() - self.cmd_started_at)}s)"
+ dirty_part = f" ★ {dirty} dirty" if dirty else ""
+ # Surface the frontend URL the moment ng serve is listening, so the
+ # user knows when the web app is actually clickable.
+ frontend_pid = self.live.pids.get("frontend")
+ url_part = " → http://localhost:4200" if frontend_pid else ""
+ self.query_one("#status-bar", Static).update(
+ f"{running}/{total} running{dirty_part}{url_part} last: {active}{elapsed}"
+ )
+
+ # State-cell renderer: when the shell has written a transitional phase
+ # (stopping / building / starting), we display that with cycling dots
+ # ("building." → "building.." → "building...") so the user sees motion
+ # while the per-service action is in flight. Cycles at 3 Hz off the
+ # main poll tick.
+ _PHASE_PALETTE = {
+ "stopping": ("◐", "yellow"),
+ "building": ("⚙", "cyan"),
+ "starting": ("⚠", "yellow"),
+ }
+
+ def _phase_text(self, phase: str) -> Text:
+ sym, style = self._PHASE_PALETTE.get(phase, ("⚠", "yellow"))
+ dots = "." * (int(time.monotonic() * 3) % 3 + 1)
+ return Text(f"{phase}{dots}", style=style)
+
+ def _refresh_table(self) -> None:
+ table = self.query_one(DataTable)
+ for svc in SERVICES:
+ phase = self.live.phases.get(svc.name, "")
+ if svc.type == "docker":
+ ds, dstatus = self.live.docker.get(svc.name, ("", ""))
+ state = docker_state(ds, dstatus)
+ pid = "—"
+ else:
+ pid = self.live.pids.get(svc.name) or "—"
+ state = "running" if self.live.pids.get(svc.name) else "stopped"
+ # If the shell signalled a transitional phase AND we haven't
+ # observed the service as fully running yet, render the phase
+ # instead. Once the poller confirms "running" we trust that
+ # over a stale stop/build/start marker.
+ if phase and state != "running":
+ sym, style = self._PHASE_PALETTE.get(phase, ("⚠", "yellow"))
+ state_cell_text = self._phase_text(phase)
+ else:
+ sym, style = STATE_STYLE.get(state, ("○", "grey50"))
+ state_cell_text = Text(state, style=style)
+ mtime = self.live.mtimes.get(svc.name) or "—"
+ dirty = self.live.dirty.get(svc.name, False)
+
+ # BUILD column tells the user whether they need to do anything to
+ # bring the running service in sync with the current source.
+ # ★ (yellow) — content/lock changed since last build; needs action.
+ # ↻ (cyan) — service auto-rebuilds on file change (ng serve /
+ # bun --watch). Reassurance that no manual step is
+ # needed for source edits.
+ # (blank) — built and up-to-date.
+ if dirty:
+ src_cell: Text | str = Text("★", style="bold yellow")
+ elif svc.type in WATCH_TYPES:
+ src_cell = Text("↻", style="cyan")
+ else:
+ src_cell = " "
+
+ uptime = self.live.uptimes.get(svc.name, "—")
+ cpu = self.live.cpu_pct.get(svc.name, "—")
+ mem = self.live.mem_use.get(svc.name, "—")
+ table.update_cell(svc.name, "sym", Text(sym, style=style))
+ table.update_cell(svc.name, "port", f":{svc.port}")
+ table.update_cell(svc.name, "pid", str(pid))
+ table.update_cell(svc.name, "uptime", uptime)
+ table.update_cell(svc.name, "cpu", cpu)
+ table.update_cell(svc.name, "mem", mem)
+ table.update_cell(svc.name, "mtime", mtime)
+ table.update_cell(svc.name, "src", src_cell)
+ table.update_cell(svc.name, "state", state_cell_text)
+
+ # ── Polling tasks (Textual will run them on the event loop) ──
+ @work(exclusive=True, group="state")
+ async def _tick_state(self) -> None:
+ # Polling cost ≈ 1 docker compose ps + lsof × N native services, run
+ # concurrently. ~200 ms total on this box.
+ docker_task = asyncio.create_task(docker_ps_all())
+ native_tasks = {
+ s.name: asyncio.create_task(lsof_port_pid(s.port))
+ for s in SERVICES if s.type != "docker"
+ }
+ docker_map = await docker_task
+ pids = {name: await task for name, task in native_tasks.items()}
+
+ # Dirty check is more expensive; only re-do every DIRTY_INTERVAL_S.
+ now = time.monotonic()
+ if now - self._last_dirty_check >= DIRTY_INTERVAL_S:
+ loop = asyncio.get_running_loop()
+ self._cached_dirty = {
+ s.name: await loop.run_in_executor(None, is_dirty, s) for s in SERVICES
+ }
+ self._cached_mtimes = {s.name: artifact_mtime_str(s) for s in SERVICES}
+ self._last_dirty_check = now
+
+ # Per-service uptime / cpu / mem.
+ # Native: cheap `ps -o etime,pcpu,rss` per pid (<5ms each).
+ # Docker: cached via `docker stats --no-stream` refreshed by a
+ # background worker every _DOCKER_STATS_TTL seconds (the
+ # call itself takes ~2s, too slow to run in-line each tick).
+ uptimes: dict[str, str] = {}
+ cpus: dict[str, str] = {}
+ mems: dict[str, str] = {}
+ for s in SERVICES:
+ if s.type == "docker":
+ state = docker_state(*docker_map.get(s.name, ("", "")))
+ if state == "running":
+ uptimes[s.name] = container_uptime(s.name)
+ container = _docker_container_for(s.name)
+ cpu_mem = _DOCKER_STATS_CACHE.get(container)
+ if cpu_mem:
+ cpus[s.name], mems[s.name] = cpu_mem
+ else:
+ cpus[s.name] = "…"; mems[s.name] = "…"
+ else:
+ uptimes[s.name] = "—"; cpus[s.name] = "—"; mems[s.name] = "—"
+ else:
+ pid = pids.get(s.name)
+ if pid:
+ up, cpu, mem = proc_uptime_cpu_mem(pid)
+ uptimes[s.name] = up; cpus[s.name] = cpu; mems[s.name] = mem
+ else:
+ uptimes[s.name] = "—"; cpus[s.name] = "—"; mems[s.name] = "—"
+ # Kick a docker-stats refresh if the cache is stale; runs in
+ # background so it never blocks the tick.
+ if time.time() - _DOCKER_STATS_TS >= _DOCKER_STATS_TTL:
+ self._refresh_docker_stats_worker()
+
+ phases = {s.name: (read_svc_phase(s.name) or "") for s in SERVICES}
+ new_state = LiveState(
+ docker=docker_map,
+ pids=pids,
+ dirty=dict(self._cached_dirty),
+ mtimes=dict(self._cached_mtimes),
+ uptimes=uptimes,
+ cpu_pct=cpus,
+ mem_use=mems,
+ phases=phases,
+ )
+ self.live = new_state
+ self._refresh_table()
+ self._update_status_bar()
+
+ @work(exclusive=True, group="docker_stats")
+ async def _refresh_docker_stats_worker(self) -> None:
+ loop = asyncio.get_running_loop()
+ await loop.run_in_executor(None, _refresh_docker_stats)
+
+ def _tick_banner(self) -> None:
+ self._update_banner()
+
+ def _tick_phase_animation(self) -> None:
+ """Refresh only the STATE cells whose phase is mid-transition, so
+ the trailing dots animate at 3 Hz without a full poll."""
+ active = {
+ name: phase for name, phase in self.live.phases.items() if phase
+ }
+ if not active:
+ return
+ table = self.query_one(DataTable)
+ for name, phase in active.items():
+ # Skip if the poller has already declared the service running —
+ # don't fight the steady-state render.
+ svc = SERVICES_BY_NAME.get(name)
+ if svc is None:
+ continue
+ if svc.type == "docker":
+ ds, dstatus = self.live.docker.get(name, ("", ""))
+ if docker_state(ds, dstatus) == "running":
+ continue
+ else:
+ if self.live.pids.get(name):
+ continue
+ with contextlib.suppress(Exception):
+ table.update_cell(name, "state", self._phase_text(phase))
+
+ @work(exclusive=True, group="log")
+ async def _tick_log(self) -> None:
+ if not REPL_LOG.exists():
+ return
+ try:
+ size = REPL_LOG.stat().st_size
+ except OSError:
+ return
+ if size < self.log_log_position:
+ # File was truncated (new command). Reset and reread tail.
+ self.log_log_position = 0
+ self.query_one("#log", RichLog).clear()
+ if size == self.log_log_position:
+ return
+ with REPL_LOG.open("rb") as f:
+ f.seek(self.log_log_position)
+ new_bytes = f.read()
+ self.log_log_position = size
+ text = new_bytes.decode(errors="replace")
+ log = self.query_one("#log", RichLog)
+ for raw_line in text.splitlines():
+ log.write(_strip_ansi_motion(raw_line))
+
+ # ── Command handling ──
+ def on_input_submitted(self, message: Input.Submitted) -> None:
+ cmd = message.value.strip()
+ message.input.value = ""
+ if isinstance(message.input, HistoricInput) and cmd:
+ message.input.push(cmd)
+ if not cmd:
+ return
+ if cmd in ("q", "quit", "exit"):
+ self.exit()
+ return
+ if cmd in ("h", "?", "help"):
+ self._show_help()
+ return
+ if cmd in ("r", "refresh"):
+ self.call_later(self._tick_state)
+ return
+ if cmd in ("clear",):
+ self.query_one("#log", RichLog).clear()
+ self._set_log_visible(False)
+ return
+ if cmd in ("log",):
+ # Toggle log pane visibility manually.
+ self._set_log_visible(not self._log_visible)
+ return
+ if cmd in ("banner",):
+ self.action_toggle_banner()
+ return
+ self._dispatch(cmd)
+
+ def _show_help(self) -> None:
+ log = self.query_one("#log", RichLog)
+ log.clear()
+ self._set_log_visible(True)
+ for line in [
+ "Commands:",
+ " r refresh state now",
+ " u build + start every service",
+ " u <svc> start one service (no rebuild)",
+ " d stop every service",
+ " d <svc> stop one service",
+ " b force incremental sbt + node deps",
+ " a / auto scan for dirty services and rebuild+bounce only those",
+ " <svc> rebuild that service and bounce it",
+ " l <svc> tail that service's log (Ctrl-C returns)",
+ " s <svc> stop one service",
+ " clear clear the log pane",
+ " log toggle log pane visibility",
+ " banner toggle banner (collapse the wordmark to save rows; Ctrl-B also works)",
+ " q quit",
+ "",
+ "Mouse: double-click a service row → tail its log.",
+ " double-click the banner → collapse / expand the wordmark.",
+ " Enter on a focused row does the same as double-click.",
+ "",
+ "BUILD column:",
+ " ★ (yellow) source or deps changed since last build — rebuild needed",
+ " ↻ (cyan) service auto-rebuilds on save (ng serve / bun --watch)",
+ " (blank) built and up-to-date",
+ "",
+ f"Known services: {', '.join(s.name for s in SERVICES)}",
+ ]:
+ log.write(line)
+
+ def _dispatch(self, cmd: str) -> None:
+ if self._cmd_proc and self._cmd_proc.returncode is None:
+ log = self.query_one("#log", RichLog)
+ log.write(Text(f"busy: '{self.active_cmd}' still running. Ctrl-C in the term to abort.", style="bold yellow"))
+ return
+
+ parts = cmd.split(None, 1)
+ verb = parts[0]
+ arg = parts[1] if len(parts) > 1 else ""
+
+ # Resolve to a bin/local-dev.sh invocation. Keeping the shell script
+ # as the canonical engine so behavior matches `bin/local-dev.sh up`
+ # from a terminal.
+ argv: Optional[list[str]] = None
+ if verb in ("u", "up"):
+ if arg:
+ if arg not in SERVICES_BY_NAME:
+ self._log_err(f"unknown service: {arg}")
+ return
+ argv = ["start", arg]
+ else:
+ argv = ["up"]
+ elif verb in ("d", "down"):
+ if arg:
+ if arg not in SERVICES_BY_NAME:
+ self._log_err(f"unknown service: {arg}")
+ return
+ argv = ["stop", arg]
+ else:
+ argv = ["down"]
+ elif verb in ("s", "stop"):
+ if not arg or arg not in SERVICES_BY_NAME:
+ self._log_err("usage: s <service>")
+ return
+ argv = ["stop", arg]
+ elif verb in ("b", "build"):
+ # Force an incremental build; the shell handles the "is this
+ # really needed" decision itself (it pre-bounces JVMs etc.).
+ argv = ["up", "--build"]
+ elif verb in ("a", "auto"):
+ # Scan for dirty services and rebuild + bounce only those.
+ argv = ["auto"]
+ elif verb in ("l", "logs", "tail"):
+ if not arg or arg not in SERVICES_BY_NAME:
+ self._log_err(f"usage: l <service> (known: {', '.join(s.name for s in SERVICES)})")
+ return
+ argv = ["logs", arg]
+ self._spawn_logs(arg)
+ return
+ elif verb in SERVICES_BY_NAME:
+ svc_obj = SERVICES_BY_NAME[verb]
+ if svc_obj.type in WATCH_TYPES:
+ # ng serve / bun --watch rebuild on save automatically. The
+ # shell's cmd_update_one already refuses this, but we get a
+ # nicer message by intercepting here.
+ self._log_msg(
+ f"{verb} runs in watch mode (↻) — source edits auto-reload. "
+ f"If you really need to bounce it, run `s {verb}` then `u {verb}`."
+ )
+ return
+ argv = [verb]
+ else:
+ self._log_err(f"unknown: {verb} (type 'h' for help)")
+ return
+
+ if argv is None:
+ return
+ self._spawn_action(verb if not arg else f"{verb} {arg}", argv)
+
+ def _log_err(self, msg: str) -> None:
+ self._set_log_visible(True)
+ self.query_one("#log", RichLog).write(Text("✗ " + msg, style="bold red"))
+
+ def _log_msg(self, msg: str) -> None:
+ self._set_log_visible(True)
+ self.query_one("#log", RichLog).write(Text("• " + msg, style="cyan"))
+ # Auto-dismiss the info pop after a few seconds.
+ if self._log_auto_hide_handle is not None:
+ with contextlib.suppress(Exception):
+ self._log_auto_hide_handle.stop()
+ self._log_auto_hide_handle = self.set_timer(4.0, self._auto_hide_log)
+
+ def _spawn_logs(self, svc: str) -> None:
+ log = self.query_one("#log", RichLog)
+ log.clear()
+ self._set_log_visible(True)
+ log.write(Text(f"── tailing {svc} (type any other command to return) ──", style="dim"))
+ self.active_cmd = f"logs {svc}"
+ self.cmd_started_at = time.monotonic()
+ self._tail_service_log(svc)
+
+ @work(exclusive=True, group="cmd")
+ async def _tail_service_log(self, svc: str) -> None:
+ log_widget = self.query_one("#log", RichLog)
+ svc_obj = SERVICES_BY_NAME[svc]
+ if svc_obj.type == "docker":
+ proc = await asyncio.create_subprocess_exec(
+ "docker", "compose", "-p", DOCKER_PROJECT, "logs", "-f", svc,
+ stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.STDOUT,
+ )
+ else:
+ log_path = LOG_DIR / f"{svc}.log"
+ if not log_path.exists():
+ log_path.touch()
+ proc = await asyncio.create_subprocess_exec(
+ "tail", "-n", "200", "-f", str(log_path),
+ stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.STDOUT,
+ )
+ self._cmd_proc = proc
+ try:
+ assert proc.stdout
+ async for raw in proc.stdout:
+ # If a newer action (e.g. `up`) has claimed _cmd_proc, our
+ # exclusive-group cancellation is in flight but hasn't hit
+ # the next await yet — stop writing to the widget *now* so
+ # we don't leak frontend.log lines into the up output.
+ if self._cmd_proc is not proc:
+ break
+ log_widget.write(_strip_ansi_motion(raw.decode(errors="replace").rstrip("\n")))
+ finally:
+ # ESC fires _cancel_active_cmd which terminates this proc AND
+ # immediately clears active_cmd/_cmd_proc so the user can run
+ # another command. By the time our async-for unwinds the proc
+ # we're holding, the user may have already kicked off `up`. Only
+ # clear the global state if it still belongs to us — otherwise
+ # we'd clobber a freshly-spawned command's state to idle, and
+ # any subsequent up/down would see a stale view.
+ if self._cmd_proc is proc:
+ self.active_cmd = None
+ self._cmd_proc = None
+
+ @work(exclusive=True, group="cmd")
+ async def _spawn_action(self, label: str, argv: list[str]) -> None:
+ log = self.query_one("#log", RichLog)
+ # Cancel any pending auto-hide from a previous command.
+ if self._log_auto_hide_handle is not None:
+ with contextlib.suppress(Exception):
+ self._log_auto_hide_handle.stop()
+ self._log_auto_hide_handle = None
+ # Trip _tail_service_log's ownership check (`self._cmd_proc is not
+ # proc → break`) immediately. Otherwise the tail keeps draining
+ # frontend.log into the log widget across our log.clear() + the
+ # async point at create_subprocess_exec below, and the user sees
+ # frontend output mixed into the up panel.
+ self._cmd_proc = None
+ # Truncate REPL_LOG so the log pane only shows this command's output.
+ REPL_LOG.write_text("")
+ log.clear()
+ self._set_log_visible(True)
+ log.write(Text(f"── {label} → bin/local-dev.sh {' '.join(shlex.quote(a) for a in argv)} ──", style="dim"))
+ self.active_cmd = label
+ self.cmd_started_at = time.monotonic()
+ self.log_log_position = 0
+
+ with REPL_LOG.open("wb") as out:
+ proc = await asyncio.create_subprocess_exec(
+ str(LOCAL_DEV_SH), *argv,
+ stdout=out, stderr=asyncio.subprocess.STDOUT,
+ cwd=str(REPO_ROOT),
+ )
+ self._cmd_proc = proc
+ await proc.wait()
+ rc = proc.returncode or 0
+ style = "bold green" if rc == 0 else "bold red"
+ log.write(Text(f"── {label}: done (exit {rc}) ──", style=style))
+ # Same guard as _tail_service_log: only release the global state if
+ # it still points at our proc. Otherwise a faster newer command has
+ # already taken over and we mustn't reset it to idle.
+ if self._cmd_proc is proc:
+ self.active_cmd = None
+ self._cmd_proc = None
+ # Right after a command, source state likely moved — force a state poll.
+ self._last_dirty_check = 0
+ self.call_later(self._tick_state)
+
+ # Successful command → auto-hide the log so the dashboard reclaims the
+ # space. Failure stays visible so the user can read the error.
+ if rc == 0:
+ self._log_auto_hide_handle = self.set_timer(3.0, self._auto_hide_log)
+
+ def _auto_hide_log(self) -> None:
+ # Only hide if no new command came in and the user hasn't engaged with
+ # the log (we keep it pinned during interactive log tailing).
+ if self.active_cmd is None:
+ self._set_log_visible(False)
+ self._log_auto_hide_handle = None
+
+ def action_toggle_banner(self) -> None:
+ """Collapse / expand the ASCII wordmark to reclaim ~7 rows."""
+ banner = self.query_one("#banner")
+ if banner.has_class("-collapsed"):
+ banner.remove_class("-collapsed")
+ else:
+ banner.add_class("-collapsed")
+
+ def action_clear_log(self) -> None:
+ self.query_one("#log", RichLog).clear()
+ self._set_log_visible(False)
+
+ def action_manual_refresh(self) -> None:
+ self.call_later(self._tick_state)
+
+ # ── ESC: leave whatever transient view we're in ──
+ def action_escape_view(self) -> None:
+ # If a log tail (or any running command) is up, cancel it.
+ if self.active_cmd is not None:
+ self._cancel_active_cmd()
+ # Hide the log pane regardless — ESC's main job is to give the
+ # dashboard the screen back.
+ if self._log_visible:
+ self._set_log_visible(False)
+ # Make sure the prompt has focus so the user can immediately type.
+ self.query_one("#prompt", HistoricInput).focus()
+
+ # ── Ctrl-C: cancel current work, or quit on a second tap ──
+ def action_soft_quit(self) -> None:
+ # 1. Active command (build / up / log tail …) → kill it, keep the
+ # REPL open.
+ if self.active_cmd is not None:
+ self._cancel_active_cmd()
+ self.notify("Ctrl-C — cancelled current command", timeout=2)
+ self._last_ctrl_c_ts = 0.0
+ return
+ # 2. Idle: require a second Ctrl-C within 2 s to actually exit. This
+ # prevents an accidental ⌃-C from killing a session you wanted to
+ # keep.
+ now = time.monotonic()
+ if now - self._last_ctrl_c_ts < 2.0:
+ self.exit()
+ return
+ self._last_ctrl_c_ts = now
+ self.notify("Press Ctrl-C again within 2 s to quit", timeout=2)
+
+ def _cancel_active_cmd(self) -> None:
+ proc = self._cmd_proc
+ if proc and proc.returncode is None:
+ with contextlib.suppress(ProcessLookupError, OSError):
+ proc.terminate()
+ # Cancel the worker too. proc.terminate() races with stdout draining
+ # — until the worker's `async for` actually returns, Textual still
+ # considers it alive in group "cmd", which can block the next
+ # `@work(exclusive=True, group="cmd")` from ever starting. That's
+ # the bug behind "tap into a log → ESC → `down` does nothing".
+ with contextlib.suppress(Exception):
+ self.workers.cancel_group(self, "cmd")
+ self.active_cmd = None
+ self._cmd_proc = None
+
+ # ── Mouse: double-click a service row to tail its log ──
+ def on_click(self, event: events.Click) -> None:
+ widget = getattr(event, "control", None) or getattr(event, "widget", None)
+
+ # Double-click on any banner element → toggle (collapse / expand).
+ # Lets the user reclaim the wordmark's rows without remembering the
+ # Ctrl-B / `banner` aliases.
+ if event.chain == 2 and self._widget_in_banner(widget):
+ self.action_toggle_banner()
+ return
+
+ # Double-click on a service row → tail that service's log.
+ if event.chain == 2:
+ table = self.query_one(DataTable)
+ if widget is table:
+ row = table.cursor_row
+ if 0 <= row < len(SERVICES):
+ self._spawn_logs(SERVICES[row].name)
+ self.query_one("#prompt", Input).focus()
+
+ def _widget_in_banner(self, widget) -> bool:
+ """True when `widget` is the banner container or any of its
+ children. We walk the parent chain rather than matching IDs so a
+ click on the wordmark / title / sub line all count."""
+ node = widget
+ while node is not None:
+ if getattr(node, "id", None) == "banner":
+ return True
+ node = getattr(node, "parent", None)
+ return False
+
+ # Enter on a focused row works too (keyboard equivalent of double-click).
+ def on_data_table_row_selected(self, event: DataTable.RowSelected) -> None:
+ svc = str(event.row_key.value) if event.row_key.value else ""
+ if svc and svc in SERVICES_BY_NAME:
+ self._spawn_logs(svc)
+
+
+# ─────────────────── ANSI motion stripper for log ───────────────────
+
+_CSI_NON_SGR = re.compile(r"\x1b\[[0-9;?]*[A-LN-Za-ln-z]") # everything except SGR (m)
+_CR = re.compile(r"\r+")
+
+
+def _strip_ansi_motion(s: str) -> str:
+ """Strip cursor-motion / erase-screen CSI sequences while keeping SGR colors,
+ and collapse \\r so spinner frames don't pile up in the log pane."""
+ s = _CSI_NON_SGR.sub("", s)
+ s = _CR.sub("", s)
+ return s
+
+
+# ─────────────────── Entry point ───────────────────
+
+def main() -> None:
+ if not LOCAL_DEV_SH.exists():
+ print(f"FATAL: {LOCAL_DEV_SH} not found", file=__import__("sys").stderr)
+ raise SystemExit(1)
+ if shutil.which("docker") is None:
+ print("FATAL: docker not on PATH", file=__import__("sys").stderr)
+ raise SystemExit(1)
+ LocalDevApp().run()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/bin/local-dev.sh b/bin/local-dev.sh
new file mode 100755
index 0000000000..a89eb29a4d
--- /dev/null
+++ b/bin/local-dev.sh
@@ -0,0 +1,2536 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# bin/local-dev.sh -- Manage the Texera local dev stack from a single script.
+#
+# Subcommands:
+# bin/local-dev.sh DEFAULT — one-shot text
+# dashboard (same as `status`).
+# Non-interactive, prints once and
+# exits — safe in scripts/CI.
+# bin/local-dev.sh -i | --interactive Launch the Textual TUI dashboard
+# (live states, SRC dirty
+# indicator, command prompt,
+# double-click for logs, ↑/↓
+# history, Ctrl-C twice to quit).
+# Requires Python + textual.
+# bin/local-dev.sh status same as no-arg invocation.
+# bin/local-dev.sh up [--fresh|--build|--no-build] [--skip=svc1,svc2]
+# Default: skip build if no source/lock
+# changes since last build. --build forces
+# incremental sbt dist + yarn/bun install.
+# --fresh runs `sbt clean dist`. --no-build
+# skips the build step entirely.
+# bin/local-dev.sh down [--skip=svc1,svc2] stop every non-skipped service.
+# bin/local-dev.sh start <service> start one service (no rebuild).
+# bin/local-dev.sh stop <service> stop one service.
+# bin/local-dev.sh <service> rebuild only that service incrementally
+# (sbt <Project>/dist), then bounce it.
+# frontend / agent-service are refused
+# (they have their own watch mode).
+# bin/local-dev.sh logs <service> tail this service's log.
+# bin/local-dev.sh w | watch [interval] Hands-off monitor: redraw the
+# dashboard every <interval>s
+# (default 2). No prompt; Ctrl-C
+# to exit.
+#
+# Managed services (start order):
+# config-service :9094 JVM (sbt ConfigService)
+# access-control-service :9096 JVM (sbt AccessControlService)
+# file-service :9092 JVM (sbt FileService)
+# workflow-compiling-service :9090 JVM (sbt WorkflowCompilingService)
+# computing-unit-managing-service :8082 JVM (sbt ComputingUnitManagingService)
+# texera-web :8080 JVM (sbt WorkflowExecutionService, amber)
+# computing-unit-master :8085 JVM (rides amber dist; no own sbt project)
+# agent-service :3001 Bun --watch (cd agent-service && bun run dev)
+# frontend :4200 ng serve via cd frontend && yarn start
+#
+# Docker infra (postgres / minio / lakefs / lakekeeper / litellm) is NOT managed
+# here -- bring those up yourself before `up`. The script will warn if expected
+# ports are unreachable.
+#
+# Logs and pid book-keeping live under: ${TEXERA_LOCAL_DEV_DIR:-/tmp/texera-local-dev}
+
+set -euo pipefail
+# Unmatched globs in bash default to the literal pattern (we handle that
+# in-place at every glob site). `failglob` / `nullglob` are opt-in per glob
+# via `( shopt -s nullglob; ... )` subshells where we need empty-on-no-match.
+
+REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+cd "$REPO_ROOT"
+
+STATE_DIR="${TEXERA_LOCAL_DEV_DIR:-/tmp/texera-local-dev}"
+LOG_DIR="$STATE_DIR/logs"
+BUILD_STAMP_DIR="$STATE_DIR/build-stamps"
+# Per-service phase markers: shell writes `<phase>\t<epoch>` here as it
+# walks each service through stop → build → start; the TUI reads them
+# every tick and renders an animated transitional state in the STATE
+# column. Removed once the service is up / on stale-after-90s.
+PHASE_DIR="$STATE_DIR/svc-phase"
+mkdir -p "$LOG_DIR" "$BUILD_STAMP_DIR" "$PHASE_DIR"
+
+# --------- associative-array shim for bash 3.2 ---------
+# Apple ships bash 3.2 at /bin/bash and we ship licensing as bash 3.2 too,
+# so we can't use `declare -A`. Every old `MAP[$key]=val` / `${MAP[$key]}`
+# is reshaped as a function call against a flat namespace of synthetic
+# variables `_amap__<MAP>__<key>`. Keys containing `-` get mangled to `_`
+# so they're valid identifier chars.
+amap_set() {
+ local _map="$1" _key="${2//-/_}" _val="$3"
+ eval "_amap__${_map}__${_key}=\$_val"
+}
+amap_get() {
+ local _map="$1" _key="${2//-/_}"
+ local _var="_amap__${_map}__${_key}"
+ eval "printf '%s' \"\${$_var:-}\""
+}
+amap_has() {
+ local _map="$1" _key="${2//-/_}"
+ local _var="_amap__${_map}__${_key}"
+ eval "[[ \${$_var+x} ]]"
+}
+amap_append() {
+ # amap_append MAP key suffix — append to existing value (or seed it).
+ local _map="$1" _key="${2//-/_}" _suffix="$3"
+ local _var="_amap__${_map}__${_key}"
+ eval "$_var=\"\${$_var:-}\$_suffix\""
+}
+
+# --------- toolchain (JDK 17 + node) ---------
+# Detect a JDK 17 installation rather than pinning one path. We try, in
+# order: (1) caller-set $JAVA_HOME if it really is 17, (2) macOS's official
+# locator `/usr/libexec/java_home -v 17`, (3) Homebrew on Apple Silicon +
+# Intel, (4) common Linux distro paths (openjdk / temurin / corretto / zulu),
+# (5) SDKMAN, (6) asdf, (7) the `java` on PATH if its `-version` says 17.
+# Fall through to a clear install hint if none match.
+_java_is_17() {
+ local home="$1"
+ [[ -x "$home/bin/java" ]] || return 1
+ "$home/bin/java" -version 2>&1 | head -1 | grep -q '"17[.]' || return 1
+ return 0
+}
+
+_find_jdk17() {
+ local cand=""
+ # 1. Respect $JAVA_HOME if the caller already set it AND it's 17.
+ if [[ -n "${JAVA_HOME:-}" ]] && _java_is_17 "$JAVA_HOME"; then
+ printf '%s\n' "$JAVA_HOME"; return 0
+ fi
+ # 2. macOS native locator (works for any vendor installed via /Library).
+ if command -v /usr/libexec/java_home >/dev/null 2>&1; then
+ cand=$(/usr/libexec/java_home -v 17 2>/dev/null) || cand=""
+ if [[ -n "$cand" ]] && _java_is_17 "$cand"; then
+ printf '%s\n' "$cand"; return 0
+ fi
+ fi
+ # 3. Homebrew — try `brew --prefix openjdk@17` first, then both well-
+ # known prefixes as a fallback (script may run without brew on PATH
+ # if /etc/zprofile didn't fire).
+ if command -v brew >/dev/null 2>&1; then
+ cand=$(brew --prefix openjdk@17 2>/dev/null) || cand=""
+ [[ -n "$cand" ]] && _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ fi
+ for cand in /opt/homebrew/opt/openjdk@17 /usr/local/opt/openjdk@17; do
+ _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ done
+ # 4. Linux distro layouts. Glob first match.
+ # bash equivalents of zsh's `*(N)` "null on no match" qualifier: enable
+ # `shopt -s nullglob` locally so an unmatched pattern expands to zero
+ # words. We localize it in a subshell so it doesn't leak out and break
+ # the unquoted globs elsewhere (e.g. unzip's zip glob).
+ local matched=""
+ matched=$(shopt -s nullglob; \
+ for cand in \
+ /usr/lib/jvm/java-17-openjdk* \
+ /usr/lib/jvm/temurin-17-jdk* \
+ /usr/lib/jvm/java-17-amazon-corretto* \
+ /usr/lib/jvm/zulu-17* \
+ /usr/lib/jvm/jdk-17*; do
+ printf '%s\n' "$cand"
+ done)
+ while IFS= read -r cand; do
+ [[ -z "$cand" ]] && continue
+ _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ done <<< "$matched"
+ # 5. SDKMAN (`sdk install java 17.x-...`) — pick the lex-largest 17.* dir.
+ if [[ -d "$HOME/.sdkman/candidates/java" ]]; then
+ matched=$(shopt -s nullglob; \
+ for cand in "$HOME"/.sdkman/candidates/java/17.*; do
+ printf '%s\n' "$cand"
+ done)
+ while IFS= read -r cand; do
+ [[ -z "$cand" ]] && continue
+ _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ done <<< "$matched"
+ fi
+ # 6. asdf.
+ if [[ -d "$HOME/.asdf/installs/java" ]]; then
+ matched=$(shopt -s nullglob; \
+ for cand in "$HOME"/.asdf/installs/java/*17*; do
+ printf '%s\n' "$cand"
+ done)
+ while IFS= read -r cand; do
+ [[ -z "$cand" ]] && continue
+ _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ done <<< "$matched"
+ fi
+ # 7. Whatever `java` is on PATH, IF it's 17 — covers cases like Docker
+ # images or distro-managed defaults.
+ cand=$(command -v java 2>/dev/null) || cand=""
+ if [[ -n "$cand" ]]; then
+ cand="$(dirname "$(dirname "$cand")")"
+ _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ fi
+ return 1
+}
+
+# Print the current java environment alongside detection results so the
+# user can see *why* JDK 17 lookup failed (wrong version pinned, JAVA_HOME
+# pointing at JDK 21, etc) — not just "couldn't find one".
+_diagnose_jdk17() {
+ echo "" >&2
+ echo " current java environment:" >&2
+ if [[ -n "${JAVA_HOME:-}" ]]; then
+ local jhver=""
+ jhver=$("$JAVA_HOME/bin/java" -version 2>&1 | head -1 || echo "(unreadable)")
+ echo " \$JAVA_HOME = $JAVA_HOME" >&2
+ echo " ↳ $jhver" >&2
+ else
+ echo " \$JAVA_HOME (unset)" >&2
+ fi
+ local path_java=""
+ path_java=$(command -v java 2>/dev/null) || true
+ if [[ -n "$path_java" ]]; then
+ local pver=""
+ pver=$("$path_java" -version 2>&1 | head -1 || echo "(unreadable)")
+ echo " \`java\` on PATH → $path_java" >&2
+ echo " ↳ $pver" >&2
+ else
+ echo " \`java\` on PATH (not found)" >&2
+ fi
+ if command -v /usr/libexec/java_home >/dev/null 2>&1; then
+ local jh17=""
+ jh17=$(/usr/libexec/java_home -v 17 2>/dev/null) || jh17="(none registered)"
+ echo " /usr/libexec/java_home -v 17 → $jh17" >&2
+ fi
+ if command -v brew >/dev/null 2>&1; then
+ local brewp=""
+ brewp=$(brew --prefix openjdk@17 2>/dev/null) || brewp="(not installed)"
+ echo " brew --prefix openjdk@17 → $brewp" >&2
+ fi
+ echo "" >&2
+ echo " likely cause:" >&2
+ if [[ -n "${JAVA_HOME:-}" ]] && [[ -n "$path_java" ]]; then
+ echo " JAVA_HOME points at the wrong JDK (probably not 17)." >&2
+ echo " Either fix JAVA_HOME or unset it to let auto-detect try other paths." >&2
+ elif [[ -n "$path_java" ]]; then
+ echo " \`java\` on PATH is not JDK 17. Install a 17 sibling or set JAVA_HOME=/path/to/jdk-17." >&2
+ else
+ echo " No JDK is installed anywhere we know to look." >&2
+ fi
+ echo "" >&2
+}
+
+JAVA_HOME_DETECTED="$(_find_jdk17)" || {
+ echo "FATAL: could not find a JDK 17 install." >&2
+ _diagnose_jdk17
+ echo " fix:" >&2
+ echo " macOS: brew install openjdk@17" >&2
+ echo " Linux: apt install openjdk-17-jdk # or yum/dnf equivalent" >&2
+ echo " SDKMAN: sdk install java 17.0.13-tem" >&2
+ echo " or set JAVA_HOME=/path/to/jdk-17 explicitly" >&2
+ echo "" >&2
+ exit 1
+}
+export JAVA_HOME="$JAVA_HOME_DETECTED"
+export PATH="$JAVA_HOME/bin:$PATH"
+
+# Node: source the user's version manager (if any) so the right `node` is on
+# PATH for yarn/bun/ng. Try nvm, fnm, volta in that order; `command -v node`
+# remains the ultimate fallback.
+if [[ -z "${NVM_DIR:-}" && -d "$HOME/.nvm" ]]; then
+ export NVM_DIR="$HOME/.nvm"
+fi
+if [[ -n "${NVM_DIR:-}" && -s "$NVM_DIR/nvm.sh" ]]; then
+ # shellcheck disable=SC1091
+ \. "$NVM_DIR/nvm.sh" >/dev/null 2>&1 || true
+elif command -v fnm >/dev/null 2>&1; then
+ eval "$(fnm env --use-on-cd 2>/dev/null)" || true
+elif [[ -s "$HOME/.volta/load.sh" ]]; then
+ # shellcheck disable=SC1091
+ \. "$HOME/.volta/load.sh" >/dev/null 2>&1 || true
+fi
+
+# --------- runtime env for backend ---------
+# Detect the host's primary LAN IP so we can use it as the MinIO endpoint.
+# It has to be the same string from both directions:
+# • host-native JVMs need it to reach localhost-published port 9000
+# • the lakekeeper container needs it to do server-side S3 ops (validation,
+# compaction) AND to return URLs to clients that *they* can reach
+# `localhost` only works for the host. `texera-minio` only works inside the
+# docker network. The host's LAN IP works from BOTH (host loopback for the
+# host, docker NAT'd out-and-back for the container).
+_detect_host_lan_ip() {
+ local iface="" ip=""
+ # 1. The interface backing the default route — most reliable on a
+ # laptop that may have wifi + thunderbolt + tailscale all active.
+ iface=$(route get default 2>/dev/null | awk '/interface:/{print $2; exit}')
+ if [[ -n "$iface" ]]; then
+ ip=$(ipconfig getifaddr "$iface" 2>/dev/null)
+ [[ -n "$ip" && "$ip" != 127.* ]] && { printf '%s\n' "$ip"; return 0; }
+ fi
+ # 2. Fallback: linux `hostname -I`-equivalent walk over en*.
+ for iface in en0 en1 en2 en3 en4 en5 en6 en7 en8 en9 en10; do
+ ip=$(ipconfig getifaddr "$iface" 2>/dev/null)
+ [[ -n "$ip" && "$ip" != 127.* ]] && { printf '%s\n' "$ip"; return 0; }
+ done
+ return 1
+}
+# Lazy resolver — called from subcommands that actually need to publish a
+# host-reachable S3 endpoint (cmd_up, cmd_auto). Subcommands like
+# `version`, `status`, `--help`, or `-i` don't talk to MinIO and shouldn't
+# refuse to run just because the laptop is offline.
+_require_host_lan_ip() {
+ [[ -n "${HOST_LAN_IP:-}" ]] && return 0
+ HOST_LAN_IP="$(_detect_host_lan_ip)" || HOST_LAN_IP=""
+ if [[ -z "$HOST_LAN_IP" ]]; then
+ echo "FATAL: could not detect a host LAN IP." >&2
+ echo " MinIO needs an address reachable from both docker (lakekeeper" >&2
+ echo " does S3 ops) and the host (JVMs read signed URLs back); none" >&2
+ echo " of \`route get default\` / en0-en10 had a non-loopback IPv4." >&2
+ echo " Connect to a network or export HOST_LAN_IP=<your-IP> explicitly." >&2
+ exit 1
+ fi
+ export HOST_LAN_IP
+ # STORAGE_S3_ENDPOINT below uses the parameter-default form so it
+ # picks up HOST_LAN_IP once we set it here.
+ export STORAGE_S3_ENDPOINT="${STORAGE_S3_ENDPOINT:-http://$HOST_LAN_IP:9000}"
+}
+
+export STORAGE_JDBC_URL="${STORAGE_JDBC_URL:-jdbc:postgresql://localhost:5432/texera_db?currentSchema=texera_db,public}"
+export STORAGE_JDBC_USERNAME="${STORAGE_JDBC_USERNAME:-texera}"
+export STORAGE_JDBC_PASSWORD="${STORAGE_JDBC_PASSWORD:-password}"
+# STORAGE_S3_ENDPOINT is set lazily by _require_host_lan_ip — only the
+# subcommands that actually touch MinIO (infra_up + cmd_up + cmd_auto)
+# trigger that detection, so `version` / `status` / `-i` work offline.
+export STORAGE_S3_AUTH_USERNAME="${STORAGE_S3_AUTH_USERNAME:-texera_minio}"
+export STORAGE_S3_AUTH_PASSWORD="${STORAGE_S3_AUTH_PASSWORD:-password}"
+export STORAGE_S3_REGION="${STORAGE_S3_REGION:-us-west-2}"
+export STORAGE_ICEBERG_CATALOG_TYPE="${STORAGE_ICEBERG_CATALOG_TYPE:-rest}"
+export STORAGE_ICEBERG_CATALOG_REST_URI="${STORAGE_ICEBERG_CATALOG_REST_URI:-http://localhost:8181/catalog}"
+export STORAGE_ICEBERG_CATALOG_REST_WAREHOUSE_NAME="${STORAGE_ICEBERG_CATALOG_REST_WAREHOUSE_NAME:-texera}"
+export STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET="${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET:-texera-iceberg}"
+export STORAGE_ICEBERG_CATALOG_POSTGRES_USERNAME="${STORAGE_ICEBERG_CATALOG_POSTGRES_USERNAME:-texera}"
+export STORAGE_ICEBERG_CATALOG_POSTGRES_PASSWORD="${STORAGE_ICEBERG_CATALOG_POSTGRES_PASSWORD:-password}"
+export STORAGE_ICEBERG_CATALOG_POSTGRES_URI_WITHOUT_SCHEME="${STORAGE_ICEBERG_CATALOG_POSTGRES_URI_WITHOUT_SCHEME:-localhost:5432/texera_iceberg_catalog}"
+export STORAGE_LAKEFS_ENDPOINT="${STORAGE_LAKEFS_ENDPOINT:-http://localhost:8000/api/v1}"
+export STORAGE_LAKEFS_AUTH_USERNAME="${STORAGE_LAKEFS_AUTH_USERNAME:-AKIAIOSFOLKFSSAMPLES}"
+export STORAGE_LAKEFS_AUTH_PASSWORD="${STORAGE_LAKEFS_AUTH_PASSWORD:-wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY}"
+export STORAGE_LAKEFS_AUTH_API_SECRET="${STORAGE_LAKEFS_AUTH_API_SECRET:-random_string_for_lakefs}"
+export UDF_PYTHON_PATH="${UDF_PYTHON_PATH:-$(command -v python3 2>/dev/null || command -v python 2>/dev/null)}"
+export TEXERA_DASHBOARD_SERVICE_ENDPOINT="${TEXERA_DASHBOARD_SERVICE_ENDPOINT:-http://localhost:8080}"
+export WORKFLOW_COMPILING_SERVICE_ENDPOINT="${WORKFLOW_COMPILING_SERVICE_ENDPOINT:-http://localhost:9090}"
+export WORKFLOW_EXECUTION_SERVICE_ENDPOINT="${WORKFLOW_EXECUTION_SERVICE_ENDPOINT:-http://localhost:8085}"
+export FILE_SERVICE_GET_PRESIGNED_URL_ENDPOINT="${FILE_SERVICE_GET_PRESIGNED_URL_ENDPOINT:-http://localhost:9092/api/dataset/presign-download}"
+export FILE_SERVICE_UPLOAD_ONE_FILE_TO_DATASET_ENDPOINT="${FILE_SERVICE_UPLOAD_ONE_FILE_TO_DATASET_ENDPOINT:-http://localhost:9092/api/dataset/did/upload}"
+export LITELLM_BASE_URL="${LITELLM_BASE_URL:-http://localhost:4000}"
+export LITELLM_MASTER_KEY="${LITELLM_MASTER_KEY:-sk-texera-internal-do-not-share}"
+export LLM_ENDPOINT="${LLM_ENDPOINT:-http://localhost:8080}"
+export LLM_API_KEY="${LLM_API_KEY:-dummy}"
+
+# --------- texera version (dynamic) ---------
+# The sbt-native-packager dist directory and jar names embed the project
+# version (e.g. target/config-service-<VERSION>/...). That version moves
+# across branches (1.3.0-incubating-SNAPSHOT on main, 1.2.0-incubating on
+# release/v1.2, …) so resolve it from build.sbt at startup rather than
+# hardcoding. Override via the TEXERA_VERSION env var to target a sibling
+# tree or if the build.sbt parse fails.
+_texera_version() {
+ grep -E '^[[:space:]]*ThisBuild[[:space:]]*/[[:space:]]*version[[:space:]]*:=[[:space:]]*"' \
+ "$REPO_ROOT/build.sbt" 2>/dev/null \
+ | head -1 \
+ | sed -E 's/.*"([^"]+)".*/\1/'
+}
+TEXERA_VERSION="${TEXERA_VERSION:-$(_texera_version)}"
+if [[ -z "$TEXERA_VERSION" ]]; then
+ # tui_warn isn't defined yet at this point in the script; print raw.
+ printf "FATAL: could not detect texera version from %s/build.sbt\n" "$REPO_ROOT" >&2
+ printf " Set the TEXERA_VERSION env var to bypass.\n" >&2
+ exit 1
+fi
+
+# --------- sbt dependency graph (parsed from build.sbt) ---------
+# We parse the build.sbt's project + .dependsOn() declarations so that the
+# per-service "did anything that affects me change?" check can prune to
+# the actual transitive closure. Without this, every JVM service watches
+# every common/* dir and `cmd_auto` rebuilds half the stack whenever
+# common/workflow-operator changes, even though config-service has zero
+# dependency on it.
+#
+# Populates two amap maps keyed by sbt project name plus a parallel
+# indexed-array of keys so we can iterate without `${!MAP[@]}`:
+# amap_get SBT_PATH ConfigService → "config-service"
+# amap_get SBT_DEPS ConfigService → "Auth Config" (space-separated)
+# SBT_DEPS_KEYS=(ConfigService AccessControlService …)
+SBT_DEPS_KEYS=()
+
+_parse_sbt() {
+ local file="$REPO_ROOT/build.sbt"
+ [[ -f "$file" ]] || return 1
+ local current="" line=""
+ local decl_re='^lazy[[:space:]]+val[[:space:]]+([A-Z][A-Za-z0-9]*)[[:space:]]*=[[:space:]]*\(project[[:space:]]+in[[:space:]]+file\("([^"]+)"\)\)'
+ local deps_re='\.dependsOn\(([^)]*)\)'
+
+ # Helper: split a comma-list of dependsOn args, dropping test-scope
+ # and non-project tokens, then append unique main-scope refs.
+ _absorb() {
+ local args="$1"
+ local IFS_OLD="$IFS"
+ IFS=','
+ # shellcheck disable=SC2206
+ local parts=($args)
+ IFS="$IFS_OLD"
+ local arg="" existing=""
+ for arg in "${parts[@]}"; do
+ arg="${arg// /}"
+ arg="${arg//$'\t'/}"
+ # Drop ONLY test-scope deps. The previous catch-all `*"%"*`
+ # match silently swallowed any future `%`-scoped main dep
+ # (e.g. `X % "compile->compile"`, `X % Provided`) and would
+ # have broken dirty-detection on it.
+ if [[ "$arg" =~ %\"?test(-\>[^\"]*)?\"? ]] || [[ "$arg" =~ %Test$ ]]; then
+ continue
+ fi
+ [[ "$arg" =~ ^[A-Z] ]] || continue
+ existing=$(amap_get SBT_DEPS "$current")
+ case " $existing " in
+ *" $arg "*) ;;
+ *) amap_append SBT_DEPS "$current" " $arg" ;;
+ esac
+ done
+ }
+
+ while IFS= read -r line; do
+ local rest="$line"
+ if [[ "$line" =~ $decl_re ]]; then
+ current="${BASH_REMATCH[1]}"
+ amap_set SBT_PATH "$current" "${BASH_REMATCH[2]}"
+ amap_set SBT_DEPS "$current" ""
+ SBT_DEPS_KEYS+=("$current")
+ # Don't `continue` — `.dependsOn(...)` can chain on the SAME
+ # line as the declaration (WorkflowOperator's one-liner does
+ # this). Fall through so the loop below catches it.
+ elif [[ "$line" =~ ^lazy[[:space:]]+val ]]; then
+ # Other `lazy val` (settings, etc) breaks attribution.
+ current=""
+ continue
+ fi
+ [[ -z "$current" ]] && continue
+ # Scan ALL `.dependsOn(...)` matches on this line — some lines
+ # chain `.dependsOn(A).dependsOn(B % "test->test")` and we must
+ # see both to filter correctly. Snapshot BASH_REMATCH BEFORE
+ # calling `_absorb` — that helper does its own `[[ … =~ … ]]`
+ # for the test-scope filter, which clobbers `BASH_REMATCH[0]`
+ # under us. Without the snapshot the `#*…` trim referenced the
+ # wrong value, the strip silently no-op'd, and the outer while
+ # looped forever on the first `.dependsOn(…)`.
+ local _match="" _captured=""
+ while [[ "$rest" =~ $deps_re ]]; do
+ _match="${BASH_REMATCH[0]}"
+ _captured="${BASH_REMATCH[1]}"
+ _absorb "$_captured"
+ rest="${rest#*"$_match"}"
+ done
+ done < "$file"
+ # Trim leading space on every entry.
+ local p="" cur=""
+ for p in "${SBT_DEPS_KEYS[@]}"; do
+ cur=$(amap_get SBT_DEPS "$p")
+ amap_set SBT_DEPS "$p" "${cur# }"
+ done
+ return 0
+}
+_parse_sbt || true
+
+# BFS from $1 over SBT_DEPS, emit one `<path>/src` per visited project.
+# Caller can also pass an explicit fallback list for the unparseable case.
+_sbt_transitive_src_dirs() {
+ local root="$1"
+ [[ -z "$root" ]] && return 1
+ amap_has SBT_PATH "$root" || return 1
+ # `_visited_` namespace is a per-call amap. We can't easily clear all
+ # entries (bash 3.2 has no `${!_visited_*}` glob over set vars without
+ # `compgen`, but `compgen -v` IS available). We unset matching vars
+ # at function exit so successive calls start clean.
+ local queue=("$root")
+ local p="" path="" d="" key="" var=""
+ while ((${#queue[@]} > 0)); do
+ p="${queue[0]}"
+ queue=("${queue[@]:1}")
+ key="${p//-/_}"
+ var="_amap__visited__${key}"
+ eval "[[ \${$var+x} ]]" && continue
+ eval "$var=1"
+ path=$(amap_get SBT_PATH "$p")
+ [[ -n "$path" ]] && printf '%s/src\n' "$path"
+ local deps=""
+ deps=$(amap_get SBT_DEPS "$p")
+ for d in $deps; do
+ queue+=("$d")
+ done
+ done
+ # Clean up our visited markers so the next caller starts fresh.
+ local v=""
+ for v in $(compgen -v _amap__visited__ 2>/dev/null); do
+ unset "$v"
+ done
+ return 0
+}
+
+# --------- service catalog ---------
+SERVICES=(
+ postgres
+ minio
+ lakefs
+ lakekeeper
+ litellm
+ config-service
+ access-control-service
+ file-service
+ workflow-compiling-service
+ computing-unit-master
+ computing-unit-managing-service
+ texera-web
+ agent-service
+ frontend
+)
+
+# Service catalog. Under bash 3.2 this would normally be `typeset -A` maps;
+# we use the `amap_*` helpers defined above instead. Each amap_set call
+# stores into a synthetic var name (e.g. `_amap__SVC_TYPE__postgres`).
+
+# Each docker service is now its own row in the dashboard. start/stop still
+# batch through infra_up/infra_down because `docker compose up -d` and
+# `docker compose down` operate at the project level.
+amap_set SVC_TYPE postgres docker; amap_set SVC_PORT postgres 5432; amap_set SVC_CWD postgres "."
+amap_set SVC_TYPE minio docker; amap_set SVC_PORT minio 9000; amap_set SVC_CWD minio "."
+amap_set SVC_TYPE lakefs docker; amap_set SVC_PORT lakefs 8000; amap_set SVC_CWD lakefs "."
+amap_set SVC_TYPE lakekeeper docker; amap_set SVC_PORT lakekeeper 8181; amap_set SVC_CWD lakekeeper "."
+amap_set SVC_TYPE litellm docker; amap_set SVC_PORT litellm 4000; amap_set SVC_CWD litellm "."
+
+amap_set SVC_TYPE config-service jvm
+amap_set SVC_PORT config-service 9094
+amap_set SVC_SBT config-service ConfigService
+amap_set SVC_LAUNCHER config-service "target/config-service-${TEXERA_VERSION}/bin/config-service"
+amap_set SVC_CWD config-service "."
+amap_set SVC_ZIP_GLOB config-service "config-service/target/universal/config-service-*.zip"
+amap_set SVC_UNZIP_DEST config-service "target/"
+amap_set SVC_HEALTH config-service "/api/healthcheck"
+
+amap_set SVC_TYPE access-control-service jvm
+amap_set SVC_PORT access-control-service 9096
+amap_set SVC_SBT access-control-service AccessControlService
+amap_set SVC_LAUNCHER access-control-service "target/access-control-service-${TEXERA_VERSION}/bin/access-control-service"
+amap_set SVC_CWD access-control-service "."
+amap_set SVC_ZIP_GLOB access-control-service "access-control-service/target/universal/access-control-service-*.zip"
+amap_set SVC_UNZIP_DEST access-control-service "target/"
+amap_set SVC_HEALTH access-control-service "/api/healthcheck"
+
+amap_set SVC_TYPE file-service jvm
+amap_set SVC_PORT file-service 9092
+amap_set SVC_SBT file-service FileService
+amap_set SVC_LAUNCHER file-service "target/file-service-${TEXERA_VERSION}/bin/file-service"
+amap_set SVC_CWD file-service "."
+amap_set SVC_ZIP_GLOB file-service "file-service/target/universal/file-service-*.zip"
+amap_set SVC_UNZIP_DEST file-service "target/"
+amap_set SVC_HEALTH file-service "/api/healthcheck"
+
+amap_set SVC_TYPE workflow-compiling-service jvm
+amap_set SVC_PORT workflow-compiling-service 9090
+amap_set SVC_SBT workflow-compiling-service WorkflowCompilingService
+amap_set SVC_LAUNCHER workflow-compiling-service "target/workflow-compiling-service-${TEXERA_VERSION}/bin/workflow-compiling-service"
+amap_set SVC_CWD workflow-compiling-service "."
+amap_set SVC_ZIP_GLOB workflow-compiling-service "workflow-compiling-service/target/universal/workflow-compiling-service-*.zip"
+amap_set SVC_UNZIP_DEST workflow-compiling-service "target/"
+amap_set SVC_HEALTH workflow-compiling-service "/api/healthcheck"
+
+amap_set SVC_TYPE computing-unit-managing-service jvm
+amap_set SVC_PORT computing-unit-managing-service 8082
+amap_set SVC_SBT computing-unit-managing-service ComputingUnitManagingService
+amap_set SVC_LAUNCHER computing-unit-managing-service "target/computing-unit-managing-service-${TEXERA_VERSION}/bin/computing-unit-managing-service"
+amap_set SVC_CWD computing-unit-managing-service "."
+amap_set SVC_ZIP_GLOB computing-unit-managing-service "computing-unit-managing-service/target/universal/computing-unit-managing-service-*.zip"
+amap_set SVC_UNZIP_DEST computing-unit-managing-service "target/"
+amap_set SVC_HEALTH computing-unit-managing-service ""
+
+amap_set SVC_TYPE texera-web jvm
+amap_set SVC_PORT texera-web 8080
+amap_set SVC_SBT texera-web WorkflowExecutionService
+amap_set SVC_LAUNCHER texera-web "target/amber-${TEXERA_VERSION}/bin/texera-web-application"
+amap_set SVC_CWD texera-web "amber"
+amap_set SVC_ZIP_GLOB texera-web "amber/target/universal/amber-*.zip"
+amap_set SVC_UNZIP_DEST texera-web "amber/target/"
+amap_set SVC_HEALTH texera-web "/api/healthcheck"
+
+# computing-unit-master shares the amber dist with texera-web: sbt-native-
+# packager emits both `bin/texera-web-application` and `bin/computing-unit-master`
+# launchers under `amber/target/amber-<VERSION>/`. We register it as a separate
+# service for status/start/stop but leave SVC_SBT / SVC_ZIP_GLOB empty so the
+# build pipeline knows to skip it (the texera-web build path already produces
+# its launcher). Source-dir and canary-jar lookups treat it identically to
+# texera-web — see _svc_src_dirs / svc_src_changed / svc_artifact_mtime.
+amap_set SVC_TYPE computing-unit-master jvm
+amap_set SVC_PORT computing-unit-master 8085
+amap_set SVC_SBT computing-unit-master ""
+amap_set SVC_LAUNCHER computing-unit-master "target/amber-${TEXERA_VERSION}/bin/computing-unit-master"
+amap_set SVC_CWD computing-unit-master "amber"
+amap_set SVC_ZIP_GLOB computing-unit-master ""
+amap_set SVC_UNZIP_DEST computing-unit-master ""
+amap_set SVC_HEALTH computing-unit-master ""
+
+amap_set SVC_TYPE agent-service bun
+amap_set SVC_PORT agent-service 3001
+amap_set SVC_CWD agent-service "agent-service"
+amap_set SVC_HEALTH agent-service "/api/healthcheck"
+
+amap_set SVC_TYPE frontend yarn
+amap_set SVC_PORT frontend 4200
+amap_set SVC_CWD frontend "frontend"
+amap_set SVC_HEALTH frontend ""
+
+# --------- docker infra config ---------
+DOCKER_PROJECT="texera-local-dev"
+DOCKER_COMPOSE_FILE="$REPO_ROOT/bin/single-node/docker-compose.yml"
+DOCKER_OVERLAY_FILE="$REPO_ROOT/bin/local-dev/docker-compose.override.yml"
+DOCKER_ENV_FILE="$REPO_ROOT/bin/single-node/.env"
+DOCKER_INFRA_SERVICES=(postgres minio minio-init lakefs lakekeeper-migrate lakekeeper lakekeeper-init litellm)
+DOCKER_INFRA_LONGLIVED=(postgres minio lakefs lakekeeper litellm) # exclude one-shot init jobs
+
+# Build the array of -f flags: base single-node compose + local-dev overlay
+# (the overlay publishes infra ports to the host, which the upstream compose
+# intentionally does not do).
+docker_compose_files() {
+ local args=(-f "$DOCKER_COMPOSE_FILE")
+ [[ -f "$DOCKER_OVERLAY_FILE" ]] && args+=(-f "$DOCKER_OVERLAY_FILE")
+ printf '%s\n' "${args[@]}"
+}
+
+# --------- TUI helpers ---------
+if [[ -t 1 ]] && [[ "${TERM:-}" != "dumb" ]]; then
+ BOLD=$'\e[1m'; DIM=$'\e[2m'
+ RED=$'\e[31m'; GREEN=$'\e[32m'; YELLOW=$'\e[33m'
+ BLUE=$'\e[34m'; MAGENTA=$'\e[35m'; CYAN=$'\e[36m'
+ GRAY=$'\e[90m'; BRIGHT=$'\e[97m'
+ RESET=$'\e[0m'
+else
+ BOLD="" DIM="" RED="" GREEN="" YELLOW="" BLUE="" MAGENTA="" CYAN="" GRAY="" BRIGHT="" RESET=""
+fi
+
+TUI_WIDTH=$(tput cols 2>/dev/null || echo 80)
+[[ -z "$TUI_WIDTH" || "$TUI_WIDTH" -lt 60 ]] && TUI_WIDTH=80
+(( TUI_WIDTH > 100 )) && TUI_WIDTH=100 # cap for readability
+
+# Symbols
+SYM_RUN="●"; SYM_STOP="○"; SYM_WARN="⚠"; SYM_OK="✓"; SYM_ERR="✗"
+SYM_SECT="▸"; SYM_LIST="•"; SYM_PROG="→"
+
+tui_hline() {
+ local ch="${1:-─}" w="${2:-$TUI_WIDTH}"
+ printf "${ch}%.0s" $(seq 1 "$w")
+}
+
+tui_trunc() {
+ local s="$1" n="$2"
+ if (( ${#s} > n )); then
+ printf "%s…" "${s:0:$((n-1))}"
+ else
+ printf "%s" "$s"
+ fi
+}
+
+tui_banner() {
+ local title="$1" subtitle="${2:-}"
+ local w=$TUI_WIDTH
+ # Row layout: │ + 2 spaces + content + 2 spaces + │ = w → inner = w - 6
+ local inner=$((w - 6))
+ title="$(tui_trunc "$title" "$inner")"
+ subtitle="$(tui_trunc "$subtitle" "$inner")"
+ printf "${BLUE}╭"; tui_hline "─" $((w-2)); printf "╮${RESET}\n"
+ printf "${BLUE}│${RESET} ${BOLD}${BRIGHT}%-*s${RESET} ${BLUE}│${RESET}\n" "$inner" "$title"
+ if [[ -n "$subtitle" ]]; then
+ printf "${BLUE}│${RESET} ${DIM}%-*s${RESET} ${BLUE}│${RESET}\n" "$inner" "$subtitle"
+ fi
+ printf "${BLUE}╰"; tui_hline "─" $((w-2)); printf "╯${RESET}\n"
+}
+
+tui_section() {
+ printf "\n${BOLD}${MAGENTA}${SYM_SECT}${RESET} ${BOLD}%s${RESET}\n" "$1"
+}
+
+tui_ok() { printf " ${GREEN}${SYM_OK}${RESET} %s\n" "$*"; }
+tui_err() { printf " ${RED}${SYM_ERR}${RESET} %s\n" "$*"; }
+tui_warn() { printf " ${YELLOW}${SYM_WARN}${RESET} %s\n" "$*"; }
+tui_info() { printf " ${CYAN}${SYM_LIST}${RESET} %s\n" "$*"; }
+tui_step() { printf " ${DIM}${SYM_PROG}${RESET} ${DIM}%s${RESET}\n" "$*"; }
+tui_skip() { printf " ${GRAY}${SYM_STOP}${RESET} ${GRAY}%s${RESET}\n" "$*"; }
+
+tui_state_symbol() {
+ case "$1" in
+ running) printf "${GREEN}${SYM_RUN}${RESET}" ;;
+ starting) printf "${YELLOW}${SYM_WARN}${RESET}" ;;
+ unhealthy|failed) printf "${RED}${SYM_ERR}${RESET}" ;;
+ partial:*|external:*) printf "${YELLOW}${SYM_WARN}${RESET}" ;;
+ exited) printf "${GRAY}${SYM_OK}${RESET}" ;;
+ *) printf "${GRAY}${SYM_STOP}${RESET}" ;;
+ esac
+}
+
+tui_state_color() {
+ case "$1" in
+ running) echo "$GREEN" ;;
+ starting) echo "$YELLOW" ;;
+ unhealthy|failed) echo "$RED" ;;
+ partial:*|external:*) echo "$YELLOW" ;;
+ exited) echo "$GRAY" ;;
+ *) echo "$GRAY" ;;
+ esac
+}
+
+# Show a spinner next to $msg while $pid runs. Caller is responsible for
+# `wait $pid` afterwards to capture exit code.
+tui_spinner() {
+ local pid="$1" msg="$2"
+ if [[ ! -t 1 ]]; then
+ printf " ${BLUE}${SYM_PROG}${RESET} ${DIM}%s (no-TTY, no spinner)${RESET}\n" "$msg"
+ return
+ fi
+ # Use an array (vs a single multibyte string + byte indexing) because
+ # bash's `${str:i:1}` is byte-wise — each braille glyph is 3 UTF-8
+ # bytes, so a byte index would print broken half-chars.
+ local frames=(⠋ ⠙ ⠹ ⠸ ⠼ ⠴ ⠦ ⠧ ⠇ ⠏)
+ local n=${#frames[@]}
+ local i=0
+ local start_ts=$SECONDS
+ printf "\e[?25l" # hide cursor
+ while kill -0 "$pid" 2>/dev/null; do
+ local elapsed=$((SECONDS - start_ts))
+ local frame="${frames[i % n]}"
+ printf "\r ${BLUE}%s${RESET} ${DIM}%s${RESET} ${GRAY}(%ds)${RESET} " \
+ "$frame" "$msg" "$elapsed"
+ sleep 0.1
+ i=$((i+1))
+ done
+ printf "\r%-${TUI_WIDTH}s\r" " "
+ printf "\e[?25h" # show cursor
+}
+
+# Run a command in the background with output captured to $log, show spinner,
+# return command's exit code.
+tui_run_with_spinner() {
+ local log="$1" msg="$2"
+ shift 2
+ "$@" >"$log" 2>&1 &
+ local pid=$!
+ tui_spinner "$pid" "$msg"
+ wait "$pid"
+}
+
+# In-place panel that polls each non-skipped service's port and redraws the
+# whole panel until all are healthy or timed out. Sets a trap so Ctrl-C
+# restores the cursor.
+tui_wait_panel() {
+ local svcs=()
+ for svc in "${SERVICES[@]}"; do
+ is_skipped "$svc" && continue
+ svcs+=("$svc")
+ done
+ local n=${#svcs[@]}
+ (( n == 0 )) && return 0
+
+ local frames=(⠋ ⠙ ⠹ ⠸ ⠼ ⠴ ⠦ ⠧ ⠇ ⠏)
+ local n_frames=${#frames[@]}
+ local start_ts=$SECONDS
+ # JVM + docker services bind their port within ~10-30s; 90s leaves slack
+ # for cold-cache machines. The frontend's `ng serve` does a full Angular
+ # compile before listening — on a fresh checkout that's 90-180s easily,
+ # so give yarn-typed services 5 minutes. bun is in between.
+ local timeout_default=90
+ local timeout_yarn=300
+ local timeout_bun=120
+ local frame_idx=0
+ local first_render=true
+ local n_done=0
+ local n_failed=0
+ local svc="" i=0 state="" state_color="" state_sym="" port_str="" elapsed=0 spinner_frame=""
+
+ if [[ ! -t 1 ]]; then
+ # Non-TTY: redrawing would just spam lines. Poll each service to
+ # completion (or per-service timeout) and print one line per
+ # transition. Crucially we WAIT for docker containers to leave
+ # `starting`/`unhealthy` rather than treating an in-flight state as
+ # a failure — the previous one-shot check exited rc=1 within ~20s
+ # of `up` if e.g. lakefs hadn't reached `running` yet, which is its
+ # normal cold-start state.
+ local n_done=0 n_failed=0
+ local svc_timeout=0 waited=0 final_state=""
+ for svc in "${svcs[@]}"; do
+ svc_timeout=$timeout_default
+ case "$(amap_get SVC_TYPE "$svc")" in
+ yarn) svc_timeout=$timeout_yarn ;;
+ bun) svc_timeout=$timeout_bun ;;
+ esac
+ waited=0
+ final_state=""
+ while (( waited < svc_timeout )); do
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ case "$(docker_svc_state "$svc")" in
+ running|exited) final_state="ok"; break ;;
+ unhealthy|failed) final_state="bad"; break ;;
+ # starting / created / restarting / paused / "" → keep waiting
+ esac
+ else
+ [[ -n "$(listen_pid_for_port "$(amap_get SVC_PORT "$svc")")" ]] && { final_state="ok"; break; }
+ fi
+ sleep 1
+ waited=$((waited+1))
+ done
+ case "$final_state" in
+ ok)
+ printf " %s %-32s :%-6s %s\n" "$SYM_OK" "$svc" "$(amap_get SVC_PORT "$svc")" "healthy"
+ n_done=$((n_done+1)) ;;
+ bad)
+ printf " %s %-32s :%-6s %s\n" "$SYM_ERR" "$svc" "$(amap_get SVC_PORT "$svc")" "unhealthy"
+ n_failed=$((n_failed+1)) ;;
+ *) # timed out without ever reaching a terminal state
+ printf " %s %-32s :%-6s %s\n" "$SYM_ERR" "$svc" "$(amap_get SVC_PORT "$svc")" "timeout (${waited}s)"
+ n_failed=$((n_failed+1)) ;;
+ esac
+ done
+ return $((n_failed > 0))
+ fi
+
+ printf "\e[?25l" # hide cursor
+ trap 'printf "\e[?25h"' EXIT INT TERM
+
+ while true; do
+ elapsed=$((SECONDS - start_ts))
+ n_done=0
+ n_failed=0
+
+ if ! $first_render && [[ -t 1 ]]; then
+ printf "\e[${n}A" # move cursor to top of panel
+ fi
+ first_render=false
+
+ spinner_frame="${frames[frame_idx % n_frames]}"
+
+ for svc in "${svcs[@]}"; do
+ state_color="" state_sym=""
+ # Per-service wait budget — yarn (ng serve cold compile) gets the
+ # most slack, bun a moderate amount, everything else the default.
+ local svc_timeout=$timeout_default
+ case "$(amap_get SVC_TYPE "$svc")" in
+ yarn) svc_timeout=$timeout_yarn ;;
+ bun) svc_timeout=$timeout_bun ;;
+ esac
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ local dstate=""
+ dstate=$(docker_svc_state "$svc")
+ case "$dstate" in
+ running|exited)
+ state="$dstate"
+ state_color="$GREEN"; state_sym="${GREEN}${SYM_OK}${RESET}"
+ n_done=$((n_done+1)) ;;
+ starting)
+ state="starting (${elapsed}s)"
+ state_color="$YELLOW"; state_sym="${YELLOW}${spinner_frame}${RESET}" ;;
+ unhealthy|failed)
+ state="$dstate"
+ state_color="$RED"; state_sym="${RED}${SYM_ERR}${RESET}"
+ n_failed=$((n_failed+1)) ;;
+ *)
+ if (( elapsed >= svc_timeout )); then
+ state="timeout"
+ state_color="$RED"; state_sym="${RED}${SYM_ERR}${RESET}"
+ n_failed=$((n_failed+1))
+ else
+ state="${dstate} (${elapsed}s)"
+ state_color="$YELLOW"; state_sym="${YELLOW}${spinner_frame}${RESET}"
+ fi ;;
+ esac
+ port_str=":$(amap_get SVC_PORT "$svc")"
+ else
+ if [[ -n "$(listen_pid_for_port "$(amap_get SVC_PORT "$svc")")" ]]; then
+ state="healthy"
+ state_color="$GREEN"; state_sym="${GREEN}${SYM_OK}${RESET}"
+ n_done=$((n_done+1))
+ elif (( elapsed >= svc_timeout )); then
+ state="timeout — see bin/local-dev.sh logs $svc"
+ state_color="$RED"; state_sym="${RED}${SYM_ERR}${RESET}"
+ n_failed=$((n_failed+1))
+ else
+ state="starting (${elapsed}s)"
+ state_color="$YELLOW"; state_sym="${YELLOW}${spinner_frame}${RESET}"
+ fi
+ port_str=":$(amap_get SVC_PORT "$svc")"
+ fi
+
+ [[ -t 1 ]] && printf "\e[2K"
+ printf " %s %-32s ${DIM}%-7s${RESET} ${state_color}%s${RESET}\n" \
+ "$state_sym" "$svc" "$port_str" "$state"
+ done
+
+ if (( n_done + n_failed == n )); then
+ break
+ fi
+
+ frame_idx=$((frame_idx + 1))
+ sleep 0.2
+ done
+
+ if [[ -t 1 ]]; then
+ printf "\e[?25h"
+ trap - EXIT INT TERM
+ fi
+
+ return $((n_failed > 0))
+}
+
+# --------- toolchain install hints ---------
+# Print install instructions for a missing toolchain. Used by both startup
+# detection failures and runtime "command not found" surfaces. Keeps the
+# guidance in one place so every failure mode looks the same.
+_install_hint() {
+ local tool="$1"
+ case "$tool" in
+ java)
+ printf " ${BOLD}install JDK 17:${RESET}\n"
+ printf " macOS: brew install openjdk@17\n"
+ printf " Linux: apt install openjdk-17-jdk ${DIM}# or yum/dnf equivalent${RESET}\n"
+ printf " SDKMAN: sdk install java 17.0.13-tem\n"
+ printf " or set JAVA_HOME=/path/to/jdk-17 explicitly\n"
+ ;;
+ python)
+ printf " ${BOLD}install Python 3.10+ and the TUI deps:${RESET}\n"
+ printf " macOS: brew install python@3.12\n"
+ printf " Linux: apt install python3 python3-pip\n"
+ printf " then: python3 -m pip install -r %s/amber/dev-requirements.txt\n" "$REPO_ROOT"
+ printf " or set TEXERA_PYTHON=/path/to/python explicitly (must have textual installed)\n"
+ ;;
+ node)
+ printf " ${BOLD}install Node 20+ (needed for frontend & agent-service):${RESET}\n"
+ printf " macOS: brew install node\n"
+ printf " nvm: nvm install --lts && nvm use --lts\n"
+ printf " fnm: fnm install --lts\n"
+ printf " volta: volta install node\n"
+ ;;
+ yarn)
+ printf " ${BOLD}install yarn (needed for the frontend):${RESET}\n"
+ printf " macOS: brew install yarn\n"
+ printf " npm: npm install -g yarn\n"
+ printf " corepack: corepack enable\n"
+ ;;
+ bun)
+ printf " ${BOLD}install bun (needed for agent-service):${RESET}\n"
+ printf " macOS: brew install oven-sh/bun/bun\n"
+ printf " curl: curl -fsSL https://bun.sh/install | bash\n"
+ ;;
+ sbt)
+ printf " ${BOLD}install sbt (needed to build the JVM services):${RESET}\n"
+ printf " macOS: brew install sbt\n"
+ printf " Linux: see https://www.scala-sbt.org/download.html\n"
+ ;;
+ docker)
+ printf " ${BOLD}install Docker (needed for postgres/minio/lakefs/lakekeeper/litellm):${RESET}\n"
+ printf " macOS: download Docker Desktop from https://docker.com/products/docker-desktop\n"
+ printf " Linux: apt install docker.io docker-compose-plugin\n"
+ ;;
+ *)
+ printf " ${DIM}no install hint for: %s${RESET}\n" "$tool"
+ ;;
+ esac
+}
+
+# Inspect the user's Node/yarn/bun setup and print what we actually find,
+# so "yarn not found" reads as "you have node 18 but no yarn" instead of
+# the generic install hint. Called from start_one's frontend/agent paths.
+_diagnose_node() {
+ printf "\n ${BOLD}node environment:${RESET}\n"
+ for tool in node npm yarn bun corepack; do
+ local p=""
+ p=$(command -v "$tool" 2>/dev/null) || true
+ if [[ -n "$p" ]]; then
+ local v=""
+ v=$("$tool" --version 2>&1 | head -1) || v="(no --version)"
+ printf " ${GREEN}✓${RESET} %-8s → %s ${DIM}(%s)${RESET}\n" "$tool" "$p" "$v"
+ else
+ printf " ${RED}✗${RESET} %-8s ${DIM}not on PATH${RESET}\n" "$tool"
+ fi
+ done
+ printf "\n ${BOLD}version manager:${RESET}\n"
+ if [[ -n "${NVM_DIR:-}" && -s "${NVM_DIR}/nvm.sh" ]]; then
+ printf " nvm → ${DIM}%s${RESET}\n" "$NVM_DIR"
+ elif command -v fnm >/dev/null 2>&1; then
+ printf " fnm → ${DIM}%s${RESET}\n" "$(command -v fnm)"
+ elif [[ -s "$HOME/.volta/load.sh" ]]; then
+ printf " volta → ${DIM}%s${RESET}\n" "$HOME/.volta"
+ else
+ printf " ${DIM}none detected (using whatever \`node\` is on PATH)${RESET}\n"
+ fi
+ printf "\n"
+}
+
+# --------- helpers ---------
+listen_pid_for_port() {
+ # || true so pipefail doesn't kill us when nothing is listening
+ lsof -nP -iTCP:"$1" -sTCP:LISTEN -t 2>/dev/null | head -1 || true
+}
+
+# Returns the count of long-lived infra services currently running under our project.
+infra_running_count() {
+ docker compose -p "$DOCKER_PROJECT" ps --services --filter status=running 2>/dev/null | grep -cxE "$(IFS=\|; echo "${DOCKER_INFRA_LONGLIVED[*]}")" || true
+}
+
+# Returns one of: stopped | running | partial:N | external — aggregate view
+# across all infra containers, used by infra_up/infra_down and the wait panel.
+infra_state() {
+ local running=""
+ running=$(infra_running_count)
+ if [[ "$running" -eq ${#DOCKER_INFRA_LONGLIVED[@]} ]]; then
+ echo "running"
+ elif [[ "$running" -gt 0 ]]; then
+ echo "partial:$running"
+ else
+ local taken=0
+ for port in 5432 9000 8000 8181 4000; do
+ [[ -n "$(listen_pid_for_port "$port")" ]] && taken=$((taken+1))
+ done
+ if [[ "$taken" -gt 0 ]]; then
+ echo "external:$taken/5"
+ else
+ echo "stopped"
+ fi
+ fi
+}
+
+# Cached snapshot of `docker compose ps -a`. Refreshed at most once per second
+# (5 docker services × per-row ~200 ms `docker ps` call would otherwise tank
+# the render).
+_docker_states_cache=""
+_docker_states_cache_ts=-1
+
+_refresh_docker_states_cache() {
+ if (( _docker_states_cache_ts < 0 || SECONDS - _docker_states_cache_ts >= 1 )); then
+ _docker_states_cache=$(docker compose -p "$DOCKER_PROJECT" ps -a \
+ --format '{{.Service}}|{{.State}}|{{.Status}}' 2>/dev/null)
+ _docker_states_cache_ts=$SECONDS
+ fi
+}
+
+# Per-service state for any of postgres/minio/lakefs/lakekeeper/litellm.
+# Returns one of: running | starting | unhealthy | exited | failed | stopped
+docker_svc_state() {
+ local svc="$1"
+ _refresh_docker_states_cache
+ local line=""
+ line=$(printf '%s\n' "$_docker_states_cache" | grep "^${svc}|" | head -1 || true)
+ if [[ -z "$line" ]]; then
+ echo "stopped"
+ return
+ fi
+ local rest="${line#*|}"
+ local dstate="${rest%%|*}" # NB: not `status` — zsh reserves $status
+ local dstatus="${rest#*|}"
+ case "$dstate" in
+ running)
+ if [[ "$dstatus" == *'(healthy)'* ]]; then echo "running"
+ elif [[ "$dstatus" == *'(health: starting)'* ]]; then echo "starting"
+ elif [[ "$dstatus" == *'(unhealthy)'* ]]; then echo "unhealthy"
+ else echo "running"
+ fi ;;
+ exited)
+ if [[ "$dstatus" == 'Exited (0)'* ]]; then echo "exited"
+ else echo "failed"
+ fi ;;
+ created|restarting|paused|removing) echo "starting" ;;
+ *) echo "stopped" ;;
+ esac
+}
+
+infra_up() {
+ # Resolve the host LAN IP now (lazy) — both the docker compose stack
+ # (lakekeeper-init reads STORAGE_S3_ENDPOINT) and the host JVMs about
+ # to start need it pointing at a host-reachable MinIO.
+ _require_host_lan_ip
+ if [[ "$(infra_state)" == external:* ]]; then
+ tui_err "infra: ports already taken by non-script containers"
+ printf " ${DIM}Likely an old project (e.g. \`texera-dev\`) is running. Stop it first:${RESET}\n"
+ printf " ${DIM} docker compose -p texera-dev down${RESET}\n"
+ return 1
+ fi
+ local files=($(docker_compose_files))
+ tui_step "infra: docker compose up -d ${DIM}(in-place TTY progress)${RESET}"
+ # No stdout redirect → docker compose detects TTY and renders an in-place
+ # progress panel that overwrites itself instead of appending event lines.
+ # --progress=tty forces it even if stdout looks like a pipe.
+ docker compose --progress auto -p "$DOCKER_PROJECT" --env-file "$DOCKER_ENV_FILE" "${files[@]}" \
+ up -d "${DOCKER_INFRA_SERVICES[@]}"
+ tui_ok "infra: 5 containers up"
+}
+
+infra_down() {
+ tui_step "infra: docker compose -p $DOCKER_PROJECT down ${DIM}(in-place TTY progress)${RESET}"
+ docker compose --progress auto -p "$DOCKER_PROJECT" down || true
+ tui_ok "infra: stopped"
+}
+
+# Ensure the texera_db schema exists in the postgres container. The compose
+# file mounts sql/*.sql to /docker-entrypoint-initdb.d, but Postgres only
+# runs those on first init (empty data dir). If the volume was carried over
+# from an older texera version (e.g. before the `feedback` table was added)
+# the schema will be missing relations that current code references, the
+# jOOQ codegen produces an incomplete Tables.java, and sbt compile fails on
+# `not found: value FEEDBACK`. Probe for a canonical table and re-run
+# texera_ddl.sql if it's absent.
+infra_ensure_db_schema() {
+ local pg="texera-postgres"
+ # Wait briefly for postgres to be ready — `up -d` returned but the
+ # container may still be running its own init sequence.
+ local i=0
+ while (( i < 30 )); do
+ if docker exec "$pg" pg_isready -U texera -d texera_db -q 2>/dev/null; then
+ break
+ fi
+ sleep 1
+ i=$((i+1))
+ done
+ if (( i >= 30 )); then
+ tui_warn "postgres: not ready after 30s -- skipping schema check"
+ return 0
+ fi
+ # `feedback` is one of the newer tables; if it exists we assume the
+ # whole schema is current. (texera_ddl.sql is idempotent with
+ # CREATE TABLE IF NOT EXISTS, so re-applying it is safe even if some
+ # tables already exist, but skipping the copy + exec is faster.)
+ local has_feedback=""
+ has_feedback=$(docker exec "$pg" psql -U texera -d texera_db -tAc \
+ "SELECT 1 FROM pg_tables WHERE schemaname='texera_db' AND tablename='feedback'" \
+ 2>/dev/null || true)
+ if [[ "$has_feedback" == "1" ]]; then
+ tui_skip "postgres: schema already current"
+ return 0
+ fi
+ tui_step "postgres: applying sql/texera_ddl.sql (one-time bootstrap)"
+ local ddl="$REPO_ROOT/sql/texera_ddl.sql"
+ if [[ ! -f "$ddl" ]]; then
+ tui_warn "postgres: $ddl not found -- skipping (jOOQ codegen may fail)"
+ return 0
+ fi
+ docker cp "$ddl" "$pg":/tmp/texera_ddl.sql >/dev/null
+ if docker exec -u postgres "$pg" psql -U texera -f /tmp/texera_ddl.sql >/dev/null 2>&1; then
+ tui_ok "postgres: schema bootstrapped"
+ else
+ tui_warn "postgres: ddl exec returned non-zero (check container logs)"
+ fi
+}
+
+svc_running_pid() {
+ listen_pid_for_port "$(amap_get SVC_PORT "$1")"
+}
+
+# Compact-format a duration in seconds:
+# "—" for <0, "12s", "5m 23s", "2h 14m", "3d 4h"
+_format_uptime() {
+ local s="${1:-0}"
+ if (( s < 0 )); then printf "—"; return; fi
+ if (( s < 60 )); then printf "%ds" "$s"; return; fi
+ if (( s < 3600 )); then printf "%dm %ds" $((s/60)) $((s%60)); return; fi
+ if (( s < 86400 )); then printf "%dh %dm" $((s/3600)) $(((s%3600)/60)); return; fi
+ printf "%dd %dh" $((s/86400)) $(((s%86400)/3600))
+}
+
+# Translate ps -o etime output (`[[DD-]hh:]mm:ss`) to whole seconds.
+_etime_to_seconds() {
+ local et="$1"
+ # bash quirk: in `local a=$x b=$a`, the right-hand `$a` resolves
+ # against the *outer* scope (before this `local` declares either).
+ # Putting `rest` on its own line ensures it sees the fresh `et`.
+ local rest="$et" days=0 h=0 m=0 s=0
+ if [[ "$rest" == *-* ]]; then
+ days="${rest%%-*}"
+ rest="${rest#*-}"
+ fi
+ # `IFS=: read -ra` reliably splits on `:` without the local-IFS-vs-
+ # array-literal trap.
+ local parts=()
+ IFS=: read -ra parts <<< "$rest"
+ case "${#parts[@]}" in
+ 3) h=${parts[0]}; m=${parts[1]}; s=${parts[2]} ;;
+ 2) m=${parts[0]}; s=${parts[1]} ;;
+ *) s=${parts[0]} ;;
+ esac
+ # Force base-10 so "08" / "09" don't blow up under bash's octal default.
+ printf '%d' "$((10#${days:-0}*86400 + 10#${h:-0}*3600 + 10#${m:-0}*60 + 10#${s:-0}))"
+}
+
+# Translate a number of bytes to a 4-char-ish human string ("85M", "1.2G").
+_format_bytes() {
+ local b="${1:-0}"
+ if (( b < 1024 )); then printf "%dB" "$b"; return; fi
+ if (( b < 1048576 )); then printf "%dK" $((b/1024)); return; fi
+ if (( b < 1073741824 )); then printf "%dM" $((b/1048576)); return; fi
+ # GB → 1 decimal place
+ local g_int=$((b / 1073741824)) g_frac=$(( (b % 1073741824) * 10 / 1073741824 ))
+ printf "%d.%dG" "$g_int" "$g_frac"
+}
+
+# Compact "8.2%" formatter from a string like "0.50" or "8.2".
+_format_pct() {
+ local p="${1:-0}"
+ # Strip optional trailing %, normalise weird inputs
+ p="${p%%%}"
+ [[ -z "$p" || "$p" == "-" ]] && { printf "—"; return; }
+ # Round to 1 decimal place using awk to avoid bash's lack of float math
+ printf '%s%%' "$(awk -v v="$p" 'BEGIN { printf "%.1f", v }' 2>/dev/null)"
+}
+
+# Container name for a docker-typed service. Most use `texera-<svc>`, but
+# `litellm` ships unprefixed in upstream's compose.
+_docker_container_for() {
+ case "$1" in
+ litellm) printf "litellm" ;;
+ *) printf "texera-%s" "$1" ;;
+ esac
+}
+
+# Uptime for a service. Native: `ps -o etime` (POSIX, works on mac+linux).
+# Docker: parse the container's ISO-8601 .State.StartedAt against `date -u`.
+# Echoes "—" when not running.
+svc_uptime() {
+ local svc="$1"
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ local container="" since=""
+ container=$(_docker_container_for "$svc")
+ since=$(docker inspect -f '{{.State.StartedAt}}' "$container" 2>/dev/null) || { printf "—"; return; }
+ [[ -z "$since" || "$since" == "0001-01-01T00:00:00Z" ]] && { printf "—"; return; }
+ local now=0 started=0
+ now=$(date -u +%s 2>/dev/null) || { printf "—"; return; }
+ # macOS `date -j` parses ISO without fractional seconds — trim them.
+ started=$(date -u -j -f "%Y-%m-%dT%H:%M:%S" "${since%%.*}" +%s 2>/dev/null) || {
+ # GNU date fallback
+ started=$(date -u -d "${since%%.*}" +%s 2>/dev/null) || { printf "—"; return; }
+ }
+ local elapsed=$((now - started))
+ (( elapsed < 0 )) && elapsed=0
+ _format_uptime "$elapsed"
+ return
+ fi
+ local pid=""
+ pid=$(svc_running_pid "$svc") || true
+ [[ -z "$pid" ]] && { printf "—"; return; }
+ local et=""
+ et=$(ps -p "$pid" -o etime= 2>/dev/null | tr -d ' ')
+ [[ -z "$et" ]] && { printf "—"; return; }
+ _format_uptime "$(_etime_to_seconds "$et")"
+}
+
+# CPU% + RSS for a running native process. Returns "cpu%|rss_bytes".
+# Cheap enough (<5ms) that we can call it per service per tick.
+svc_proc_cpu_mem() {
+ local pid=""
+ pid=$(svc_running_pid "$1") || true
+ [[ -z "$pid" ]] && { printf "—|—"; return; }
+ local cpu="" rss=""
+ # macOS ps emits RSS in KB. Linux's `ps -o rss=` is also KB.
+ read -r cpu rss <<< "$(ps -p "$pid" -o pcpu=,rss= 2>/dev/null | tr -s ' ')"
+ [[ -z "$cpu" || -z "$rss" ]] && { printf "—|—"; return; }
+ printf "%s|%s" "$(_format_pct "$cpu")" "$(_format_bytes $((rss * 1024)))"
+}
+
+# Bulk docker stats — one slow `docker stats --no-stream` call returns CPU
+# + memory for every running container. Callers can grep their service out
+# of the result instead of paying the 1-2 second cost N times. Format:
+# <container>|<cpu%>|<used-bytes>
+_docker_stats_snapshot() {
+ docker stats --no-stream --format '{{.Name}}|{{.CPUPerc}}|{{.MemUsage}}' 2>/dev/null \
+ | while IFS='|' read -r name cpu memu; do
+ # MemUsage is like "85.2MiB / 1.95GiB" — keep only the first
+ # half and translate IEC units to bytes.
+ local used="${memu%% *}" # "85.2MiB"
+ local val="${used%[A-Za-z]*}"
+ local unit="${used##*[0-9.]}"
+ local bytes=0
+ case "$unit" in
+ KB|KiB) bytes=$(awk -v v="$val" 'BEGIN { printf "%d", v * 1024 }') ;;
+ MB|MiB) bytes=$(awk -v v="$val" 'BEGIN { printf "%d", v * 1048576 }') ;;
+ GB|GiB) bytes=$(awk -v v="$val" 'BEGIN { printf "%d", v * 1073741824 }') ;;
+ *) bytes=$(awk -v v="$val" 'BEGIN { printf "%d", v }') ;;
+ esac
+ printf "%s|%s|%s\n" "$name" "$(_format_pct "${cpu%%%}")" "$(_format_bytes "$bytes")"
+ done
+}
+
+svc_artifact_mtime() {
+ local svc="$1" type=""
+ type=$(amap_get SVC_TYPE "$svc")
+ case "$type" in
+ jvm)
+ local launcher="$(amap_get SVC_CWD "$svc")/$(amap_get SVC_LAUNCHER "$svc")"
+ launcher="${launcher#./}"
+ local jar_dir=""
+ jar_dir="$(dirname "$(dirname "$launcher")")/lib"
+ if [[ -d "$jar_dir" ]]; then
+ # `shopt -s nullglob` localised in a subshell so an
+ # unmatched glob yields no words instead of the literal
+ # pattern. Capture into a string then split into an array.
+ local globbed=""
+ globbed=$(shopt -s nullglob; \
+ printf '%s\n' "$jar_dir"/org.apache.texera."${svc}"-*.jar)
+ local main_jars=()
+ while IFS= read -r _f; do
+ [[ -n "$_f" ]] && main_jars+=("$_f")
+ done <<< "$globbed"
+ if [[ ${#main_jars[@]} -eq 0 && ( "$svc" == "texera-web" || "$svc" == "computing-unit-master" ) ]]; then
+ globbed=$(shopt -s nullglob; \
+ printf '%s\n' "$jar_dir"/org.apache.texera.amber-*.jar)
+ while IFS= read -r _f; do
+ [[ -n "$_f" ]] && main_jars+=("$_f")
+ done <<< "$globbed"
+ fi
+ if [[ ${#main_jars[@]} -gt 0 ]]; then
+ stat -f "%Sm" -t "%Y-%m-%d %H:%M" "${main_jars[0]}"
+ return
+ fi
+ fi
+ echo "—"
+ ;;
+ bun) stat -f "%Sm" -t "%Y-%m-%d %H:%M" "$(amap_get SVC_CWD "$svc")/bun.lock" 2>/dev/null || echo "—" ;;
+ yarn) stat -f "%Sm" -t "%Y-%m-%d %H:%M" "$(amap_get SVC_CWD "$svc")/yarn.lock" 2>/dev/null || echo "—" ;;
+ docker) echo "—" ;;
+ esac
+}
+
+is_skipped() {
+ [[ ",${SKIP_LIST:-}," == *",$1,"* ]]
+}
+
+wait_for_port() {
+ local port="$1" timeout="${2:-90}" i=0
+ while (( i < timeout )); do
+ [[ -n "$(listen_pid_for_port "$port")" ]] && return 0
+ sleep 1
+ i=$((i+1))
+ done
+ return 1
+}
+
+# Write a transitional phase for $svc so the TUI dashboard can render it
+# in the STATE column with animated dots. Cleared by phase_clear or after
+# 90s (TUI side stale check).
+phase_set() {
+ local svc="$1" phase="$2"
+ printf '%s\t%s\n' "$phase" "$(date +%s)" > "$PHASE_DIR/$svc" 2>/dev/null || true
+}
+phase_clear() {
+ local svc="$1"
+ rm -f "$PHASE_DIR/$svc" 2>/dev/null || true
+}
+
+stop_one() {
+ local svc="$1"
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ phase_set "$svc" stopping
+ tui_step "$svc: docker compose stop $svc"
+ docker compose -p "$DOCKER_PROJECT" stop "$svc" >/dev/null 2>&1 || true
+ phase_clear "$svc"
+ tui_ok "$svc: stopped"
+ return
+ fi
+ local pid=""
+ pid=$(svc_running_pid "$svc")
+ if [[ -z "$pid" ]]; then
+ tui_skip "$svc: already stopped"
+ return 0
+ fi
+ phase_set "$svc" stopping
+ tui_step "$svc: stopping PID $pid"
+ kill "$pid" 2>/dev/null || true
+ local i=0
+ while (( i < 30 )) && kill -0 "$pid" 2>/dev/null; do
+ sleep 0.5
+ i=$((i+1))
+ done
+ if kill -0 "$pid" 2>/dev/null; then
+ tui_warn "$svc: SIGKILL $pid"
+ kill -9 "$pid" 2>/dev/null || true
+ fi
+ phase_clear "$svc"
+ tui_ok "$svc: stopped"
+}
+
+start_one() {
+ local svc="$1"
+ local type=""
+ type=$(amap_get SVC_TYPE "$svc")
+ # JVMs and docker services need STORAGE_S3_ENDPOINT exported before
+ # launch (JVM clients dial MinIO; lakekeeper bakes the URL into the
+ # warehouse storage profile). yarn/bun watch services don't.
+ if [[ "$type" == "jvm" || "$type" == "docker" ]]; then
+ _require_host_lan_ip
+ fi
+ if [[ "$type" == "docker" ]]; then
+ local dstate=""
+ dstate=$(docker_svc_state "$svc")
+ if [[ "$dstate" == "running" ]]; then
+ tui_ok "$svc: already running"
+ return 0
+ fi
+ phase_set "$svc" starting
+ tui_step "$svc: docker compose up -d $svc"
+ local files=()
+ while read -r f; do files+=("$f"); done < <(docker_compose_files | tr ' ' '\n')
+ docker compose --progress auto -p "$DOCKER_PROJECT" --env-file "$DOCKER_ENV_FILE" \
+ "${files[@]}" up -d "$svc" >/dev/null 2>&1
+ # Don't clear the phase yet — docker `up -d` returns before the
+ # container is healthy. The TUI's docker_state poller will flip
+ # the row to "running" once the container reports healthy; the
+ # stale-after-90s rule covers cleanup if that never happens.
+ tui_ok "$svc: started"
+ return
+ fi
+ if [[ -n "$(svc_running_pid "$svc")" ]]; then
+ tui_ok "$svc: already running ${DIM}(PID $(svc_running_pid "$svc"))${RESET}"
+ return 0
+ fi
+ local cwd="" log="$LOG_DIR/$svc.log"
+ cwd=$(amap_get SVC_CWD "$svc")
+ phase_set "$svc" starting
+ tui_step "$svc: starting ${DIM}(log: $log)${RESET}"
+ case "$type" in
+ jvm)
+ local launcher=""
+ launcher=$(amap_get SVC_LAUNCHER "$svc")
+ if [[ ! -x "$cwd/$launcher" ]]; then
+ phase_clear "$svc"
+ tui_err "$svc: launcher missing at $cwd/$launcher -- run \`bin/local-dev.sh up\` to build first"
+ return 1
+ fi
+ ( cd "$cwd" && nohup "./$launcher" >"$log" 2>&1 </dev/null & )
+ ;;
+ bun)
+ if ! command -v bun >/dev/null 2>&1; then
+ phase_clear "$svc"
+ tui_err "$svc: \`bun\` not found on PATH"
+ _diagnose_node
+ _install_hint bun
+ return 1
+ fi
+ ( cd "$cwd" && nohup bun run dev >"$log" 2>&1 </dev/null & )
+ ;;
+ yarn)
+ if ! command -v yarn >/dev/null 2>&1; then
+ phase_clear "$svc"
+ tui_err "$svc: \`yarn\` not found on PATH"
+ _diagnose_node
+ if ! command -v node >/dev/null 2>&1; then
+ _install_hint node
+ else
+ _install_hint yarn
+ fi
+ return 1
+ fi
+ ( cd "$cwd" && nohup yarn start >"$log" 2>&1 </dev/null & )
+ ;;
+ esac
+}
+
+build_one_jvm() {
+ local svc="$1" proj=""
+ proj=$(amap_get SVC_SBT "$svc")
+ local log="$LOG_DIR/sbt-${svc}.log"
+ # Empty SVC_SBT means this service rides another service's dist (e.g.
+ # computing-unit-master shares amber's). Nothing to build directly — the
+ # launcher is produced when its sibling builds. Stamp `svc` so the dirty
+ # indicator can clear if amber/src actually matches.
+ if [[ -z "$proj" ]]; then
+ tui_skip "$svc: no own sbt project (built with its sibling)"
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc" 2>/dev/null || true
+ return 0
+ fi
+ phase_set "$svc" building
+ if tui_run_with_spinner "$log" "sbt $proj/dist ${DIM}(log: $log)${RESET}" \
+ sbt -no-colors "$proj/dist"; then
+ local zip_glob="" unzip_dest=""
+ zip_glob=$(amap_get SVC_ZIP_GLOB "$svc")
+ unzip_dest=$(amap_get SVC_UNZIP_DEST "$svc")
+ tui_step "unzip ${zip_glob} → ${unzip_dest}"
+ # shellcheck disable=SC2086
+ unzip -oq ${zip_glob} -d "${unzip_dest}"
+ # Stamp = SHA-1 of the source we just built from. Clears the `*` and
+ # lets us tell content-vs-mtime apart on the next dirty check.
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc"
+ tui_ok "$svc: build done"
+ # Don't clear the phase yet — the caller (cmd_update_one) will
+ # transition us through stop_one/start_one which overwrite it.
+ # If something else is the caller, the TUI's "phase cleared once
+ # poller sees running" rule covers us.
+ else
+ phase_clear "$svc"
+ tui_err "$svc: sbt $proj/dist FAILED ${DIM}(tail -f $log)${RESET}"
+ return 1
+ fi
+}
+
+# True if ANY JVM service's source changed (content-hash) since its last build.
+# This is what `up`/`build` use to decide whether to skip the sbt step. It
+# is the same check the dashboard's SRC `*` indicator uses, just OR'd across
+# all JVM services.
+any_jvm_src_changed() {
+ local svc=""
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "jvm" ]] || continue
+ if svc_src_changed "$svc" 2>/dev/null; then
+ return 0
+ fi
+ done
+ return 1
+}
+
+needs_yarn_install() {
+ [[ ! -f frontend/node_modules/.yarn-state.yml ]] && return 0
+ [[ frontend/yarn.lock -nt frontend/node_modules/.yarn-state.yml ]] && return 0
+ [[ frontend/package.json -nt frontend/node_modules/.yarn-state.yml ]] && return 0
+ return 1
+}
+
+needs_bun_install() {
+ [[ ! -d agent-service/node_modules ]] && return 0
+ [[ agent-service/bun.lock -nt agent-service/node_modules ]] && return 0
+ [[ agent-service/package.json -nt agent-service/node_modules ]] && return 0
+ return 1
+}
+
+# Cache of per-service transitive src dirs, populated once at startup by
+# _precompute_src_dirs(). Each entry (in the SVC_SRC_DIRS amap) is a
+# newline-separated list — callers consume via `while read` (same shape
+# as the old _svc_src_dirs output).
+
+_precompute_src_dirs() {
+ local svc="" entry="" dirs=""
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "jvm" ]] || continue
+ entry=$(amap_get SVC_SBT "$svc")
+ # computing-unit-master rides amber's dist (no SVC_SBT of its
+ # own); enter the graph at the producing project explicitly.
+ if [[ -z "$entry" && "$svc" == "computing-unit-master" ]]; then
+ entry="WorkflowExecutionService"
+ fi
+ if [[ -z "$entry" ]]; then
+ echo "FATAL: service '$svc' has no SVC_SBT entry and no" >&2
+ echo " SHARED_SBT_PROJECT mapping — can't compute its" >&2
+ echo " transitive source-dir closure." >&2
+ exit 1
+ fi
+ if ! amap_has SBT_PATH "$entry"; then
+ echo "FATAL: sbt project '$entry' (for service '$svc') is not" >&2
+ echo " in the parsed build.sbt graph. Either build.sbt" >&2
+ echo " parsing failed or the project was renamed/removed." >&2
+ exit 1
+ fi
+ dirs="$(_sbt_transitive_src_dirs "$entry")"
+ if [[ -z "$dirs" ]]; then
+ echo "FATAL: transitive source-dir closure for '$svc' is empty." >&2
+ exit 1
+ fi
+ amap_set SVC_SRC_DIRS "$svc" "$dirs"
+ done
+}
+
+# Look up the pre-populated cache.
+_svc_src_dirs() {
+ local svc="$1"
+ local cached=""
+ cached=$(amap_get SVC_SRC_DIRS "$svc")
+ if [[ -z "$cached" ]]; then
+ echo "FATAL: _svc_src_dirs: '$svc' missing from SVC_SRC_DIRS" >&2
+ echo " (did _precompute_src_dirs run?)" >&2
+ exit 1
+ fi
+ printf '%s\n' "$cached"
+}
+
+# Compute a SHA-1 over the content of every .scala/.java/.proto file that
+# matters for this service. ~100 ms; called only on the slow path of
+# svc_src_changed and from the post-build stamp write.
+svc_source_hash() {
+ local svc="$1"
+ local dirs=()
+ local d=""
+ while IFS= read -r d; do
+ [[ -n "$d" ]] && dirs+=("$d")
+ done < <(_svc_src_dirs "$svc")
+ find "${dirs[@]}" \
+ \( -name "*.scala" -o -name "*.java" -o -name "*.proto" \) \
+ -type f -print0 2>/dev/null \
+ | sort -z \
+ | xargs -0 cat 2>/dev/null \
+ | shasum -a 1 \
+ | awk '{print $1}'
+}
+
+# Per-service dirty check (the SRC * indicator). Two-stage:
+# Fast path (~22 ms): is any tracked source newer than the stamp file's
+# mtime? If not, definitely clean.
+# Slow path (~100 ms): compute current source hash and compare to the hash
+# we stored at last build time. If they match, the
+# mtime moved without content moving (typical for git
+# checkout) — refresh the stamp mtime so we skip the
+# slow path next tick. If they differ, dirty.
+svc_src_changed() {
+ local svc="$1"
+ case "$(amap_get SVC_TYPE "$svc")" in
+ jvm)
+ local stamp="$BUILD_STAMP_DIR/$svc"
+ # Lazy seed: if we have a jar but no stamp, assume the jar matches
+ # current sources and seed with the hash. First REPL after a fresh
+ # checkout pays this once (~100 ms) and is clean afterwards.
+ if [[ ! -s "$stamp" ]]; then
+ local jar=""
+ if [[ "$svc" == "texera-web" || "$svc" == "computing-unit-master" ]]; then
+ jar="amber/target/amber-${TEXERA_VERSION}/lib/org.apache.texera.amber-${TEXERA_VERSION}.jar"
+ else
+ jar="target/${svc}-${TEXERA_VERSION}/lib/org.apache.texera.${svc}-${TEXERA_VERSION}.jar"
+ fi
+ [[ ! -f "$jar" ]] && return 0 # no jar, definitely dirty
+ svc_source_hash "$svc" > "$stamp"
+ return 1
+ fi
+
+ # Fast path: any tracked source mtime newer than stamp's mtime?
+ local dirs=() d=""
+ while IFS= read -r d; do
+ [[ -n "$d" ]] && dirs+=("$d")
+ done < <(_svc_src_dirs "$svc")
+ local newer=""
+ newer=$(find "${dirs[@]}" \
+ \( -name "*.scala" -o -name "*.java" -o -name "*.proto" \) \
+ -newer "$stamp" -type f -print 2>/dev/null | head -1)
+ if [[ -z "$newer" ]]; then
+ return 1 # nothing changed since last stamp → clean
+ fi
+
+ # Slow path: did the content actually change, or just mtimes?
+ local current_hash="" stored_hash=""
+ current_hash=$(svc_source_hash "$svc")
+ stored_hash=$(cat "$stamp" 2>/dev/null)
+ if [[ "$current_hash" == "$stored_hash" ]]; then
+ # Same content, just newer mtimes (git checkout, touch, etc.).
+ # Refresh stamp mtime to skip the slow path next tick.
+ touch "$stamp"
+ return 1
+ fi
+ return 0 # content really changed → dirty
+ ;;
+ yarn) needs_yarn_install ;;
+ bun) needs_bun_install ;;
+ docker) return 1 ;;
+ esac
+}
+
+build_all() {
+ BUILD_DID_RUN=false
+ local log="$LOG_DIR/sbt-dist.log"
+ if [[ "${FRESH:-false}" == "true" ]]; then
+ if tui_run_with_spinner "$log" "sbt clean dist ${DIM}(log: $log)${RESET}" \
+ sbt -no-colors clean dist; then
+ tui_ok "sbt: clean dist done"
+ BUILD_DID_RUN=true
+ else
+ tui_err "sbt: clean dist FAILED ${DIM}(tail -f $log)${RESET}"
+ return 1
+ fi
+ elif [[ "${BUILD:-auto}" == "auto" ]] && ! any_jvm_src_changed; then
+ tui_skip "sbt dist: skipped (no source changes since last build)"
+ return 0
+ else
+ if tui_run_with_spinner "$log" "sbt dist ${DIM}(log: $log)${RESET}" \
+ sbt -no-colors dist; then
+ tui_ok "sbt: dist done"
+ BUILD_DID_RUN=true
+ else
+ tui_err "sbt: dist FAILED ${DIM}(tail -f $log)${RESET}"
+ return 1
+ fi
+ fi
+ # Stop any running JVMs BEFORE unzip — overwriting jars under a live JVM
+ # corrupts its lazy class loads and the service silently dies later.
+ if [[ "$BUILD_DID_RUN" == "true" ]]; then
+ local svc="" pid=""
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "jvm" ]] || continue
+ pid=$(svc_running_pid "$svc")
+ [[ -z "$pid" ]] && continue
+ tui_step "$svc: pre-bouncing PID $pid (jars about to change)"
+ kill "$pid" 2>/dev/null || true
+ done
+ # Wait briefly for them to actually exit
+ local waited=0
+ while (( waited < 10 )); do
+ local still_up=0
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "jvm" ]] || continue
+ [[ -n "$(svc_running_pid "$svc")" ]] && still_up=$((still_up+1))
+ done
+ (( still_up == 0 )) && break
+ sleep 0.5
+ waited=$((waited+1))
+ done
+ fi
+ tui_step "unzipping dist artifacts"
+ local zip_glob="" unzip_dest=""
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "jvm" ]] || continue
+ zip_glob=$(amap_get SVC_ZIP_GLOB "$svc")
+ unzip_dest=$(amap_get SVC_UNZIP_DEST "$svc")
+ # Sibling services (empty ZIP_GLOB) share another service's dist —
+ # just stamp them as clean since the unzip already happened for the
+ # twin holding the build.
+ if [[ -z "$zip_glob" ]]; then
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc" 2>/dev/null || true
+ continue
+ fi
+ # shellcheck disable=SC2086
+ if unzip -oq ${zip_glob} -d "${unzip_dest}" 2>/dev/null; then
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc"
+ else
+ tui_warn "${zip_glob} not produced"
+ fi
+ done
+ tui_ok "artifacts unzipped"
+}
+
+refresh_node_deps() {
+ if ! is_skipped frontend; then
+ if [[ "${BUILD:-auto}" == "auto" ]] && ! needs_yarn_install; then
+ tui_skip "yarn install: skipped (lock unchanged)"
+ else
+ local log="$LOG_DIR/yarn-install.log"
+ if tui_run_with_spinner "$log" "yarn install (frontend) ${DIM}(log: $log)${RESET}" \
+ bash -c "cd frontend && yarn install"; then
+ tui_ok "yarn: deps refreshed"
+ else
+ tui_warn "yarn install failed ${DIM}(tail -f $log)${RESET}"
+ fi
+ fi
+ fi
+ if ! is_skipped agent-service; then
+ if [[ "${BUILD:-auto}" == "auto" ]] && ! needs_bun_install; then
+ tui_skip "bun install: skipped (lock unchanged)"
+ else
+ local log="$LOG_DIR/bun-install.log"
+ if tui_run_with_spinner "$log" "bun install (agent-service) ${DIM}(log: $log)${RESET}" \
+ bash -c "cd agent-service && bun install"; then
+ tui_ok "bun: deps refreshed"
+ else
+ tui_warn "bun install failed ${DIM}(tail -f $log)${RESET}"
+ fi
+ fi
+ fi
+}
+
+# --------- subcommands ---------
+cmd_status() {
+ local branch="" sha=""
+ branch=$(git -C "$REPO_ROOT" rev-parse --abbrev-ref HEAD 2>/dev/null || echo "?")
+ sha=$(git -C "$REPO_ROOT" rev-parse --short HEAD 2>/dev/null || echo "?")
+ tui_banner "Texera Local Dev" "branch: $branch @ $sha"
+
+ # One docker stats call up front — paying the ~2s docker-API cost once
+ # is cheaper than running it per docker service. Indexed by container
+ # name → "cpu%|mem" so the per-row formatting is just a lookup.
+ # `_DSTATS` is an amap (bash 3.2: no associative arrays).
+ while IFS='|' read -r name cpu mem; do
+ [[ -n "$name" ]] && amap_set _DSTATS "$name" "$cpu|$mem"
+ done < <(_docker_stats_snapshot)
+
+ printf "\n"
+ printf " ${BOLD}%-32s %-6s %-9s %-10s %-7s %-7s %s${RESET}\n" \
+ "SERVICE" "PORT" "PID" "UPTIME" "CPU%" "MEM" "STATE"
+ printf " ${GRAY}"
+ tui_hline "─" 32; printf " "
+ tui_hline "─" 6; printf " "
+ tui_hline "─" 9; printf " "
+ tui_hline "─" 10; printf " "
+ tui_hline "─" 7; printf " "
+ tui_hline "─" 7; printf " "
+ tui_hline "─" 12; printf "${RESET}\n"
+
+ local n_running=0 n_total=0 frontend_up=false
+ for svc in "${SERVICES[@]}"; do
+ n_total=$((n_total+1))
+ local pid="—" state="stopped" uptime="—" cpu="—" mem="—"
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ state=$(docker_svc_state "$svc")
+ if [[ "$state" == "running" ]]; then
+ local container="" dstat=""
+ container=$(_docker_container_for "$svc")
+ dstat=$(amap_get _DSTATS "$container")
+ if [[ -n "$dstat" ]]; then
+ cpu="${dstat%|*}"
+ mem="${dstat#*|}"
+ fi
+ uptime=$(svc_uptime "$svc")
+ fi
+ else
+ local found_pid=""
+ found_pid=$(svc_running_pid "$svc")
+ if [[ -n "$found_pid" ]]; then
+ state="running"; pid="$found_pid"
+ uptime=$(svc_uptime "$svc")
+ local cm=""
+ cm=$(svc_proc_cpu_mem "$svc")
+ cpu="${cm%|*}"
+ mem="${cm#*|}"
+ fi
+ fi
+ [[ "$state" == "running" ]] && n_running=$((n_running+1))
+ [[ "$svc" == "frontend" && "$state" == "running" ]] && frontend_up=true
+
+ local sym="" color=""
+ sym=$(tui_state_symbol "$state")
+ color=$(tui_state_color "$state")
+ printf " %s " "$sym"
+ printf "${color}%-32s${RESET} %-6s ${DIM}%-9s${RESET} %-10s %-7s %-7s ${color}%s${RESET}\n" \
+ "$svc" "$(amap_get SVC_PORT "$svc")" "$pid" "$uptime" "$cpu" "$mem" "$state"
+ done
+ printf "\n"
+
+ local summary_color="$YELLOW"
+ (( n_running == n_total )) && summary_color="$GREEN"
+ (( n_running == 0 )) && summary_color="$GRAY"
+ printf " ${BOLD}Status${RESET}: ${summary_color}%d of %d services running${RESET}\n" "$n_running" "$n_total"
+
+ printf "\n"
+ printf " ${CYAN}${SYM_LIST}${RESET} Logs: ${DIM}%s${RESET}\n" "$LOG_DIR/<service>.log"
+ printf " ${CYAN}${SYM_LIST}${RESET} Docker: ${DIM}docker compose -p %s ps${RESET}\n" "$DOCKER_PROJECT"
+ if $frontend_up; then
+ printf " ${CYAN}${SYM_LIST}${RESET} Open: ${BOLD}${GREEN}http://localhost:4200${RESET} ${GREEN}(frontend live)${RESET}\n"
+ else
+ printf " ${CYAN}${SYM_LIST}${RESET} Open: ${DIM}http://localhost:4200 (frontend not running yet)${RESET}\n"
+ fi
+
+ printf "\n ${BOLD}Common operations${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh up${RESET} ${DIM}# bring up the whole stack (build + start)${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh down${RESET} ${DIM}# stop every service${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh auto${RESET} ${DIM}# rebuild + bounce only the services whose source changed${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh <svc>${RESET} ${DIM}# rebuild that one JVM service and bounce it${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh start <svc>${RESET} ${DIM}# start one service without rebuilding${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh stop <svc>${RESET} ${DIM}# stop one service${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh logs <svc>${RESET} ${DIM}# tail a service's log${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh -i${RESET} ${DIM}# open the live TUI (needs Python + textual)${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh --help${RESET} ${DIM}# full reference${RESET}\n"
+ printf "\n"
+}
+
+cmd_up() {
+ SKIP_LIST=""
+ FRESH=false
+ BUILD=auto # auto (skip if no source change) | force | no
+ while [[ $# -gt 0 ]]; do
+ case "$1" in
+ --skip=*) SKIP_LIST="${1#--skip=}" ;;
+ --fresh) FRESH=true; BUILD=force ;;
+ --build) BUILD=force ;;
+ --no-build) BUILD=no ;;
+ *) tui_err "unknown flag: $1" >&2; exit 2 ;;
+ esac
+ shift
+ done
+
+ local n_skip=0
+ [[ -n "$SKIP_LIST" ]] && n_skip=$(echo "$SKIP_LIST" | tr ',' '\n' | wc -l | tr -d ' ')
+ local skip_label="none"
+ (( n_skip > 0 )) && skip_label="$n_skip service(s)"
+ tui_banner "Texera Local Dev — bringing stack up" "JDK 17 · skip=$skip_label · build=$BUILD"
+
+ # ── Pre-flight short-circuit ───────────────────────────────────────────
+ # If nothing's changed AND every service is already running, just say so
+ # and exit. Saves the user from scrolling through 30+ "already running"
+ # lines for the common "I just want to check" case.
+ if [[ "$BUILD" == "auto" && -z "$SKIP_LIST" ]]; then
+ local nothing_to_build=true
+ any_jvm_src_changed && nothing_to_build=false
+ needs_yarn_install && nothing_to_build=false
+ needs_bun_install && nothing_to_build=false
+
+ local all_running=true svc=""
+ for svc in "${SERVICES[@]}"; do
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ [[ "$(docker_svc_state "$svc")" == "running" ]] || { all_running=false; break; }
+ else
+ [[ -n "$(svc_running_pid "$svc")" ]] || { all_running=false; break; }
+ fi
+ done
+
+ if $nothing_to_build && $all_running; then
+ tui_section "Pre-flight"
+ tui_ok "no source/lock changes since last build"
+ tui_ok "all ${#SERVICES[@]} services already running"
+ printf "\n ${BOLD}${GREEN}${SYM_OK} nothing to do${RESET} ${DIM}(use \`u --build\` to force a rebuild, or \`<svc>\` to bounce just one)${RESET}\n\n"
+ return 0
+ fi
+ fi
+
+ if [[ "$BUILD" != "no" ]]; then
+ tui_section "Build"
+ build_all
+ refresh_node_deps
+ else
+ tui_section "Build"
+ tui_skip "build: --no-build (using existing artifacts)"
+ fi
+
+ # ▸ Services -- docker compose has its own TTY panel; native services we
+ # kick off silently in the background, then a single redrawing panel below
+ # shows progress for ALL of them.
+ tui_section "Services ${DIM}(launching)${RESET}"
+ local svc="" cwd="" log="" type="" launcher=""
+
+ # One project-level `docker compose up -d` for every non-skipped docker
+ # row — much faster than five separate calls.
+ local has_docker_targets=false
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]] || continue
+ is_skipped "$svc" && continue
+ has_docker_targets=true
+ break
+ done
+ if $has_docker_targets; then
+ case "$(infra_state)" in
+ running) tui_ok "infra: already running" ;;
+ external:*) tui_err "infra: ports taken by non-script containers"
+ printf " ${DIM}docker compose -p texera-dev down${RESET}\n" ;;
+ *) infra_up || true ;;
+ esac
+ # Whether infra is fresh or already up, make sure the texera_db
+ # schema is current — JVMs about to start expect it (jOOQ + Iceberg
+ # both need real tables).
+ infra_ensure_db_schema
+ fi
+
+ for svc in "${SERVICES[@]}"; do
+ is_skipped "$svc" && { tui_skip "$svc: --skip"; continue; }
+ type=$(amap_get SVC_TYPE "$svc")
+ [[ "$type" == "docker" ]] && continue # already handled by infra_up
+ if [[ -n "$(svc_running_pid "$svc")" ]]; then
+ tui_ok "$svc: already running ${DIM}(PID $(svc_running_pid "$svc"))${RESET}"
+ continue
+ fi
+ cwd=$(amap_get SVC_CWD "$svc")
+ log="$LOG_DIR/$svc.log"
+ tui_step "$svc: launching → ${DIM}$log${RESET}"
+ case "$type" in
+ jvm)
+ launcher=$(amap_get SVC_LAUNCHER "$svc")
+ if [[ ! -x "$cwd/$launcher" ]]; then
+ tui_err "$svc: launcher missing at $cwd/$launcher"
+ continue
+ fi
+ ( cd "$cwd" && nohup "./$launcher" >"$log" 2>&1 </dev/null & ) ;;
+ bun) ( cd "$cwd" && nohup bun run dev >"$log" 2>&1 </dev/null & ) ;;
+ yarn) ( cd "$cwd" && nohup yarn start >"$log" 2>&1 </dev/null & ) ;;
+ esac
+ done
+
+ tui_section "Health ${DIM}(refreshing in place)${RESET}"
+ local ec=0
+ tui_wait_panel || ec=$?
+
+ printf "\n"
+ local ok=0 total=0 failed=0
+ for svc in "${SERVICES[@]}"; do
+ is_skipped "$svc" && continue
+ total=$((total+1))
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ case "$(docker_svc_state "$svc")" in
+ running|exited) ok=$((ok+1)) ;;
+ *) failed=$((failed+1)) ;;
+ esac
+ else
+ [[ -n "$(svc_running_pid "$svc")" ]] && ok=$((ok+1)) || failed=$((failed+1))
+ fi
+ done
+ if (( failed == 0 )); then
+ printf " ${BOLD}${GREEN}${SYM_OK} %d of %d services healthy${RESET}\n" "$ok" "$total"
+ else
+ printf " ${BOLD}${YELLOW}${SYM_WARN} %d of %d services healthy${RESET} ${RED}(%d failed)${RESET}\n" \
+ "$ok" "$total" "$failed"
+ fi
+ printf "\n"
+
+ cmd_status
+ [[ $ec -eq 0 ]]
+}
+
+# `auto`: the minimal "make the running services match my current source" path.
+# Walks every service, identifies what's actually dirty (content-hash for JVM,
+# lock mtime for yarn/bun, never for docker), and only touches those:
+# - dirty JVM, currently running → rebuild + bounce
+# - dirty JVM, currently stopped → rebuild only (don't auto-start)
+# - dirty yarn (frontend lock) → yarn install + bounce frontend if up
+# - dirty bun (agent-service lock) → bun install (bun --watch reloads itself)
+# Clean services are left alone — no pre-bounce, no restart.
+cmd_auto() {
+ SKIP_LIST=""
+ while [[ $# -gt 0 ]]; do
+ case "$1" in
+ --skip=*) SKIP_LIST="${1#--skip=}" ;;
+ *) tui_err "unknown flag: $1" >&2; exit 2 ;;
+ esac
+ shift
+ done
+
+ tui_banner "Texera Local Dev — auto bounce" \
+ "rebuild + bounce only what changed since last build"
+
+ # ── Scan ──────────────────────────────────────────────────────────────
+ tui_section "Scan"
+ local svc=""
+ local dirty_jvms=()
+ local need_yarn=false
+ local need_bun=false
+ for svc in "${SERVICES[@]}"; do
+ is_skipped "$svc" && continue
+ case "$(amap_get SVC_TYPE "$svc")" in
+ jvm)
+ if svc_src_changed "$svc"; then
+ dirty_jvms+=("$svc")
+ tui_warn "$svc: source changed since last build"
+ fi ;;
+ yarn)
+ if needs_yarn_install; then
+ need_yarn=true
+ tui_warn "frontend: yarn.lock newer than node_modules — needs install"
+ fi ;;
+ bun)
+ if needs_bun_install; then
+ need_bun=true
+ tui_warn "agent-service: bun.lock newer than node_modules — needs install"
+ fi ;;
+ esac
+ done
+
+ if (( ${#dirty_jvms[@]} == 0 )) && ! $need_yarn && ! $need_bun; then
+ tui_ok "everything up-to-date — nothing to bounce"
+ printf "\n"
+ return 0
+ fi
+
+ # ── Build ─────────────────────────────────────────────────────────────
+ # One `sbt dist` covers every dirty JVM in a single sbt invocation; sbt's
+ # own incremental compiler only recompiles the subprojects that need it,
+ # and we only unzip + bounce the dirty ones below — clean services don't
+ # get pre-bounced just because the build ran.
+ if (( ${#dirty_jvms[@]} > 0 )); then
+ tui_section "Build ${DIM}(${#dirty_jvms[@]} JVM service(s) dirty)${RESET}"
+ # Mark each dirty service as "building" so the TUI shows the
+ # animation across the whole sbt window (~30s+). Without this the
+ # dashboard stays on the prior STATE during the slow build.
+ local _s=""
+ for _s in "${dirty_jvms[@]}"; do
+ phase_set "$_s" building
+ done
+ local log="$LOG_DIR/sbt-dist.log"
+ if ! tui_run_with_spinner "$log" "sbt dist ${DIM}(log: $log)${RESET}" \
+ sbt -no-colors dist; then
+ for _s in "${dirty_jvms[@]}"; do
+ phase_clear "$_s"
+ done
+ tui_err "sbt dist failed ${DIM}(tail -f $log)${RESET}"
+ return 1
+ fi
+ tui_ok "sbt: dist done"
+ fi
+
+ # ── Bounce dirty JVMs ────────────────────────────────────────────────
+ # Two passes so siblings can coexist:
+ # 1) stop + (maybe-)unzip + stamp every dirty service
+ # 2) start the ones that were running when we entered
+ #
+ # Pass 1 needs to finish before pass 2 because computing-unit-master
+ # shares amber's dist with texera-web: cu-master has empty SVC_ZIP_GLOB
+ # / SVC_UNZIP_DEST, so its "rebuild" is just waiting for texera-web's
+ # iteration to unzip amber. If we kept the previous single-pass loop
+ # we'd `continue` past start_one whenever ZIP_GLOB was empty and the
+ # sibling never came back up.
+ local n_bounced=0 n_rebuilt=0
+ # `_was_running` is an amap (bash 3.2: no associative arrays). Keyed by
+ # svc → the PID we observed at the start of pass 1, "" if not running.
+ # Cleared with `unset` at the top so a re-invocation (e.g. interactive
+ # REPL would have, if we still had one) starts fresh.
+ local _wr_var=""
+ for _wr_var in $(compgen -v _amap___was_running__ 2>/dev/null); do
+ unset "$_wr_var"
+ done
+ if (( ${#dirty_jvms[@]} > 0 )); then
+ tui_section "Bounce"
+ # Pass 1: stop running pids; unzip own dist if we have one.
+ for svc in "${dirty_jvms[@]}"; do
+ local pid=""
+ pid=$(svc_running_pid "$svc")
+ amap_set _was_running "$svc" "$pid"
+ if [[ -n "$pid" ]]; then
+ # Flip the dashboard from `building` to `stopping` while we
+ # SIGTERM/SIGKILL the JVM, then back to `building` for the
+ # unzip step below. Without these the user only sees one
+ # state during the whole bounce.
+ phase_set "$svc" stopping
+ tui_step "$svc: stopping PID $pid before unzip"
+ kill "$pid" 2>/dev/null || true
+ local i=0
+ while (( i < 30 )) && kill -0 "$pid" 2>/dev/null; do
+ sleep 0.5
+ i=$((i+1))
+ done
+ kill -0 "$pid" 2>/dev/null && kill -9 "$pid" 2>/dev/null
+ fi
+ local zip_glob="" unzip_dest=""
+ zip_glob=$(amap_get SVC_ZIP_GLOB "$svc")
+ unzip_dest=$(amap_get SVC_UNZIP_DEST "$svc")
+ if [[ -z "$zip_glob" ]]; then
+ # Sibling service (e.g. computing-unit-master): no own dist
+ # to unzip — its launcher comes from the twin's unzip later
+ # in this pass. Just stamp it clean.
+ phase_set "$svc" building
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc" 2>/dev/null || true
+ n_rebuilt=$((n_rebuilt+1))
+ continue
+ fi
+ phase_set "$svc" building
+ # shellcheck disable=SC2086
+ if unzip -oq ${zip_glob} -d "${unzip_dest}" 2>/dev/null; then
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc"
+ n_rebuilt=$((n_rebuilt+1))
+ else
+ tui_warn "$svc: ${zip_glob} not produced — skipping"
+ fi
+ done
+ # amber's two siblings (texera-web + computing-unit-master) are
+ # *intended* to run together — they share the dist and they're
+ # both required for a working stack. If either was running before
+ # the auto pass, treat the whole group as "should be running" so
+ # the other doesn't end up silently dead just because it had
+ # already crashed / been left stopped from an earlier session.
+ local AMBER_SIBLINGS=(texera-web computing-unit-master)
+ local amber_group_active=false
+ local s=""
+ for s in "${AMBER_SIBLINGS[@]}"; do
+ [[ -n "$(amap_get _was_running "$s")" ]] && amber_group_active=true
+ done
+
+ # Pass 2: start the ones that had been running, plus any amber
+ # sibling that is dirty when the group is "active". By now every
+ # sibling's launcher is on disk (the twin's unzip in pass 1
+ # populated `amber/target/amber-<VERSION>/bin/*`).
+ for svc in "${dirty_jvms[@]}"; do
+ local should_start=false
+ if [[ -n "$(amap_get _was_running "$svc")" ]]; then
+ should_start=true
+ elif $amber_group_active && { [[ "$svc" == "texera-web" ]] || [[ "$svc" == "computing-unit-master" ]]; }; then
+ should_start=true
+ tui_step "$svc: was stopped but its sibling is active — starting too"
+ fi
+ if $should_start; then
+ start_one "$svc"
+ n_bounced=$((n_bounced+1))
+ else
+ # Pass 1 left phase=building set so the dashboard animates
+ # during sbt + unzip; clear it now since we won't be
+ # starting (otherwise the row spins building… until the
+ # 90s stale rule kicks in).
+ phase_clear "$svc"
+ tui_skip "$svc: was stopped — rebuilt but not started"
+ fi
+ done
+ fi
+
+ # ── Node deps ────────────────────────────────────────────────────────
+ if $need_yarn; then
+ tui_section "Frontend deps"
+ local log="$LOG_DIR/yarn-install.log"
+ if tui_run_with_spinner "$log" "yarn install ${DIM}(log: $log)${RESET}" \
+ bash -c "cd frontend && yarn install"; then
+ tui_ok "yarn: deps refreshed"
+ # ng serve doesn't pick up dependency-tree changes from a running
+ # process; bounce if it was up.
+ if [[ -n "$(svc_running_pid frontend)" ]]; then
+ stop_one frontend
+ start_one frontend
+ n_bounced=$((n_bounced+1))
+ else
+ tui_skip "frontend: was stopped — deps refreshed but not started"
+ fi
+ else
+ tui_err "yarn install failed ${DIM}(tail -f $log)${RESET}"
+ fi
+ fi
+
+ if $need_bun; then
+ tui_section "Agent-service deps"
+ local log="$LOG_DIR/bun-install.log"
+ if tui_run_with_spinner "$log" "bun install ${DIM}(log: $log)${RESET}" \
+ bash -c "cd agent-service && bun install"; then
+ tui_ok "bun: deps refreshed"
+ # bun --watch reloads itself when node_modules changes; no manual
+ # bounce needed.
+ if [[ -n "$(svc_running_pid agent-service)" ]]; then
+ tui_skip "agent-service: bun --watch will reload"
+ else
+ tui_skip "agent-service: was stopped — deps refreshed but not started"
+ fi
+ else
+ tui_err "bun install failed ${DIM}(tail -f $log)${RESET}"
+ fi
+ fi
+
+ # ── Summary + final dashboard ────────────────────────────────────────
+ printf "\n"
+ printf " ${BOLD}${GREEN}${SYM_OK} auto bounce done${RESET}: %d rebuilt, %d bounced\n\n" \
+ "$n_rebuilt" "$n_bounced"
+ cmd_status
+}
+
+cmd_down() {
+ SKIP_LIST=""
+ while [[ $# -gt 0 ]]; do
+ case "$1" in
+ --skip=*) SKIP_LIST="${1#--skip=}" ;;
+ *) tui_err "unknown flag: $1" >&2; exit 2 ;;
+ esac
+ shift
+ done
+ tui_banner "Texera Local Dev — stopping stack" "skip=${SKIP_LIST:-none}"
+ tui_section "Stopping (reverse order)"
+ local svc=""
+ # Stop native services first (reverse declaration order)
+ # bash arrays are 0-indexed: last element is at N-1.
+ for (( i=${#SERVICES[@]} - 1; i>=0; i-- )); do
+ svc="${SERVICES[i]}"
+ [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]] && continue
+ if is_skipped "$svc"; then
+ tui_skip "$svc: --skip"
+ continue
+ fi
+ stop_one "$svc"
+ done
+ # Then one project-level docker compose down for any docker target.
+ local has_docker_targets=false
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]] || continue
+ if is_skipped "$svc"; then
+ tui_skip "$svc: --skip"
+ continue
+ fi
+ has_docker_targets=true
+ done
+ $has_docker_targets && infra_down
+ printf "\n"
+}
+
+cmd_update_one() {
+ local svc="$1"
+ if ! amap_has SVC_TYPE "$svc"; then
+ tui_err "unknown service: ${BOLD}$svc${RESET}"
+ printf " ${DIM}Known:${RESET} ${SERVICES[*]}\n"
+ exit 1
+ fi
+ local type=""
+ type=$(amap_get SVC_TYPE "$svc")
+ case "$type" in
+ docker)
+ tui_banner "Restarting ${svc}" "docker compose restart $svc"
+ tui_step "$svc: docker compose restart $svc"
+ docker compose -p "$DOCKER_PROJECT" restart "$svc" >/dev/null 2>&1 \
+ && tui_ok "$svc: restarted" \
+ || { tui_err "$svc: restart failed"; exit 1; }
+ exit 0
+ ;;
+ yarn)
+ tui_warn "frontend uses ng's watch -- source changes hot-reload automatically."
+ printf " ${DIM}For dep changes: kill PID ${RESET}$(svc_running_pid frontend)${DIM}; then bin/local-dev.sh up${RESET}\n"
+ exit 0
+ ;;
+ bun)
+ tui_banner "Updating ${svc}" "bun install + bounce"
+ tui_section "Deps"
+ ( cd "$(amap_get SVC_CWD "$svc")" && bun install )
+ tui_section "Bounce"
+ stop_one "$svc"
+ start_one "$svc"
+ ;;
+ jvm)
+ local _sbt_proj=""
+ _sbt_proj=$(amap_get SVC_SBT "$svc")
+ if [[ -n "$_sbt_proj" ]]; then
+ tui_banner "Updating ${svc}" "sbt ${_sbt_proj}/dist + bounce"
+ else
+ tui_banner "Updating ${svc}" "bounce only (shares dist with its sibling)"
+ fi
+ tui_section "Build"
+ build_one_jvm "$svc"
+ tui_section "Bounce"
+ # amber's two siblings share a dist — rebuilding one moves the
+ # jar bytes underneath the other. Always bounce them together
+ # so neither ends up running stale code (or silently dead).
+ local sibling=""
+ case "$svc" in
+ texera-web) sibling="computing-unit-master" ;;
+ computing-unit-master) sibling="texera-web" ;;
+ esac
+ local sibling_was_running=false
+ if [[ -n "$sibling" ]] && [[ -n "$(svc_running_pid "$sibling")" ]]; then
+ sibling_was_running=true
+ tui_step "$sibling: stopping (shares amber dist with $svc)"
+ stop_one "$sibling"
+ fi
+ stop_one "$svc"
+ start_one "$svc"
+ if $sibling_was_running; then
+ start_one "$sibling"
+ fi
+ ;;
+ esac
+ tui_section "Health"
+ local _port=""
+ _port=$(amap_get SVC_PORT "$svc")
+ if wait_for_port "$_port" 60; then
+ printf " ${GREEN}${SYM_OK}${RESET} %-32s ${DIM}:%s${RESET}\n" "$svc" "$_port"
+ else
+ printf " ${RED}${SYM_ERR}${RESET} %-32s ${DIM}:%s${RESET} ${RED}timeout${RESET} ${DIM}(bin/local-dev.sh logs %s)${RESET}\n" \
+ "$svc" "$_port" "$svc"
+ exit 1
+ fi
+ printf "\n"
+}
+
+cmd_logs() {
+ local svc="${1:?usage: bin/local-dev.sh logs <service>}"
+ if ! amap_has SVC_TYPE "$svc"; then
+ echo "Unknown service: $svc" >&2
+ exit 1
+ fi
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ exec docker compose -p "$DOCKER_PROJECT" logs -f "$svc"
+ fi
+ exec tail -f "$LOG_DIR/$svc.log"
+}
+
+# Render the interactive dashboard panel (banner + service table + hint + summary).
+tui_render_dashboard() {
+ printf "\e[2J\e[H" # clear screen + home cursor (scrollback preserved)
+ local branch="" sha=""
+ branch=$(git -C "$REPO_ROOT" rev-parse --abbrev-ref HEAD 2>/dev/null || echo "?")
+ sha=$(git -C "$REPO_ROOT" rev-parse --short HEAD 2>/dev/null || echo "?")
+ tui_banner "Texera Local Dev — interactive" "branch: $branch @ $sha · $(date '+%H:%M:%S') · type ? for help"
+ printf "\n"
+
+ printf " ${BOLD}%-32s %-7s %-9s %-18s %-3s %s${RESET}\n" \
+ "SERVICE" "PORT" "PID" "ARTIFACT MTIME" "SRC" "STATE"
+ printf " ${GRAY}"; tui_hline "─" 32; printf " "
+ tui_hline "─" 7; printf " "; tui_hline "─" 9; printf " "
+ tui_hline "─" 18; printf " "; tui_hline "─" 3; printf " "
+ tui_hline "─" 12; printf "${RESET}\n"
+
+ local n_run=0 n_total=0 n_dirty=0
+ local svc=""
+ for svc in "${SERVICES[@]}"; do
+ n_total=$((n_total+1))
+ local pid="—" state="stopped" mtime="—" port_str="—" src_disp=" "
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ state=$(docker_svc_state "$svc")
+ mtime="docker"
+ port_str=":$(amap_get SVC_PORT "$svc")"
+ else
+ local found_pid=""
+ found_pid=$(svc_running_pid "$svc")
+ if [[ -n "$found_pid" ]]; then state="running"; pid="$found_pid"; fi
+ mtime=$(svc_artifact_mtime "$svc")
+ port_str=":$(amap_get SVC_PORT "$svc")"
+ fi
+ [[ "$state" == "running" ]] && n_run=$((n_run+1))
+
+ if svc_src_changed "$svc" 2>/dev/null; then
+ src_disp="${YELLOW}${BOLD}*${RESET} "
+ n_dirty=$((n_dirty+1))
+ fi
+
+ local sym="" color=""
+ sym=$(tui_state_symbol "$state")
+ color=$(tui_state_color "$state")
+ printf " %s " "$sym"
+ printf "${color}%-32s${RESET} %-7s ${DIM}%-9s${RESET} %-18s %s ${color}%s${RESET}\n" \
+ "$svc" "$port_str" "$pid" "$mtime" "$src_disp" "$state"
+ done
+
+ printf "\n"
+ local sum_color="$YELLOW"
+ (( n_run == n_total )) && sum_color="$GREEN"
+ (( n_run == 0 )) && sum_color="$GRAY"
+ printf " ${BOLD}${sum_color}%d of %d running${RESET}" "$n_run" "$n_total"
+ if (( n_dirty > 0 )); then
+ printf " ${YELLOW}${BOLD}*${RESET} ${DIM}%d with source changes${RESET}" "$n_dirty"
+ fi
+ printf "\n\n"
+ printf " ${DIM}Commands:${RESET} "
+ printf "${BOLD}r${RESET}efresh${DIM} (or just ↩)${RESET} · "
+ printf "${BOLD}u${RESET}p · ${BOLD}d${RESET}own · "
+ printf "${BOLD}b${RESET}uild · "
+ printf "${BOLD}<svc>${RESET}${DIM}=rebuild+bounce${RESET} · "
+ printf "${BOLD}l${RESET}ogs ${DIM}<svc>${RESET} · "
+ printf "${BOLD}s${RESET}top ${DIM}<svc>${RESET} · "
+ printf "${BOLD}q${RESET}uit\n\n"
+}
+
+# Pure monitoring mode: redraw the dashboard every $1 seconds, no prompt.
+# Ctrl-C to exit. Useful when watching a build/restart from another terminal.
+cmd_watch() {
+ if [[ ! -t 1 ]]; then
+ tui_err "watch mode requires a TTY"
+ exit 1
+ fi
+ local interval="${1:-2}"
+ trap 'printf "\e[?25h\n${DIM}bye${RESET}\n"; exit 0' EXIT INT TERM
+ printf "\e[?25l" # hide cursor
+ while true; do
+ tui_render_dashboard
+ printf " ${DIM}watch: refreshing every %ss · Ctrl-C to exit${RESET}\n" "$interval"
+ sleep "$interval"
+ done
+}
+
+# Pause and let the user read command output before re-rendering the dashboard.
+
+# Print the ordered list of Python interpreters we consider for launching the
+# Textual TUI: an explicit override, then any active venv, then the canonical
+# texera dev venv, then whatever `python3`/`python` happen to resolve to. We
+# de-duplicate as we go so the diagnostic doesn't show the same path twice.
+_probed_pythons() {
+ local seen=""
+ local cand=""
+ local raw=(
+ "${TEXERA_PYTHON:-}"
+ "${VIRTUAL_ENV:+$VIRTUAL_ENV/bin/python}"
+ "$(command -v python3 2>/dev/null)"
+ "$(command -v python 2>/dev/null)"
+ )
+ for cand in "${raw[@]}"; do
+ [[ -z "$cand" ]] && continue
+ case ":$seen:" in *":$cand:"*) continue ;; esac
+ seen="$seen:$cand"
+ printf '%s\n' "$cand"
+ done
+}
+
+# Walk `_probed_pythons` and return the first interpreter where `import
+# textual` succeeds, or empty string if none.
+_find_python_with_textual() {
+ local cand=""
+ while IFS= read -r cand; do
+ [[ -x "$cand" ]] || continue
+ if "$cand" -c "import textual" >/dev/null 2>&1; then
+ printf '%s\n' "$cand"
+ return 0
+ fi
+ done < <(_probed_pythons)
+ return 1
+}
+
+# Show every candidate Python the picker considered, its version, and
+# whether it can `import textual`. Pinpoints "I have the right python on
+# PATH but textual isn't installed THERE" vs "no python found at all".
+_diagnose_python() {
+ printf "\n ${BOLD}candidate interpreters:${RESET}\n"
+ local any=0
+ while IFS= read -r cand; do
+ any=1
+ if [[ ! -x "$cand" ]]; then
+ printf " ${RED}✗${RESET} %s ${DIM}(not executable)${RESET}\n" "$cand"
+ continue
+ fi
+ local ver=""
+ ver=$("$cand" --version 2>&1 | head -1) || ver="(unreadable)"
+ if "$cand" -c "import textual; print(textual.__version__)" >/dev/null 2>&1; then
+ local tv=""
+ tv=$("$cand" -c "import textual; print(textual.__version__)" 2>/dev/null) || tv="?"
+ printf " ${GREEN}✓${RESET} %s ${DIM}(%s, textual %s)${RESET}\n" "$cand" "$ver" "$tv"
+ else
+ printf " ${YELLOW}!${RESET} %s ${DIM}(%s, textual MISSING)${RESET}\n" "$cand" "$ver"
+ fi
+ done < <(_probed_pythons)
+ if (( any == 0 )); then
+ printf " ${RED}✗${RESET} ${DIM}no candidate python on PATH or in TEXERA_PYTHON / VIRTUAL_ENV${RESET}\n"
+ fi
+ printf "\n ${BOLD}env:${RESET}\n"
+ printf " \$TEXERA_PYTHON = ${DIM}%s${RESET}\n" "${TEXERA_PYTHON:-(unset)}"
+ printf " \$VIRTUAL_ENV = ${DIM}%s${RESET}\n" "${VIRTUAL_ENV:-(unset)}"
+ printf "\n"
+}
+
+# Hand off to the Python + Textual TUI. Hard requirement now (no more zsh
+# REPL fallback) — if we can't find a Python with `textual` installed,
+# print install instructions and exit non-zero. Use the non-interactive
+# `status` (or any other subcommand) when you don't have Python set up.
+cmd_interactive() {
+ if [[ ! -t 0 || ! -t 1 ]]; then
+ tui_err "interactive mode requires a TTY"
+ exit 1
+ fi
+ # zsh + `set -e` kills the script when a command substitution's command
+ # exits non-zero (`var=$(false)` → silent abort). Suppress with || true
+ # so the error path below actually gets to print the install hint.
+ local py=""
+ py="$(_find_python_with_textual)" || true
+ if [[ -z "$py" ]]; then
+ tui_err "interactive mode requires Python with the ${BOLD}textual${RESET} package"
+ _diagnose_python
+ _install_hint python
+ exit 1
+ fi
+ exec "$py" "$REPO_ROOT/bin/local-dev-tui.py"
+}
+
+# --------- main ---------
+# Warm the per-service source-dir cache so dirty checks below skip the
+# repeated BFS over the sbt graph. ~5ms of one-time cost; saves N×BFS
+# work per `cmd_up` / `cmd_auto` / status loop.
+_precompute_src_dirs
+
+case "${1:-}" in
+ ""|status) cmd_status ;; # default: one-shot dashboard (safe in scripts/CI)
+ -i|--interactive) cmd_interactive ;; # opt in to the live TUI
+ up) shift; cmd_up "$@" ;;
+ auto) shift; cmd_auto "$@" ;;
+ down) shift; cmd_down "$@" ;;
+ start) shift; start_one "${1:?need service name}" ;;
+ stop) shift; stop_one "${1:?need service name}" ;;
+ logs) shift; cmd_logs "${1:-}" ;;
+ w|watch) shift; cmd_watch "${1:-2}" ;;
+ version) printf "%s\n" "$TEXERA_VERSION" ;;
+ -h|--help) sed -n '18,67p' "$0" ;;
+ *) cmd_update_one "$1" ;;
+esac
diff --git a/bin/local-dev/docker-compose.override.yml b/bin/local-dev/docker-compose.override.yml
new file mode 100644
index 0000000000..94156e9e56
--- /dev/null
+++ b/bin/local-dev/docker-compose.override.yml
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Overlay for bin/local-dev.sh on top of bin/single-node/docker-compose.yml.
+# bin/single-node was designed so the texera microservices run inside the same
+# compose network as the infra — postgres/lakefs/lakekeeper/litellm therefore
+# don't publish their ports to the host. The local-dev flow runs the JVM and
+# Node services *natively* on the host, so we expose each backend port here.
+
+services:
+ postgres:
+ ports:
+ - "5432:5432"
+
+ lakefs:
+ ports:
+ - "8000:8000"
+
+ lakekeeper:
+ ports:
+ - "8181:8181"
+
+ litellm:
+ ports:
+ - "4000:4000"
+
+ # The default .env hardcodes STORAGE_S3_ENDPOINT=http://texera-minio:9000
+ # — only routable inside the docker compose network. lakekeeper-init bakes
+ # that string into the warehouse's storage profile, which Lakekeeper then
+ # both dials itself (for s3 ops) AND returns to clients. The host-native
+ # JVMs in local-dev mode can't resolve `texera-minio`.
+ #
+ # Use whatever STORAGE_S3_ENDPOINT the parent process exported. The
+ # wrapper script (bin/local-dev.sh) detects the host's LAN IP at startup
+ # and exports e.g. http://192.168.1.x:9000 — reachable from both sides:
+ # • lakekeeper container: out via docker bridge → host's published :9000
+ # • host JVM: straight to host's published :9000
+ # If the env wasn't exported (script not used), fall back to localhost
+ # which at least keeps the warehouse creation from blowing up.
+ lakekeeper-init:
+ environment:
+ STORAGE_S3_ENDPOINT: ${STORAGE_S3_ENDPOINT:-http://localhost:9000}
diff --git a/amber/dev-requirements.txt b/bin/local-dev/tests/__init__.py
similarity index 56%
copy from amber/dev-requirements.txt
copy to bin/local-dev/tests/__init__.py
index 4ab14960cb..13a83393a9 100644
--- a/amber/dev-requirements.txt
+++ b/bin/local-dev/tests/__init__.py
@@ -14,21 +14,3 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-
-# Test- and dev-only Python dependencies. Installed in CI after the
-# LICENSE-binary snapshot is taken, so packages here never appear in
-# pip-licenses output and never need to be tracked in LICENSE-binary
-# / NOTICE-binary. Not installed by packaging.
-
-# pytest test runner and plugins.
-pytest==9.0.3
-pytest-reraise==2.1.2
-pytest-timeout==2.4.0
-
-# Coverage instrumentation for pytest; emits coverage.xml consumed by
-# Codecov's Phase 1 upload.
-pytest-cov==7.1.0
-
-# protoc plugin for bin/python-proto-gen.sh; runtime needs only the
-# base `betterproto` in requirements.txt.
-betterproto[compiler]==2.0.0b7
diff --git a/bin/local-dev/tests/conftest.py b/bin/local-dev/tests/conftest.py
new file mode 100644
index 0000000000..7525cf3a87
--- /dev/null
+++ b/bin/local-dev/tests/conftest.py
@@ -0,0 +1,44 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""pytest fixtures for the local-dev TUI tests.
+
+The TUI lives as `bin/local-dev-tui.py` rather than a package, so we load it
+once here with `importlib.util` and expose the resulting module via the
+`tui` fixture. `sys.modules` registration is required so the module's
+`@dataclass` decorators can resolve `__module__` lookups."""
+
+from __future__ import annotations
+
+import importlib.util
+import sys
+from pathlib import Path
+
+import pytest
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+TUI_PATH = REPO_ROOT / "bin" / "local-dev-tui.py"
+
+
+@pytest.fixture(scope="session")
+def tui():
+ spec = importlib.util.spec_from_file_location("local_dev_tui", TUI_PATH)
+ assert spec is not None and spec.loader is not None
+ module = importlib.util.module_from_spec(spec)
+ sys.modules["local_dev_tui"] = module
+ spec.loader.exec_module(module)
+ return module
diff --git a/bin/local-dev/tests/test_local_dev_sh.sh b/bin/local-dev/tests/test_local_dev_sh.sh
new file mode 100755
index 0000000000..e467259b13
--- /dev/null
+++ b/bin/local-dev/tests/test_local_dev_sh.sh
@@ -0,0 +1,167 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Smoke tests for bin/local-dev.sh. Run from the repo root:
+# bash bin/local-dev/tests/test_local_dev_sh.sh
+# Exits 0 if every check passes, 1 otherwise.
+#
+# Kept deliberately small: bringing up the actual stack needs Docker /
+# sbt / a Mac and is out of scope for CI here. We cover the things that
+# regress quietly — script syntax, version-detection, the subcommand
+# dispatch, and graceful failure on garbage input.
+
+set -u
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
+SCRIPT="$REPO_ROOT/bin/local-dev.sh"
+
+PASS=0
+FAIL=0
+
+_pass() { printf " \e[32m✓\e[0m %s\n" "$1"; PASS=$((PASS+1)); }
+_fail() {
+ printf " \e[31m✗\e[0m %s\n" "$1"
+ [[ $# -ge 2 ]] && printf " %s\n" "$2"
+ FAIL=$((FAIL+1))
+}
+
+# 1) bash -n: syntax check. Catches everything from typos to unbalanced
+# heredocs without executing a line of the script.
+if bash -n "$SCRIPT" 2>/tmp/.local-dev-syntax.err; then
+ _pass "bash -n bin/local-dev.sh"
+else
+ _fail "bash -n bin/local-dev.sh" "$(cat /tmp/.local-dev-syntax.err)"
+fi
+rm -f /tmp/.local-dev-syntax.err
+
+# 2) `version` subcommand returns the same string we'd extract by hand
+# from build.sbt. This is the single source of truth that all the
+# dist / launcher / canary-jar paths in the script and the TUI build
+# off of, so we'd rather catch a regression here.
+script_version=$("$SCRIPT" version 2>/dev/null | head -1 | tr -d '[:space:]')
+sbt_version=$(
+ grep -E '^[[:space:]]*ThisBuild[[:space:]]*/[[:space:]]*version[[:space:]]*:=[[:space:]]*"' \
+ "$REPO_ROOT/build.sbt" 2>/dev/null \
+ | head -1 \
+ | sed -E 's/.*"([^"]+)".*/\1/' \
+ | tr -d '[:space:]'
+)
+if [[ -n "$script_version" && "$script_version" == "$sbt_version" ]]; then
+ _pass "version matches build.sbt ($script_version)"
+else
+ _fail "version mismatch" "script=$script_version build.sbt=$sbt_version"
+fi
+
+# 3) TEXERA_VERSION env var should override.
+override=$(TEXERA_VERSION="9.9.9-TEST" "$SCRIPT" version 2>/dev/null | head -1 | tr -d '[:space:]')
+if [[ "$override" == "9.9.9-TEST" ]]; then
+ _pass "TEXERA_VERSION env var overrides build.sbt"
+else
+ _fail "env override didn't take" "got: $override"
+fi
+
+# 4) `--help` prints usage.
+help_out=$("$SCRIPT" --help 2>&1 | head -20)
+if [[ "$help_out" == *"local-dev.sh"* && "$help_out" == *"Subcommands"* ]]; then
+ _pass "--help shows usage"
+else
+ _fail "--help didn't show usage" "$(echo "$help_out" | head -3)"
+fi
+
+# 5) An unknown service name routes through cmd_update_one and exits
+# non-zero rather than silently doing nothing.
+out=$("$SCRIPT" definitely-not-a-real-service 2>&1)
+rc=$?
+if (( rc != 0 )) && [[ "$out" == *"unknown service"* || "$out" == *"Unknown service"* ]]; then
+ _pass "unknown service exits non-zero with clear error"
+else
+ _fail "unknown service didn't error properly" "rc=$rc out=$out"
+fi
+
+# 6) `start` with no service name fails immediately (zsh parameter expansion
+# `${1:?...}` exits with the message).
+out=$("$SCRIPT" start 2>&1)
+rc=$?
+if (( rc != 0 )) && [[ "$out" == *"need service name"* ]]; then
+ _pass "start without arg refuses cleanly"
+else
+ _fail "start without arg should refuse" "rc=$rc out=$out"
+fi
+
+# 7) No-arg invocation must be non-interactive (= `status`). Previously the
+# default launched the TUI, which made the script unsafe to drop into
+# cron jobs or CI smoke tests. Anything that prints the banner without
+# hanging counts.
+out=$("$SCRIPT" 2>&1 | head -5)
+rc=$?
+if (( rc == 0 )) && [[ "$out" == *"Texera Local Dev"* ]]; then
+ _pass "no-arg invocation prints status (non-interactive)"
+else
+ _fail "no-arg invocation didn't print status" "rc=$rc out=$(echo "$out" | head -1)"
+fi
+
+# 8) `-i` without a TTY must refuse cleanly, not crash or hang. We pipe
+# stdin from /dev/null so the TTY check fires. Avoid piping into `head`
+# here — that masks the script's exit code under zsh.
+out=$("$SCRIPT" -i </dev/null 2>&1)
+rc=$?
+if (( rc != 0 )) && [[ "$out" == *"requires a TTY"* || "$out" == *"requires Python"* ]]; then
+ _pass "-i refuses cleanly without a TTY or Python"
+else
+ _fail "-i didn't refuse cleanly" "rc=$rc out=$(echo "$out" | head -1)"
+fi
+
+# 9) Regression: when no Python on the candidate list has textual, the
+# error message + install hint must actually print. The bug it
+# guards against: zsh's `set -e` aborts the script silently when a
+# command substitution's command exits non-zero (`var=$(returns 1)`),
+# so the install-hint code below the assignment never ran. `script`
+# allocates a pty so the TTY check passes and we get to the python
+# check.
+#
+# `script` has two incompatible invocation styles:
+# macOS BSD: script -q OUT_FILE CMD ARGS...
+# util-linux: script -qc "CMD ARGS..." OUT_FILE
+# Probe with `script --version`: util-linux supports it, BSD doesn't.
+if command -v script >/dev/null 2>&1; then
+ bad_py="/usr/bin/python3"
+ if [[ -x "$bad_py" ]] && ! "$bad_py" -c "import textual" >/dev/null 2>&1; then
+ if script --version >/dev/null 2>&1; then
+ # util-linux dialect
+ out=$(env -i HOME="$HOME" PATH=/usr/bin:/bin TERM="${TERM:-xterm}" \
+ script -qc "$SCRIPT -i; echo __rc=\$?" /dev/null </dev/null 2>&1)
+ else
+ # macOS BSD dialect
+ out=$(env -i HOME="$HOME" PATH=/usr/bin:/bin TERM="${TERM:-xterm}" \
+ script -q /dev/null sh -c "$SCRIPT -i; echo __rc=\$?" </dev/null 2>&1)
+ fi
+ if [[ "$out" == *"requires Python"* && "$out" == *"install Python"* && "$out" == *"__rc=1"* ]]; then
+ _pass "-i with no textual prints the install hint (regression for zsh set -e bug)"
+ else
+ _fail "-i with no textual didn't print install hint" \
+ "got: $(echo "$out" | head -3 | tr '\n' '|')"
+ fi
+ else
+ _pass "skip: no textual-less python available to test against"
+ fi
+else
+ _pass "skip: 'script' not on PATH"
+fi
+
+printf "\n%d passed, %d failed\n" "$PASS" "$FAIL"
+(( FAIL == 0 ))
diff --git a/bin/local-dev/tests/test_local_dev_tui.py b/bin/local-dev/tests/test_local_dev_tui.py
new file mode 100644
index 0000000000..6d61531a32
--- /dev/null
+++ b/bin/local-dev/tests/test_local_dev_tui.py
@@ -0,0 +1,376 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Unit tests for `bin/local-dev-tui.py`.
+
+We cover the pure helper functions that don't need a running Textual app:
+version detection, docker state mapping, ANSI scrubbing, source-hash
+fingerprinting, and the service catalog's structural invariants. The
+interactive widgets are exercised manually."""
+
+from __future__ import annotations
+
+import os
+import textwrap
+from pathlib import Path
+
+import pytest
+
+
+# ─────────────────── texera_version() ───────────────────
+
+def test_texera_version_parses_build_sbt(tmp_path, monkeypatch, tui):
+ sbt = tmp_path / "build.sbt"
+ sbt.write_text(textwrap.dedent("""\
+ ThisBuild / scalaVersion := "2.13.18"
+ ThisBuild / version := "9.9.9-FIXTURE"
+ ThisBuild / organization := "org.apache.texera"
+ """))
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.delenv("TEXERA_VERSION", raising=False)
+ assert tui.texera_version() == "9.9.9-FIXTURE"
+
+
+def test_texera_version_env_var_wins(tmp_path, monkeypatch, tui):
+ (tmp_path / "build.sbt").write_text('ThisBuild / version := "ignored-by-env"\n')
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.setenv("TEXERA_VERSION", "from-env")
+ assert tui.texera_version() == "from-env"
+
+
+def test_texera_version_raises_when_build_sbt_missing(tmp_path, monkeypatch, tui):
+ # No build.sbt in tmp_path. No fallback — must raise so a stale or
+ # broken checkout doesn't silently pick the wrong version.
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.delenv("TEXERA_VERSION", raising=False)
+ with pytest.raises(RuntimeError, match="build.sbt not found"):
+ tui.texera_version()
+
+
+def test_texera_version_raises_when_version_unparseable(tmp_path, monkeypatch, tui):
+ # build.sbt exists but the `ThisBuild / version := "…"` line isn't there.
+ (tmp_path / "build.sbt").write_text("name := \"texera\"\n")
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.delenv("TEXERA_VERSION", raising=False)
+ with pytest.raises(RuntimeError, match="could not find.*version"):
+ tui.texera_version()
+
+
+# ─────────────────── docker_state() ───────────────────
+
+@pytest.mark.parametrize("state,status,expected", [
+ ("running", "Up 5 minutes (healthy)", "running"),
+ ("running", "Up 1s (health: starting)", "starting"),
+ ("running", "Up 30s (unhealthy)", "unhealthy"),
+ ("running", "Up 2 minutes", "running"),
+ ("exited", "Exited (0) 4 minutes ago", "exited"),
+ ("exited", "Exited (137) 1 minute ago", "failed"),
+ ("created", "Created", "starting"),
+ ("restarting", "Restarting (1) 2 seconds ago", "starting"),
+ ("paused", "Up 10 minutes (Paused)", "starting"),
+ ("", "", "stopped"),
+ ("dead", "Dead", "stopped"),
+])
+def test_docker_state_mapping(state, status, expected, tui):
+ assert tui.docker_state(state, status) == expected
+
+
+# ─────────────────── _strip_ansi_motion() ───────────────────
+
+def test_strip_ansi_keeps_sgr_drops_motion(tui):
+ raw = "\x1b[32m✓\x1b[0m hello \x1b[2K\x1b[Hworld\r\r"
+ out = tui._strip_ansi_motion(raw)
+ # SGR (\e[32m, \e[0m) must survive so colours render
+ assert "\x1b[32m" in out
+ assert "\x1b[0m" in out
+ # cursor-positioning / erase / CR must be gone
+ assert "\x1b[2K" not in out
+ assert "\x1b[H" not in out
+ assert "\r" not in out
+
+
+def test_strip_ansi_idempotent(tui):
+ plain = "no escape codes here"
+ assert tui._strip_ansi_motion(plain) == plain
+
+
+# ─────────────────── source_hash() / is_dirty() ───────────────────
+
+def _seed_jvm_layout(repo: Path, svc_own_src: str, version: str = "9.9.9-T") -> None:
+ """Drop the directory layout source_hash() / is_dirty() walk.
+
+ Six shared `common/*/src` dirs + the per-service src dir + the dist
+ artifact jar at `target/<svc>-<version>/lib/...` so the artifact-mtime
+ lookup works."""
+ for d in ["dao", "config", "auth", "workflow-core", "workflow-operator", "pybuilder"]:
+ p = repo / "common" / d / "src"
+ p.mkdir(parents=True, exist_ok=True)
+ (p / "Stub.scala").write_text(f"// stub for common/{d}\n")
+ own = repo / svc_own_src
+ own.mkdir(parents=True, exist_ok=True)
+ (own / "Main.scala").write_text("object Main\n")
+
+
+def test_source_hash_stable_across_calls(tmp_path, monkeypatch, tui):
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ _seed_jvm_layout(tmp_path, "config-service/src")
+ svc = tui.SERVICES_BY_NAME["config-service"]
+ a = tui.source_hash(svc)
+ b = tui.source_hash(svc)
+ assert a == b
+ assert len(a) == 40 # SHA-1 hex
+
+
+def test_source_hash_changes_with_content(tmp_path, monkeypatch, tui):
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ _seed_jvm_layout(tmp_path, "config-service/src")
+ svc = tui.SERVICES_BY_NAME["config-service"]
+ before = tui.source_hash(svc)
+ (tmp_path / "config-service/src/Main.scala").write_text("object Main { def x = 1 }\n")
+ after = tui.source_hash(svc)
+ assert before != after
+
+
+def test_is_dirty_docker_always_clean(tui):
+ svc = tui.SERVICES_BY_NAME["postgres"]
+ assert tui.is_dirty(svc) is False
+
+
+def test_is_dirty_after_seed_then_edit(tmp_path, monkeypatch, tui):
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.setattr(tui, "BUILD_STAMP_DIR", tmp_path / "stamps")
+ (tmp_path / "stamps").mkdir()
+ _seed_jvm_layout(tmp_path, "config-service/src")
+
+ # Drop in a "jar" so the lazy seed path kicks in (it just needs to exist;
+ # the file's content isn't read by the dirty check, only the stamp's).
+ svc = tui.SERVICES_BY_NAME["config-service"]
+ jar = tmp_path / svc.artifact_jar
+ jar.parent.mkdir(parents=True, exist_ok=True)
+ jar.write_bytes(b"fake-jar-bytes")
+
+ # First call seeds the stamp and reports clean.
+ assert tui.is_dirty(svc) is False
+ stamp = tmp_path / "stamps" / svc.name
+ assert stamp.exists()
+ seeded_hash = stamp.read_text().strip()
+ assert len(seeded_hash) == 40
+
+ # Edit source → next dirty check sees a hash mismatch.
+ (tmp_path / "config-service/src/Main.scala").write_text("object Main { def y = 2 }\n")
+ assert tui.is_dirty(svc) is True
+
+
+def test_is_dirty_mtime_bump_without_content_change_stays_clean(tmp_path, monkeypatch, tui):
+ """Robustness against `git checkout` touching mtimes — the whole reason we
+ moved off pure-mtime detection. After seeding the stamp, simulating a
+ checkout (re-touching source files without changing content) must NOT
+ flash dirty."""
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.setattr(tui, "BUILD_STAMP_DIR", tmp_path / "stamps")
+ (tmp_path / "stamps").mkdir()
+ _seed_jvm_layout(tmp_path, "config-service/src")
+ svc = tui.SERVICES_BY_NAME["config-service"]
+ jar = tmp_path / svc.artifact_jar
+ jar.parent.mkdir(parents=True, exist_ok=True)
+ jar.touch()
+
+ tui.is_dirty(svc) # seed
+
+ # Simulate `git checkout` by touching all source files to "now" without
+ # changing content.
+ for f in (tmp_path / "config-service/src").rglob("*"):
+ if f.is_file():
+ os.utime(f, None)
+
+ assert tui.is_dirty(svc) is False
+
+
+# ─────────────────── Service catalog invariants ───────────────────
+
+def test_service_names_unique(tui):
+ names = [s.name for s in tui.SERVICES]
+ assert len(names) == len(set(names)), "duplicate service name in catalog"
+
+
+def test_service_ports_unique(tui):
+ ports = [s.port for s in tui.SERVICES]
+ assert len(ports) == len(set(ports)), "two services claim the same port"
+
+
+def test_jvm_services_have_sbt_project_and_src(tui):
+ for s in tui.SERVICES:
+ if s.type == "jvm":
+ # sbt_project is optional: sibling services (e.g.
+ # computing-unit-master) ride another service's dist and have
+ # no separate sbt invocation. own_src + artifact_jar are still
+ # required so dirty-detection and start_one work.
+ assert s.own_src, f"{s.name} missing own_src"
+ assert s.artifact_jar, f"{s.name} missing artifact_jar"
+ assert tui.TEXERA_VERSION in s.artifact_jar, (
+ f"{s.name}'s jar path should embed the dynamic version, got {s.artifact_jar}"
+ )
+
+
+def test_jvm_siblings_share_artifact_jar(tui):
+ """computing-unit-master is a launcher shipped inside amber's dist; it must
+ point at the same canary jar as texera-web so dirty-state stays in sync."""
+ cum = tui.SERVICES_BY_NAME["computing-unit-master"]
+ web = tui.SERVICES_BY_NAME["texera-web"]
+ assert cum.sbt_project is None, "computing-unit-master must not own an sbt project"
+ assert cum.artifact_jar == web.artifact_jar
+ assert cum.own_src == web.own_src
+
+
+def test_docker_services_have_no_jar(tui):
+ for s in tui.SERVICES:
+ if s.type == "docker":
+ assert s.artifact_jar is None
+
+
+def test_watch_types_constant(tui):
+ assert tui.WATCH_TYPES == {"yarn", "bun"}
+ # Every watch-type service must show up in the catalog
+ for t in tui.WATCH_TYPES:
+ assert any(s.type == t for s in tui.SERVICES), f"no service of type {t}"
+
+
+# ─────────────────── CommandHistory navigation ───────────────────
+
+def test_command_history_back_and_forward(tmp_path, tui):
+ h = tui.CommandHistory(history_file=tmp_path / "h")
+ for cmd in ["u", "d", "config-service"]:
+ h.push(cmd)
+ assert h._history == ["u", "d", "config-service"]
+
+ # ↑ from a fresh draft walks back from newest
+ assert h.back("draft-typing") == "config-service"
+ assert h.back("config-service") == "d"
+ assert h.back("d") == "u"
+ # At oldest, further ↑ returns None (caller keeps current value)
+ assert h.back("u") is None
+
+ # ↓ walks forward; once past newest it restores the draft
+ assert h.forward() == "d"
+ assert h.forward() == "config-service"
+ assert h.forward() == "draft-typing"
+ # At the live draft, ↓ is a no-op
+ assert h.forward() is None
+
+
+def test_command_history_dedup_consecutive(tmp_path, tui):
+ h = tui.CommandHistory(history_file=tmp_path / "h")
+ h.push("u")
+ h.push("u") # same as previous → not appended
+ h.push("d")
+ h.push("u") # not consecutive, gets appended
+ assert h._history == ["u", "d", "u"]
+
+
+def test_command_history_persists(tmp_path, tui):
+ f = tmp_path / "h"
+ a = tui.CommandHistory(history_file=f)
+ a.push("u")
+ a.push("d")
+
+ b = tui.CommandHistory(history_file=f)
+ assert b._history == ["u", "d"]
+
+
+def test_command_history_max_size(tmp_path, tui):
+ h = tui.CommandHistory(history_file=tmp_path / "h", max_size=3)
+ for cmd in ["a", "b", "c", "d", "e"]:
+ h.push(cmd)
+ reloaded = tui.CommandHistory(history_file=tmp_path / "h", max_size=3)
+ # On-disk cap: only the most recent 3 survive a reload.
+ assert reloaded._history == ["c", "d", "e"]
+
+
+# ─────────────────── build.sbt dependency parsing ───────────────────
+
+def test_sbt_graph_parsed_from_real_build_sbt(tui):
+ """The parser must extract at least the well-known subset of projects
+ (DAO, Config, Auth, the seven JVM services) and their direct deps from
+ the live repo's build.sbt. If any of these go missing we've drifted
+ from sbt's view of the world and dirty-detection is silently wrong."""
+ g = tui._SBT_GRAPH
+ assert "DAO" in g and "Config" in g and "Auth" in g
+ assert "ConfigService" in g
+ assert "WorkflowExecutionService" in g
+ # Auth depends on DAO + Config — anchor invariant from build.sbt.
+ assert set(g["Auth"]["deps"]) >= {"DAO", "Config"}
+ # WorkflowCore depends on DAO/Config/PyBuilder.
+ assert set(g["WorkflowCore"]["deps"]) >= {"DAO", "Config", "PyBuilder"}
+
+
+def test_sbt_transitive_closure_skips_unrelated(tui):
+ """config-service should NOT pull workflow-operator/workflow-core into
+ its dep set. That's the whole point of using build.sbt as source of
+ truth instead of a shared common/* list."""
+ dirs = tui._transitive_src_dirs("ConfigService", tui._SBT_GRAPH)
+ assert "common/workflow-operator/src" not in dirs
+ assert "common/workflow-core/src" not in dirs
+ # Should still include its own + Auth's reach (DAO/Config).
+ assert "config-service/src" in dirs
+ assert "common/auth/src" in dirs
+ assert "common/dao/src" in dirs
+ assert "common/config/src" in dirs
+
+
+def test_sbt_transitive_for_amber_includes_workflow_chain(tui):
+ """texera-web / computing-unit-master ride WorkflowExecutionService;
+ its closure must include WorkflowOperator → WorkflowCore → DAO/Config/PyBuilder."""
+ dirs = tui._transitive_src_dirs("WorkflowExecutionService", tui._SBT_GRAPH)
+ expected = {
+ "amber/src",
+ "common/workflow-operator/src",
+ "common/workflow-core/src",
+ "common/dao/src",
+ "common/config/src",
+ "common/pybuilder/src",
+ "common/auth/src",
+ }
+ assert expected <= set(dirs), f"missing: {expected - set(dirs)}"
+
+
+def test_sbt_unknown_project_raises(tui):
+ """Asking the graph for an unknown sbt project name raises rather
+ than silently falling back to a hardcoded list — that fallback would
+ mask a real build.sbt drift / typo."""
+ with pytest.raises(RuntimeError, match="not found in build.sbt graph"):
+ tui._transitive_src_dirs("NoSuchProject", tui._SBT_GRAPH)
+
+
+def test_sbt_empty_graph_raises(tui):
+ """A parse that returned no projects (very old / mangled build.sbt)
+ must raise. Silently using the prior pre-parse list would let dirty
+ detection report wrong results without anyone noticing."""
+ with pytest.raises(RuntimeError, match="graph is empty"):
+ tui._transitive_src_dirs("ConfigService", {})
+
+
+def test_sbt_test_scope_deps_ignored(tui):
+ """`X % "test->test"` is test-only — must not pollute the runtime dep
+ list (services would otherwise rebuild on test-only edits, defeating
+ the build skip)."""
+ # Auth has a `.dependsOn(DAO % "test->test")` next to its main
+ # `.dependsOn(DAO, Config)`. The test-scope reference should not
+ # appear as a second entry.
+ auth_deps = tui._SBT_GRAPH["Auth"]["deps"]
+ # DAO appears once (from the main-scope dependsOn) — not twice from
+ # the test-scope one.
+ assert auth_deps.count("DAO") == 1