You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@texera.apache.org by gi...@apache.org on 2026/06/28 09:07:46 UTC
(texera) branch gh-readonly-queue/main/pr-5961-01b9254bea697b936724c46c20cdd353f00ba226 created (now de6c31ac48)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch gh-readonly-queue/main/pr-5961-01b9254bea697b936724c46c20cdd353f00ba226
in repository https://gitbox.apache.org/repos/asf/texera.git
at de6c31ac48 feat(infra): add `bin/local-dev.sh` dev-stack manager + TUI (#5961)
This branch includes the following new commits:
new de6c31ac48 feat(infra): add `bin/local-dev.sh` dev-stack manager + TUI (#5961)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(texera) 01/01: feat(infra): add `bin/local-dev.sh` dev-stack manager + TUI (#5961)
Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-5961-01b9254bea697b936724c46c20cdd353f00ba226
in repository https://gitbox.apache.org/repos/asf/texera.git
commit de6c31ac48ddf829b00c78b1d4f45b826236ff9d
Author: Yicong Huang <17...@users.noreply.github.com>
AuthorDate: Sun Jun 28 02:07:25 2026 -0700
feat(infra): add `bin/local-dev.sh` dev-stack manager + TUI (#5961)
### What changes were proposed in this PR?
- Introduce `bin/local-dev.sh` to build/start/stop/status/logs for
docker infra + JVM services + agent-service + frontend.
- Add `bin/local-dev-tui.py` (Textual) to provide a live dashboard,
command prompt, and log tailing by shelling out to bin/local-dev.sh.
- Add tests and CI plumbing (labeler + required-checks stack + build
workflow job) to run shell smoke tests and pytest unit tests for the
tooling.
#### Demo 1: Starting Texera Local Dev
https://github.com/user-attachments/assets/beff0dd7-5afa-4de7-8c5c-72f8ed58bfe7
#### Demo 2: Auto-updating Changed Services
https://github.com/user-attachments/assets/c84cfe12-2b9b-4e80-8651-b1375a6a82fb
#### Demo 3: Tailing Service Logs
https://github.com/user-attachments/assets/d53b35f4-c821-4fcc-bd11-5dd696946895
#### What you can do
| Command | What it does |
| --------------------------------------------- |
-----------------------------------------------------------------------
|
| `bin/local-dev.sh` / `… status` | One-shot dashboard (state · uptime ·
CPU% · mem · PID per service). |
| `bin/local-dev.sh -i` | Live Textual TUI of the same dashboard, with a
command prompt. |
| `bin/local-dev.sh up` | Build dirty JVMs, install yarn/bun deps if
locks moved, bring all 14 services up. Bootstraps the postgres schema on
stale volumes. |
| `bin/local-dev.sh down` | Stop everything, reverse order. |
| `bin/local-dev.sh auto` | Detect which services' source changed (`★`)
and rebuild + bounce only those. |
| `bin/local-dev.sh <svc>` | Rebuild that one service and bounce it. |
| `bin/local-dev.sh logs <svc>` | Tail a service's log. |
#### Key behaviours
- **Dirty detection** drives `auto` / the `★` indicator: SHA-1 of source
bytes against a per-service stamp, walking the per-service transitive
sbt dependency graph parsed from `build.sbt`. Editing
`common/workflow-operator/src` doesn't trigger services that don't
depend on it.
- **Dynamic everything** — texera version, sbt graph, host LAN IP (for
the Iceberg/MinIO endpoint that Lakekeeper hands back to host JVMs), JDK
17 location, Python interpreter, Node toolchain — all auto-detected at
startup with clear "what we saw / what to install" diagnostics on miss.
- **Bash 3.2 compatible** — runs on Apple's pinned `/bin/bash` 3.2 with
no re-exec.
### Any related issues, documentation, discussions?
Closes #5960.
### How was this PR tested?
- 36 pytest + 10 shell smoke under bash 3.2 / bash 5 / zsh.
- Manual cold-start on macOS: `up` brings all 14 services to healthy
from an empty state; `down` tears them down. Editing `.scala` files
marks only the dependent services dirty; `auto` rebuilds + bounces just
those (amber's two siblings always paired). `up --build / --fresh /
--skip=…` all exercised. Toolchain-missing paths print the diagnostic
env + install hint.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Anthropic, Claude Opus 4.7).
---------
Co-authored-by: Claude Opus 4.7 (1M context) <no...@anthropic.com>
---
.github/labeler.yml | 11 +
.github/workflows/build.yml | 42 +
.github/workflows/required-checks.yml | 9 +-
amber/dev-requirements.txt | 3 +
bin/local-dev-tui.py | 1651 +++++++++++++
bin/local-dev.sh | 2536 ++++++++++++++++++++
bin/local-dev/docker-compose.override.yml | 56 +
.../local-dev/tests/__init__.py | 18 -
bin/local-dev/tests/conftest.py | 44 +
bin/local-dev/tests/test_local_dev_sh.sh | 167 ++
bin/local-dev/tests/test_local_dev_tui.py | 376 +++
11 files changed, 4894 insertions(+), 19 deletions(-)
diff --git a/.github/labeler.yml b/.github/labeler.yml
index 75f20d2c33..e520c6a6df 100644
--- a/.github/labeler.yml
+++ b/.github/labeler.yml
@@ -113,6 +113,17 @@ dev:
- any-glob-to-any-file:
- 'bin/**'
+infra:
+ # Dev-loop tooling for bringing up / tailing the local stack lives under
+ # bin/local-dev*. Tracked on its own label so the matching CI stack runs
+ # on changes here without dragging the broader `dev` label (which is a
+ # no-op stack-mapping-wise) into the equation.
+ - changed-files:
+ - any-glob-to-any-file:
+ - 'bin/local-dev.sh'
+ - 'bin/local-dev-tui.py'
+ - 'bin/local-dev/**'
+
dependencies:
- changed-files:
- any-glob-to-any-file:
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index addb2d3236..b44f035552 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -60,6 +60,10 @@ on:
required: false
type: boolean
default: true
+ run_infra:
+ required: false
+ type: boolean
+ default: true
mode:
# PR (default) | nightly | release. Only "PR" passes
# --ignore-transitive-version to check_binary_deps.py.
@@ -917,3 +921,41 @@ jobs:
report_type: test_results
disable_search: true
fail_ci_if_error: false
+
+ infra:
+ # Tests for the dev-loop tooling under bin/local-dev*.
+ # Light job: bash syntax + smoke tests, plus pytest unit tests for the
+ # pure helpers in local-dev-tui.py. No docker / sbt / live stack — those
+ # belong in an e2e job we'll add when the tool is exercised more widely.
+ if: ${{ inputs.run_infra }}
+ name: ${{ format('infra{0} ({1})', inputs.job_name_suffix, matrix.os) }}
+ runs-on: ${{ matrix.os }}
+ strategy:
+ fail-fast: false
+ matrix:
+ os: [ubuntu-latest, macos-latest]
+ python-version: ["3.12"]
+ steps:
+ - name: Checkout Texera
+ uses: actions/checkout@v5
+ with:
+ ref: ${{ inputs.checkout_ref || github.sha }}
+ fetch-depth: 0
+ - name: Prepare backport workspace
+ if: ${{ inputs.backport_target_branch != '' }}
+ run: bash ./.github/scripts/prepare-backport-checkout.sh "${{ inputs.backport_target_branch }}" "${{ inputs.backport_commit_range }}"
+ - name: Setup Python
+ uses: actions/setup-python@v6
+ with:
+ python-version: ${{ matrix.python-version }}
+ - name: Install Python test deps
+ # amber/dev-requirements.txt pins both `pytest` and `textual` — the
+ # latter so a fresh dev who follows the AGENTS.md venv-setup
+ # instructions ends up with the TUI working.
+ run: |
+ python -m pip install --upgrade pip
+ python -m pip install -r amber/dev-requirements.txt
+ - name: Run shell smoke tests (bash)
+ run: bash bin/local-dev/tests/test_local_dev_sh.sh
+ - name: Run Python unit tests
+ run: python -m pytest bin/local-dev/tests/ -v --tb=short
diff --git a/.github/workflows/required-checks.yml b/.github/workflows/required-checks.yml
index 2d7012fe33..d854be3f17 100644
--- a/.github/workflows/required-checks.yml
+++ b/.github/workflows/required-checks.yml
@@ -65,6 +65,7 @@ jobs:
run_platform: ${{ steps.decide.outputs.run_platform }}
run_pyamber: ${{ steps.decide.outputs.run_pyamber }}
run_agent_service: ${{ steps.decide.outputs.run_agent_service }}
+ run_infra: ${{ steps.decide.outputs.run_infra }}
backport_targets: ${{ steps.decide.outputs.backport_targets }}
steps:
- name: Wait for Pull Request Labeler
@@ -153,7 +154,8 @@ jobs:
"agent-service": ["agent-service"],
common: ["amber", "amber-integration", "platform"],
"ddl-change": ["amber", "amber-integration", "platform"],
- ci: ["frontend", "amber", "amber-integration", "platform", "pyamber", "agent-service"],
+ infra: ["infra"],
+ ci: ["frontend", "amber", "amber-integration", "platform", "pyamber", "agent-service", "infra"],
};
let runFrontend = true;
@@ -162,6 +164,7 @@ jobs:
let runPlatform = true;
let runPyamber = true;
let runAgentService = true;
+ let runInfra = true;
if (eventName === "pull_request") {
const stacks = new Set();
@@ -176,6 +179,7 @@ jobs:
runPlatform = stacks.has("platform");
runPyamber = stacks.has("pyamber");
runAgentService = stacks.has("agent-service");
+ runInfra = stacks.has("infra");
core.info(
`Stacks selected by label union: ${[...stacks].sort().join(", ") || "(none)"}`
);
@@ -187,6 +191,7 @@ jobs:
core.setOutput("run_platform", runPlatform ? "true" : "false");
core.setOutput("run_pyamber", runPyamber ? "true" : "false");
core.setOutput("run_agent_service", runAgentService ? "true" : "false");
+ core.setOutput("run_infra", runInfra ? "true" : "false");
// Backport targets: all current release/* labels on the PR.
const targets = [...new Set(labels.filter((n) => /^release\/.+$/.test(n)))].sort();
@@ -242,6 +247,7 @@ jobs:
run_platform: ${{ needs.precheck.outputs.run_platform == 'true' }}
run_pyamber: ${{ needs.precheck.outputs.run_pyamber == 'true' }}
run_agent_service: ${{ needs.precheck.outputs.run_agent_service == 'true' }}
+ run_infra: ${{ needs.precheck.outputs.run_infra == 'true' }}
secrets: inherit
backport:
@@ -263,6 +269,7 @@ jobs:
run_platform: ${{ needs.precheck.outputs.run_platform == 'true' }}
run_pyamber: ${{ needs.precheck.outputs.run_pyamber == 'true' }}
run_agent_service: ${{ needs.precheck.outputs.run_agent_service == 'true' }}
+ run_infra: ${{ needs.precheck.outputs.run_infra == 'true' }}
secrets: inherit
required-checks:
diff --git a/amber/dev-requirements.txt b/amber/dev-requirements.txt
index 4ab14960cb..3ac549283d 100644
--- a/amber/dev-requirements.txt
+++ b/amber/dev-requirements.txt
@@ -32,3 +32,6 @@ pytest-cov==7.1.0
# protoc plugin for bin/python-proto-gen.sh; runtime needs only the
# base `betterproto` in requirements.txt.
betterproto[compiler]==2.0.0b7
+
+# Required by `bin/local-dev.sh -i` (the interactive Textual TUI).
+textual==8.2.7
diff --git a/bin/local-dev-tui.py b/bin/local-dev-tui.py
new file mode 100644
index 0000000000..3b294491f7
--- /dev/null
+++ b/bin/local-dev-tui.py
@@ -0,0 +1,1651 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""bin/local-dev-tui.py — Textual dashboard for the Texera local dev stack.
+
+Lives next to bin/local-dev.sh; the shell script remains the canonical engine
+(build, start, stop, status) and this TUI shells out to it for every action.
+The dashboard itself owns state polling, dirty-source detection, and the
+prompt loop. Textual handles diff rendering so the screen doesn't accrete in
+scrollback the way the old zsh `\\e[H` redraw did.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import contextlib
+import hashlib
+import os
+import re
+import shlex
+import shutil
+import subprocess
+import time
+from dataclasses import dataclass, field
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Optional
+
+from rich.text import Text
+from textual import events, work
+from textual.app import App, ComposeResult
+from textual.binding import Binding
+from textual.containers import Vertical
+from textual.reactive import reactive
+from textual.widgets import DataTable, Input, RichLog, Static
+
+# ─────────────────── Constants ───────────────────
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+STATE_DIR = Path(os.environ.get("TEXERA_LOCAL_DEV_DIR", "/tmp/texera-local-dev"))
+LOG_DIR = STATE_DIR / "logs"
+BUILD_STAMP_DIR = STATE_DIR / "build-stamps"
+# Per-service phase markers written by the shell during stop/build/start.
+# Each file holds `<phase>\t<epoch>` — we read it back in _tick_state and
+# render an animated transitional STATE if it's recent (<90s).
+PHASE_DIR = STATE_DIR / "svc-phase"
+PHASE_STALE_S = 90.0
+REPL_LOG = LOG_DIR / "repl.log"
+LOG_DIR.mkdir(parents=True, exist_ok=True)
+BUILD_STAMP_DIR.mkdir(parents=True, exist_ok=True)
+
+LOCAL_DEV_SH = REPO_ROOT / "bin" / "local-dev.sh"
+DOCKER_PROJECT = "texera-local-dev"
+
+HISTORY_FILE = STATE_DIR / "tui-history"
+MAX_HISTORY = 500
+
+SOURCE_SUFFIXES = {".scala", ".java", ".proto"}
+
+# Single source of truth for per-JVM-service source dirs is the SBT build
+# graph in build.sbt. We parse it instead of hardcoding `common/*` so that
+# adding a `lazy val NewCommon = (project in file("common/new"))` and a
+# corresponding `.dependsOn(NewCommon)` automatically flows through to
+# dirty-detection without anyone touching this file.
+
+# Matches test-scope dependsOn args we want to drop from the runtime
+# graph. `X % "test->test"`, `X % "test"`, `X % Test`. Tighter than the
+# previous "any %" check, which would have silently dropped a future
+# `.dependsOn(X % "compile->compile")` and broken dirty-detection on X.
+# No trailing `\b` — `\b` after a literal `"` doesn't match because both
+# sides are non-word chars; we anchor the end of the quoted form on the
+# closing `"` itself, and the `Test` form on (?:\b|$).
+_TEST_SCOPE_RE = re.compile(r'%\s*(?:"test(?:->[^"]*)?"|Test(?:\b|$))')
+
+
+def _parse_sbt_deps() -> dict[str, dict]:
+ """Walk `build.sbt` and return a dependency graph keyed by sbt project
+ name. Each entry is `{"path": <repo-relative dir>, "deps": [..main-scope
+ deps..]}`. Test-only `% "test->test"` deps are filtered out — they
+ can't affect a service's runtime artifact, so they don't drive dirty.
+ """
+ bs = REPO_ROOT / "build.sbt"
+ try:
+ text = bs.read_text(encoding="utf-8", errors="replace")
+ except OSError:
+ return {}
+
+ proj_re = re.compile(
+ r'^lazy\s+val\s+([A-Z][A-Za-z0-9]*)\s*=\s*\(project\s+in\s+file\("([^"]+)"\)\)',
+ re.MULTILINE,
+ )
+ matches = list(proj_re.finditer(text))
+ if not matches:
+ return {}
+
+ graph: dict[str, dict] = {}
+ for i, m in enumerate(matches):
+ name = m.group(1)
+ path = m.group(2)
+ # The "block" for this project spans from this match to the next
+ # `lazy val ... = (project in file(...))` (or end of file). All the
+ # chained `.dependsOn(...)` calls live in there.
+ end = matches[i + 1].start() if i + 1 < len(matches) else len(text)
+ block = text[m.start():end]
+
+ deps: list[str] = []
+ for arg_list in re.findall(r"\.dependsOn\(([^)]*)\)", block):
+ for arg in arg_list.split(","):
+ arg = arg.strip()
+ # Skip ONLY test-scope deps — match `% "test->test"` and
+ # `% Test` exactly. Filtering all `%`-scoped args was too
+ # broad: a future `.dependsOn(X % "compile->compile")` or
+ # `% Provided` would silently disappear from the dep
+ # graph and break dirty-detection on `X`.
+ if _TEST_SCOPE_RE.search(arg):
+ continue
+ # Strip any trailing whitespace / comments. Project refs
+ # are bare identifiers.
+ ref = arg.split()[0] if arg else ""
+ if ref and ref[0].isupper():
+ deps.append(ref)
+ graph[name] = {"path": path, "deps": deps}
+ return graph
+
+
+def _transitive_src_dirs(sbt_project: Optional[str], graph: dict[str, dict]) -> list[str]:
+ """Return source dirs for sbt_project AND every project it transitively
+ depends on. Each dir is `<project_path>/src` (the canonical sbt source
+ location). Order is stable; duplicates removed. Raises if the graph
+ is empty or `sbt_project` isn't in it — silently falling back to a
+ hardcoded common/* list would mask a real build.sbt drift."""
+ if not graph:
+ raise RuntimeError("build.sbt dependency graph is empty — parse failed")
+ if not sbt_project or sbt_project not in graph:
+ raise RuntimeError(
+ f"sbt project {sbt_project!r} not found in build.sbt graph; "
+ f"known projects: {sorted(graph.keys())}"
+ )
+ visited: set[str] = set()
+ out: list[str] = []
+
+ def walk(name: str) -> None:
+ if name in visited or name not in graph:
+ return
+ visited.add(name)
+ path = graph[name]["path"]
+ src = f"{path}/src"
+ if src not in out:
+ out.append(src)
+ for d in graph[name]["deps"]:
+ walk(d)
+ walk(sbt_project)
+ return out
+
+
+_SBT_GRAPH = _parse_sbt_deps()
+
+
+# Per-sbt-project transitive closure, memoised. Computed once on first
+# touch; subsequent dirty checks are O(1) lookup vs the prior O(deps)
+# BFS per call.
+_SBT_TRANSITIVE_CACHE: dict[str, list[str]] = {}
+
+
+def _transitive_src_dirs_cached(sbt_project: Optional[str]) -> list[str]:
+ if sbt_project is None:
+ return _transitive_src_dirs(None, _SBT_GRAPH)
+ if sbt_project not in _SBT_TRANSITIVE_CACHE:
+ _SBT_TRANSITIVE_CACHE[sbt_project] = _transitive_src_dirs(sbt_project, _SBT_GRAPH)
+ return _SBT_TRANSITIVE_CACHE[sbt_project]
+
+POLL_INTERVAL_S = 1.0 # how often to refresh service state
+DIRTY_INTERVAL_S = 2.0 # how often to recompute dirty indicators
+
+# Services whose own runtime watches the filesystem and rebuilds on source
+# change: `yarn start` runs `ng serve` (Angular dev server, hot-reload) and
+# `bun run --watch` reloads the agent-service on change. The dashboard
+# surfaces this in the SRC column so the user doesn't try to bounce them
+# unnecessarily; the ★ "dirty" indicator only flashes for them when the lock
+# file changes (i.e. an actual dep refresh is needed).
+WATCH_TYPES = {"yarn", "bun"}
+
+
+# ─────────────────── Texera version (dynamic) ───────────────────
+
+_VERSION_RE = re.compile(
+ r'^\s*ThisBuild\s*/\s*version\s*:=\s*"([^"]+)"', re.MULTILINE
+)
+
+
+def texera_version() -> str:
+ """Parse the project version from build.sbt so artifact paths track
+ whatever branch the developer is on (it's 1.3.0-incubating-SNAPSHOT on
+ main today, was 1.2.0-incubating on release/v1.2, will differ again).
+ Override with `TEXERA_VERSION` env var to bypass parsing."""
+ env = os.environ.get("TEXERA_VERSION")
+ if env:
+ return env
+ bs = REPO_ROOT / "build.sbt"
+ if not bs.exists():
+ raise RuntimeError(
+ f"build.sbt not found at {bs} — set TEXERA_VERSION to bypass"
+ )
+ m = _VERSION_RE.search(bs.read_text(errors="replace"))
+ if not m:
+ raise RuntimeError(
+ f"could not find `ThisBuild / version := \"…\"` in {bs} — "
+ f"set TEXERA_VERSION to bypass"
+ )
+ return m.group(1)
+
+
+TEXERA_VERSION = texera_version()
+
+
+# ─────────────────── Service catalog ───────────────────
+
+@dataclass
+class Service:
+ name: str
+ type: str # "docker" | "jvm" | "yarn" | "bun"
+ port: int
+ sbt_project: Optional[str] = None # for jvm
+ own_src: Optional[str] = None # for jvm
+ artifact_jar: Optional[str] = None # for jvm
+
+
+def _jvm(name: str, port: int, project: Optional[str], own_src: str) -> Service:
+ """sbt-native-packager lays the dist out as
+ `target/<artifact>-<VERSION>/lib/org.apache.texera.<artifact>-<VERSION>.jar`
+ for every subproject. amber is the exception: its sbt subproject is
+ named `amber` (not `texera-web`) and the dist goes under `amber/target/`
+ rather than the repo-level `target/`. computing-unit-master rides that
+ same amber dist as a sibling launcher — its `project` is None because
+ no separate sbt invocation produces it."""
+ is_amber_svc = name in ("texera-web", "computing-unit-master")
+ artifact = "amber" if is_amber_svc else name
+ target_prefix = "amber/" if is_amber_svc else ""
+ jar = (
+ f"{target_prefix}target/{artifact}-{TEXERA_VERSION}/lib/"
+ f"org.apache.texera.{artifact}-{TEXERA_VERSION}.jar"
+ )
+ return Service(name, "jvm", port, sbt_project=project,
+ own_src=own_src, artifact_jar=jar)
+
+
+SERVICES: list[Service] = [
+ Service("postgres", "docker", 5432),
+ Service("minio", "docker", 9000),
+ Service("lakefs", "docker", 8000),
+ Service("lakekeeper", "docker", 8181),
+ Service("litellm", "docker", 4000),
+ _jvm("config-service", 9094, "ConfigService",
+ "config-service/src"),
+ _jvm("access-control-service", 9096, "AccessControlService",
+ "access-control-service/src"),
+ _jvm("file-service", 9092, "FileService",
+ "file-service/src"),
+ _jvm("workflow-compiling-service", 9090, "WorkflowCompilingService",
+ "workflow-compiling-service/src"),
+ _jvm("computing-unit-master", 8085, None,
+ "amber/src"),
+ _jvm("computing-unit-managing-service", 8082, "ComputingUnitManagingService",
+ "computing-unit-managing-service/src"),
+ _jvm("texera-web", 8080, "WorkflowExecutionService",
+ "amber/src"),
+ Service("agent-service", "bun", 3001),
+ Service("frontend", "yarn", 4200),
+]
+
+SERVICES_BY_NAME = {s.name: s for s in SERVICES}
+
+
+# ─────────────────── Live state model ───────────────────
+
+def read_svc_phase(svc_name: str) -> Optional[str]:
+ """Return the shell-written phase (`stopping` / `building` / `starting`)
+ or None if absent / stale. Stale-after-90s guards against a crashed
+ `bin/local-dev.sh` leaving an orphan file."""
+ f = PHASE_DIR / svc_name
+ try:
+ text = f.read_text().strip()
+ except OSError:
+ return None
+ if not text:
+ return None
+ parts = text.split("\t", 1)
+ phase = parts[0]
+ try:
+ ts = float(parts[1]) if len(parts) > 1 else 0.0
+ except ValueError:
+ ts = 0.0
+ if ts and (time.time() - ts) > PHASE_STALE_S:
+ return None
+ return phase
+
+
+@dataclass
+class LiveState:
+ """Snapshot of the world this tick — what the dashboard renders."""
+ docker: dict[str, tuple[str, str]] = field(default_factory=dict) # name -> (state, status)
+ pids: dict[str, Optional[str]] = field(default_factory=dict) # name -> pid or None
+ dirty: dict[str, bool] = field(default_factory=dict)
+ mtimes: dict[str, Optional[str]] = field(default_factory=dict)
+ # Per-service resource usage (computed every poll for native services
+ # via cheap `ps`; refreshed less often for docker via `docker stats`
+ # since that call costs ~2s).
+ uptimes: dict[str, str] = field(default_factory=dict) # name -> "12m" / "—"
+ cpu_pct: dict[str, str] = field(default_factory=dict) # name -> "8.2%" / "—"
+ mem_use: dict[str, str] = field(default_factory=dict) # name -> "85M" / "—"
+ phases: dict[str, str] = field(default_factory=dict) # name -> "stopping"/"building"/"starting" or ""
+
+
+# ─────────────────── Helpers ───────────────────
+
+async def _run_capture(*argv: str) -> str:
+ proc = await asyncio.create_subprocess_exec(
+ *argv,
+ stdout=asyncio.subprocess.PIPE,
+ stderr=asyncio.subprocess.DEVNULL,
+ )
+ stdout, _ = await proc.communicate()
+ return stdout.decode(errors="replace")
+
+
+async def lsof_port_pid(port: int) -> Optional[str]:
+ out = await _run_capture("lsof", "-nP", f"-iTCP:{port}", "-sTCP:LISTEN", "-t")
+ out = out.strip()
+ return out.split("\n", 1)[0] if out else None
+
+
+async def docker_ps_all() -> dict[str, tuple[str, str]]:
+ out = await _run_capture(
+ "docker", "compose", "-p", DOCKER_PROJECT, "ps", "-a",
+ "--format", "{{.Service}}|{{.State}}|{{.Status}}",
+ )
+ result: dict[str, tuple[str, str]] = {}
+ for line in out.splitlines():
+ parts = line.split("|", 2)
+ if len(parts) == 3:
+ result[parts[0]] = (parts[1], parts[2])
+ return result
+
+
+def docker_state(svc_state: str, svc_status: str) -> str:
+ """Map docker's raw state/status to the small palette the dashboard renders."""
+ if svc_state == "running":
+ if "(healthy)" in svc_status:
+ return "running"
+ if "(health: starting)" in svc_status:
+ return "starting"
+ if "(unhealthy)" in svc_status:
+ return "unhealthy"
+ return "running"
+ if svc_state == "exited":
+ return "exited" if svc_status.startswith("Exited (0)") else "failed"
+ if svc_state in ("created", "restarting", "paused", "removing"):
+ return "starting"
+ return "stopped"
+
+
+# ─────────────────── Dirty-source detection (content hash) ───────────────────
+
+# ─────────────────── Uptime / resource helpers ───────────────────
+
+def _format_uptime(secs: int) -> str:
+ """Compact duration: `12s`, `5m 23s`, `2h 14m`, `3d 4h`."""
+ if secs < 0:
+ return "—"
+ if secs < 60:
+ return f"{secs}s"
+ if secs < 3600:
+ return f"{secs // 60}m {secs % 60}s"
+ if secs < 86400:
+ return f"{secs // 3600}h {(secs % 3600) // 60}m"
+ return f"{secs // 86400}d {(secs % 86400) // 3600}h"
+
+
+def _format_bytes(n: int) -> str:
+ """Bytes → short human string (`85M`, `1.2G`, `467K`)."""
+ if n < 1024:
+ return f"{n}B"
+ if n < 1024 ** 2:
+ return f"{n // 1024}K"
+ if n < 1024 ** 3:
+ return f"{n // (1024 ** 2)}M"
+ return f"{n / (1024 ** 3):.1f}G"
+
+
+def _parse_etime(et: str) -> int:
+ """`ps -o etime` format `[[DD-]hh:]mm:ss` → seconds."""
+ et = et.strip()
+ if not et:
+ return -1
+ days = 0
+ if "-" in et:
+ d, et = et.split("-", 1)
+ days = int(d or 0)
+ parts = et.split(":")
+ try:
+ if len(parts) == 3:
+ h, m, s = (int(p or 0) for p in parts)
+ elif len(parts) == 2:
+ h = 0
+ m, s = (int(p or 0) for p in parts)
+ else:
+ h = m = 0
+ s = int(parts[0] or 0)
+ except ValueError:
+ return -1
+ return days * 86400 + h * 3600 + m * 60 + s
+
+
+def _docker_container_for(svc_name: str) -> str:
+ """Container names: most are `texera-<svc>`; `litellm` is unprefixed."""
+ if svc_name == "litellm":
+ return "litellm"
+ return f"texera-{svc_name}"
+
+
+_DOCKER_STATS_CACHE: dict[str, tuple[str, str]] = {}
+_DOCKER_STATS_TS: float = 0.0
+_DOCKER_STATS_TTL = 5.0 # seconds — docker stats --no-stream costs ~2s, don't run it more than this
+
+
+def _refresh_docker_stats() -> None:
+ """Single `docker stats --no-stream` pass; refreshes the cached
+ container → (cpu%, mem) map. Called from a background worker so the
+ main poll tick stays cheap."""
+ global _DOCKER_STATS_CACHE, _DOCKER_STATS_TS
+ try:
+ out = subprocess.run(
+ ["docker", "stats", "--no-stream",
+ "--format", "{{.Name}}|{{.CPUPerc}}|{{.MemUsage}}"],
+ capture_output=True, text=True, timeout=8,
+ ).stdout
+ except (subprocess.TimeoutExpired, FileNotFoundError, OSError):
+ return
+ fresh: dict[str, tuple[str, str]] = {}
+ for line in out.splitlines():
+ parts = line.split("|")
+ if len(parts) != 3:
+ continue
+ name, cpu, memu = parts
+ # MemUsage is "85.2MiB / 1.95GiB" — strip the "/ total" half.
+ used = memu.split(" /")[0].strip()
+ # Normalise to our own _format_bytes shape: parse the IEC unit suffix.
+ m = re.match(r"^([0-9.]+)\s*([KMGTPE]?i?B)?$", used)
+ if m:
+ val = float(m.group(1))
+ unit = (m.group(2) or "B").lower()
+ mult = {
+ "b": 1,
+ "kb": 1024, "kib": 1024,
+ "mb": 1024**2, "mib": 1024**2,
+ "gb": 1024**3, "gib": 1024**3,
+ "tb": 1024**4, "tib": 1024**4,
+ }.get(unit, 1)
+ mem_str = _format_bytes(int(val * mult))
+ else:
+ mem_str = used or "—"
+ fresh[name] = (cpu, mem_str)
+ _DOCKER_STATS_CACHE = fresh
+ _DOCKER_STATS_TS = time.time()
+
+
+def proc_uptime_cpu_mem(pid: str) -> tuple[str, str, str]:
+ """For a native PID return (uptime, cpu%, mem) — `—` if not running."""
+ try:
+ out = subprocess.run(
+ ["ps", "-p", pid, "-o", "etime=,pcpu=,rss="],
+ capture_output=True, text=True, timeout=2,
+ ).stdout.strip()
+ except (subprocess.TimeoutExpired, FileNotFoundError, OSError):
+ return ("—", "—", "—")
+ if not out:
+ return ("—", "—", "—")
+ parts = out.split()
+ if len(parts) < 3:
+ return ("—", "—", "—")
+ etime, cpu, rss_kb = parts[0], parts[1], parts[2]
+ secs = _parse_etime(etime)
+ try:
+ mem = _format_bytes(int(rss_kb) * 1024)
+ except ValueError:
+ mem = "—"
+ return (_format_uptime(secs), f"{cpu}%", mem)
+
+
+def container_uptime(svc_name: str) -> str:
+ """ISO StartedAt → human duration. Empty if not started."""
+ container = _docker_container_for(svc_name)
+ try:
+ out = subprocess.run(
+ ["docker", "inspect", "-f", "{{.State.StartedAt}}", container],
+ capture_output=True, text=True, timeout=2,
+ ).stdout.strip()
+ except (subprocess.TimeoutExpired, FileNotFoundError, OSError):
+ return "—"
+ if not out or out.startswith("0001-01-01"):
+ return "—"
+ try:
+ # ISO 8601 with fractional + Z. Trim to the second.
+ trimmed = out.split(".", 1)[0].rstrip("Z")
+ started = datetime.strptime(trimmed, "%Y-%m-%dT%H:%M:%S").replace(tzinfo=timezone.utc)
+ except ValueError:
+ return "—"
+ elapsed = (datetime.now(timezone.utc) - started).total_seconds()
+ return _format_uptime(max(0, int(elapsed)))
+
+
+def _jvm_src_dirs(svc: Service) -> list[Path]:
+ # Derive the source-dir set from the SBT dependency graph in build.sbt.
+ # Services that share another's sbt project (e.g. computing-unit-master
+ # rides amber's dist) enter the graph at the *producing* project via
+ # SHARED_SBT_PROJECT.
+ SHARED_SBT_PROJECT = {
+ "computing-unit-master": "WorkflowExecutionService",
+ }
+ project = svc.sbt_project or SHARED_SBT_PROJECT.get(svc.name)
+ if not project:
+ raise RuntimeError(
+ f"service {svc.name!r} has no sbt_project and no SHARED_SBT_PROJECT "
+ f"mapping — can't compute its source-dir closure"
+ )
+ src_dirs = _transitive_src_dirs_cached(project)
+ return [REPO_ROOT / d for d in src_dirs if (REPO_ROOT / d).exists()]
+
+
+def _jvm_source_files(svc: Service) -> list[Path]:
+ files: list[Path] = []
+ for d in _jvm_src_dirs(svc):
+ for f in d.rglob("*"):
+ if f.is_file() and f.suffix in SOURCE_SUFFIXES:
+ files.append(f)
+ # Sort by raw bytes of the string representation so the order matches
+ # GNU `sort -z` exactly (the shell-side `svc_source_hash` uses that).
+ # Python's default `Path.sort()` is component-wise and could disagree
+ # at e.g. `Foo.scala` vs `Foo/` sibling-name collisions, which would
+ # silently break stamp agreement between the two implementations.
+ files.sort(key=lambda p: str(p).encode())
+ return files
+
+
+def source_hash(svc: Service, files: Optional[list[Path]] = None) -> str:
+ """SHA-1 of every source byte under `svc`'s transitive sbt deps.
+ Pass a pre-computed `files` list to skip the rglob walk — the
+ dirty-check fast path already needs the file list to compare mtimes
+ against the stamp, no point rglob'ing twice."""
+ if files is None:
+ files = _jvm_source_files(svc)
+ h = hashlib.sha1()
+ for f in files:
+ try:
+ h.update(f.read_bytes())
+ except OSError:
+ pass
+ return h.hexdigest()
+
+
+def _newest_mtime_after(files: list[Path], stamp_mtime: float) -> bool:
+ for f in files:
+ try:
+ if f.stat().st_mtime > stamp_mtime:
+ return True
+ except OSError:
+ continue
+ return False
+
+
+def is_dirty(svc: Service) -> bool:
+ """Did the service's relevant source change since the last build?
+
+ JVM: SHA-1 of all .scala/.java/.proto bytes vs. the hash we wrote at the
+ last build, with an mtime fast filter so 99% of ticks are O(stat).
+ yarn/bun: lock vs. node_modules dir mtime — cheap.
+ docker: always clean.
+ """
+ if svc.type == "docker":
+ return False
+ if svc.type == "jvm":
+ return _jvm_is_dirty(svc)
+ if svc.type == "yarn":
+ nm = REPO_ROOT / "frontend" / "node_modules" / ".yarn-state.yml"
+ lock = REPO_ROOT / "frontend" / "yarn.lock"
+ if not nm.exists() or not lock.exists():
+ return True
+ return lock.stat().st_mtime > nm.stat().st_mtime
+ if svc.type == "bun":
+ nm = REPO_ROOT / "agent-service" / "node_modules"
+ lock = REPO_ROOT / "agent-service" / "bun.lock"
+ if not nm.exists() or not lock.exists():
+ return True
+ return lock.stat().st_mtime > nm.stat().st_mtime
+ return False
+
+
+def _jvm_is_dirty(svc: Service) -> bool:
+ stamp = BUILD_STAMP_DIR / svc.name
+ if not stamp.exists() or stamp.stat().st_size == 0:
+ # Lazy seed: if a jar is present, assume it matches current source and
+ # write the hash. First REPL after a fresh checkout pays this once.
+ jar = REPO_ROOT / svc.artifact_jar if svc.artifact_jar else None
+ if jar is None or not jar.exists():
+ return True
+ stamp.write_text(source_hash(svc))
+ return False
+
+ files = _jvm_source_files(svc)
+ stamp_mtime = stamp.stat().st_mtime
+
+ # Fast filter: any source newer than the stamp? If not, definitely clean.
+ if not _newest_mtime_after(files, stamp_mtime):
+ return False
+
+ # Slow path — did the content actually move? Reuse the file list above
+ # instead of having `source_hash` rglob the tree a second time.
+ stored = stamp.read_text().strip()
+ current = source_hash(svc, files)
+ if current == stored:
+ # Same content, only mtimes moved (git checkout / touch). Refresh
+ # the stamp's mtime so the fast filter passes next tick.
+ os.utime(stamp, None)
+ return False
+ return True
+
+
+def artifact_mtime_str(svc: Service) -> Optional[str]:
+ if svc.type == "jvm" and svc.artifact_jar:
+ jar = REPO_ROOT / svc.artifact_jar
+ if jar.exists():
+ return datetime.fromtimestamp(jar.stat().st_mtime).strftime("%m-%d %H:%M")
+ return None
+ if svc.type == "bun":
+ f = REPO_ROOT / "agent-service" / "bun.lock"
+ elif svc.type == "yarn":
+ f = REPO_ROOT / "frontend" / "yarn.lock"
+ else:
+ return None
+ if f.exists():
+ return datetime.fromtimestamp(f.stat().st_mtime).strftime("%m-%d %H:%M")
+ return None
+
+
+# ─────────────────── Banner state (cheap) ───────────────────
+
+def git_head() -> tuple[str, str]:
+ branch = subprocess_run("git", "-C", str(REPO_ROOT), "rev-parse", "--abbrev-ref", "HEAD") or "?"
+ sha = subprocess_run("git", "-C", str(REPO_ROOT), "rev-parse", "--short", "HEAD") or "?"
+ return branch, sha
+
+
+def worktree_info() -> tuple[str, bool]:
+ """Return (label, is_worktree). Label is the leaf directory name of the
+ checkout — for the canonical clone this is `texera`, for a worktree it
+ matches the worktree's directory name (which by our convention reflects
+ the branch). is_worktree distinguishes the main checkout from extras so
+ the banner can flag it."""
+ name = REPO_ROOT.name
+ git_dir = subprocess_run("git", "-C", str(REPO_ROOT), "rev-parse", "--git-dir")
+ common_dir = subprocess_run("git", "-C", str(REPO_ROOT), "rev-parse", "--git-common-dir")
+ is_worktree = False
+ try:
+ if git_dir and common_dir:
+ g = Path(git_dir) if Path(git_dir).is_absolute() else (REPO_ROOT / git_dir)
+ c = Path(common_dir) if Path(common_dir).is_absolute() else (REPO_ROOT / common_dir)
+ is_worktree = g.resolve() != c.resolve()
+ except Exception:
+ pass
+ return name, is_worktree
+
+
+def subprocess_run(*argv: str) -> str:
+ import subprocess as sp
+ try:
+ return sp.check_output(argv, stderr=sp.DEVNULL, text=True).strip()
+ except Exception:
+ return ""
+
+
+# ─────────────────── Input with shell-style history ───────────────────
+
+class CommandHistory:
+ """Pure-Python state machine for command history navigation.
+
+ Kept separate from `HistoricInput` (which subclasses Textual's `Input`)
+ so the navigation logic can be unit-tested without a running app —
+ Textual's reactive setters need an active App context, so they can't
+ be exercised from a bare pytest. `HistoricInput` is a thin wrapper that
+ delegates here and forwards the resulting value to its `Input.value`.
+
+ Conventions match bash/zsh: ↑ walks back from newest to oldest, the
+ in-progress draft is saved when stepping off it the first time, ↓
+ walks forward and restores the draft once you step past the newest
+ entry. Consecutive duplicates are coalesced on `push`."""
+
+ def __init__(self, history_file: Optional[Path] = None, max_size: int = MAX_HISTORY) -> None:
+ self._file = history_file
+ self._max = max_size
+ self._history: list[str] = self._load()
+ self._idx: int = -1 # -1 = at the live draft; 0+ = back in history
+ self._draft: str = ""
+
+ def _load(self) -> list[str]:
+ if not self._file or not self._file.exists():
+ return []
+ try:
+ lines = self._file.read_text(errors="replace").splitlines()
+ return [s for s in (l.strip() for l in lines) if s][-self._max:]
+ except OSError:
+ return []
+
+ def _save(self) -> None:
+ if not self._file:
+ return
+ try:
+ self._file.write_text("\n".join(self._history[-self._max:]) + "\n")
+ except OSError:
+ pass
+
+ def push(self, cmd: str) -> None:
+ cmd = cmd.strip()
+ if not cmd:
+ return
+ if self._history and self._history[-1] == cmd:
+ self._reset()
+ return
+ self._history.append(cmd)
+ self._save()
+ self._reset()
+
+ def _reset(self) -> None:
+ self._idx = -1
+ self._draft = ""
+
+ def back(self, current_value: str) -> Optional[str]:
+ """Step one entry back in history. Returns the new value to display,
+ or None if we're already at the oldest entry (caller should leave
+ the input alone)."""
+ if not self._history:
+ return None
+ if self._idx == -1:
+ self._draft = current_value
+ if self._idx + 1 >= len(self._history):
+ return None
+ self._idx += 1
+ return self._history[-1 - self._idx]
+
+ def forward(self) -> Optional[str]:
+ """Step one entry forward. Returns the draft when you cross the
+ newest entry. Returns None if we weren't browsing history."""
+ if self._idx == -1:
+ return None
+ self._idx -= 1
+ if self._idx == -1:
+ return self._draft
+ return self._history[-1 - self._idx]
+
+
+class HistoricInput(Input):
+ """Textual Input wired up to `CommandHistory` for shell-style ↑/↓.
+
+ History is persisted to `HISTORY_FILE` under STATE_DIR so it survives
+ across REPL sessions."""
+
+ BINDINGS = [
+ Binding("up", "history_back", "history back", show=False),
+ Binding("down", "history_forward", "history forward", show=False),
+ ]
+
+ def __init__(self, *args, **kwargs) -> None:
+ super().__init__(*args, **kwargs)
+ self._hist = CommandHistory(HISTORY_FILE)
+
+ def push(self, cmd: str) -> None:
+ self._hist.push(cmd)
+
+ def _set_value(self, v: str) -> None:
+ self.value = v
+ with contextlib.suppress(Exception):
+ self.cursor_position = len(v)
+
+ def action_history_back(self) -> None:
+ v = self._hist.back(self.value)
+ if v is not None:
+ self._set_value(v)
+
+ def action_history_forward(self) -> None:
+ v = self._hist.forward()
+ if v is not None:
+ self._set_value(v)
+
+
+# ─────────────────── Textual app ───────────────────
+
+class LogResizeHandle(Static):
+ """The one-row grip above the log pane. Drag it up/down with the mouse
+ to resize the log; the height is clamped by the RichLog CSS min/max.
+ Doubles as the static "drag to resize" label when idle."""
+
+ DEFAULT_LABEL = " ⇅ log (drag this row to resize)"
+
+ def __init__(self, **kwargs) -> None:
+ super().__init__(self.DEFAULT_LABEL, **kwargs)
+ self._dragging = False
+ self._drag_origin_y = 0
+ self._drag_origin_h = 0
+
+ def on_mouse_down(self, event: events.MouseDown) -> None:
+ # We only care about the primary button.
+ if event.button != 1:
+ return
+ log = self.app.query_one("#log", RichLog)
+ # `region.height` is the current rendered height in cells. Lock it
+ # in as the baseline so we can compute deltas without races against
+ # auto-layout while the user drags.
+ self._drag_origin_y = event.screen_y
+ self._drag_origin_h = log.region.height
+ self._dragging = True
+ self.capture_mouse(True)
+ # Replace the label so users know they grabbed it.
+ self.update(f" ⇅ resizing… (height {self._drag_origin_h})")
+
+ def on_mouse_move(self, event: events.MouseMove) -> None:
+ if not self._dragging:
+ return
+ # Dragging UP (smaller screen_y) should GROW the log since the log
+ # sits below this handle. Invert the delta and clamp; CSS max/min
+ # are a fallback but the explicit clamp keeps the label honest.
+ delta = self._drag_origin_y - event.screen_y
+ new_h = max(4, min(60, int(self._drag_origin_h + delta)))
+ self.app.query_one("#log", RichLog).styles.height = new_h
+ self.update(f" ⇅ resizing… (height {new_h})")
+
+ def on_mouse_up(self, event: events.MouseUp) -> None:
+ if not self._dragging:
+ return
+ self._dragging = False
+ self.capture_mouse(False)
+ self.update(self.DEFAULT_LABEL)
+
+
+# "Apache Texera" wordmark in box-drawing block characters. Single line,
+# 6 rows × ~104 cols. Coloured teal to match the brand.
+LOGO_TEXERA = (
+ " █████╗ ██████╗ █████╗ ██████╗██╗ ██╗███████╗ ████████╗███████╗██╗ ██╗███████╗██████╗ █████╗ \n"
+ "██╔══██╗██╔══██╗██╔══██╗██╔════╝██║ ██║██╔════╝ ╚══██╔══╝██╔════╝╚██╗██╔╝██╔════╝██╔══██╗██╔══██╗\n"
+ "███████║██████╔╝███████║██║ ███████║█████╗ ██║ █████╗ ╚███╔╝ █████╗ ██████╔╝███████║\n"
+ "██╔══██║██╔═══╝ ██╔══██║██║ ██╔══██║██╔══╝ ██║ ██╔══╝ ██╔██╗ ██╔══╝ ██╔══██╗██╔══██║\n"
+ "██║ ██║██║ ██║ ██║╚██████╗██║ ██║███████╗ ██║ ███████╗██╔╝ ██╗███████╗██║ ██║██║ ██║\n"
+ "╚═╝ ╚═╝╚═╝ ╚═╝ ╚═╝ ╚═════╝╚═╝ ╚═╝╚══════╝ ╚═╝ ╚══════╝╚═╝ ╚═╝╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝"
+)
+
+
+STATE_STYLE = {
+ "running": ("●", "green"),
+ "starting": ("⚠", "yellow"),
+ "unhealthy": ("✗", "red"),
+ "failed": ("✗", "red"),
+ "exited": ("✓", "grey50"),
+ "stopped": ("○", "grey50"),
+}
+
+
+def state_cell(state: str) -> Text:
+ sym, style = STATE_STYLE.get(state, ("○", "grey50"))
+ return Text(f"{sym} {state}", style=style)
+
+
+class LocalDevApp(App):
+ """Live dashboard + REPL for the texera local dev stack."""
+
+ CSS = """
+ Screen { layout: vertical; }
+ #banner {
+ height: 11;
+ background: $boost;
+ color: $text;
+ padding: 0 2;
+ border-bottom: heavy $primary;
+ }
+ /* Collapsed mode: hide the wordmark, leave the title/sub two-liner.
+ Use `Ctrl-B` to toggle — useful on short terminals where the dashboard
+ gets cramped by the logo. */
+ #banner.-collapsed {
+ height: 3;
+ }
+ #banner.-collapsed #banner-logo {
+ display: none;
+ }
+ #banner-logo {
+ color: #20b2aa;
+ text-style: bold;
+ height: 7;
+ }
+ #banner-title { text-style: bold; }
+ #banner-sub { color: $text-muted; }
+ DataTable {
+ height: 1fr;
+ border-bottom: solid $primary-darken-1;
+ }
+ #log-header {
+ height: 1;
+ color: $text-muted;
+ background: $primary-darken-3;
+ padding: 0 2;
+ }
+ #log-header.-hidden { display: none; }
+ #log-header:hover { color: $accent; background: $primary-darken-2; }
+ RichLog {
+ height: 18;
+ min-height: 4;
+ max-height: 60;
+ background: $surface;
+ scrollbar-size: 1 1;
+ padding: 0 1;
+ }
+ RichLog.-hidden { display: none; }
+ #status-bar {
+ height: 1;
+ background: $primary-darken-2;
+ color: $text;
+ padding: 0 2;
+ }
+ Input { background: $surface; }
+ """
+
+ BINDINGS = [
+ # Ctrl-C: first press cancels the current command (or log tail); if
+ # nothing's active, requires a second press within 2 s to actually
+ # quit. Matches the way shells & many TUIs treat it.
+ Binding("ctrl+c", "soft_quit", "Cancel / Quit", priority=True, show=False),
+ Binding("escape", "escape_view", "Exit log view", priority=True, show=False),
+ Binding("ctrl+l", "clear_log", "Clear log", show=False),
+ Binding("ctrl+r", "manual_refresh", "Refresh", show=False),
+ Binding("ctrl+b", "toggle_banner", "Toggle banner", show=False),
+ ]
+
+ # Reactive state — Textual diffs widget content when these change.
+ live: reactive[LiveState] = reactive(LiveState, recompose=False)
+ active_cmd: reactive[Optional[str]] = reactive(None)
+ cmd_started_at: reactive[float] = reactive(0.0)
+ log_log_position: reactive[int] = reactive(0) # byte offset we've already read
+
+ # Non-reactive bookkeeping
+ _branch: str = "?"
+ _sha: str = "?"
+ _last_dirty_check: float = 0.0
+ _cached_dirty: dict[str, bool]
+ _cached_mtimes: dict[str, Optional[str]]
+ _cmd_proc: Optional[asyncio.subprocess.Process] = None
+
+ def __init__(self) -> None:
+ super().__init__()
+ self._cached_dirty = {s.name: False for s in SERVICES}
+ self._cached_mtimes = {s.name: None for s in SERVICES}
+ self._branch, self._sha = git_head()
+ self._worktree_name, self._is_worktree = worktree_info()
+ self._log_visible = False
+ self._log_auto_hide_handle = None # type: ignore
+ self._last_ctrl_c_ts: float = 0.0
+
+ # ── Log visibility ──
+ def _set_log_visible(self, show: bool) -> None:
+ self._log_visible = show
+ log = self.query_one("#log", RichLog)
+ header = self.query_one("#log-header", LogResizeHandle)
+ if show:
+ log.remove_class("-hidden")
+ header.remove_class("-hidden")
+ else:
+ log.add_class("-hidden")
+ header.add_class("-hidden")
+
+ # Column keys we keep so update_cell() can address cells reliably. The
+ # string labels passed to add_columns() are NOT the keys (Textual hands
+ # back auto-generated ColumnKey objects), so doing
+ # `update_cell(row, "STATE", ...)` silently fails.
+ _COL_LABELS = ("●", "SERVICE", "PORT", "PID", "UPTIME", "CPU%", "MEM", "ARTIFACT", "BUILD", "STATE")
+ _COL_KEYS = ("sym", "svc", "port", "pid", "uptime", "cpu", "mem", "mtime", "src", "state")
+ # Min width per column key. Without these, Textual sizes the column to
+ # its header (e.g. PID = 3 chars), then truncates wider cell values
+ # like "76348" → "763", which made every JVM look like it shared a
+ # PID. Numeric/timestamp columns deserve enough room for the
+ # widest plausible value.
+ _COL_MIN_WIDTH = {
+ "svc": 32,
+ "port": 6,
+ "pid": 7, # up to 7-digit PIDs (Linux default max is 4194304)
+ "uptime": 10,
+ "cpu": 7,
+ "mem": 7,
+ "mtime": 18,
+ "src": 5,
+ "state": 12,
+ }
+
+ # ── Layout ──
+ def compose(self) -> ComposeResult:
+ yield Vertical(
+ Static(LOGO_TEXERA, id="banner-logo"),
+ Static("", id="banner-title"),
+ Static("", id="banner-sub"),
+ id="banner",
+ )
+ table = DataTable(zebra_stripes=False, cursor_type="row")
+ for label, key in zip(self._COL_LABELS, self._COL_KEYS):
+ table.add_column(label, key=key, width=self._COL_MIN_WIDTH.get(key))
+ for s in SERVICES:
+ table.add_row("○", s.name, f":{s.port}", "—", "—", "—", "—", "—", " ", "stopped", key=s.name)
+ yield table
+ yield LogResizeHandle(id="log-header", classes="-hidden")
+ yield RichLog(id="log", highlight=False, markup=False, wrap=False,
+ auto_scroll=True, classes="-hidden")
+ yield Static("", id="status-bar")
+ yield HistoricInput(placeholder="type a command (h for help · ↑/↓ history · q to quit)", id="prompt")
+
+ def on_mount(self) -> None:
+ self.title = "Texera Local Dev"
+ self._update_banner()
+ self._update_status_bar()
+ self.query_one("#prompt", Input).focus()
+ # Background polling tasks
+ self.set_interval(POLL_INTERVAL_S, self._tick_state)
+ self.set_interval(0.2, self._tick_log)
+ self.set_interval(0.5, self._tick_banner)
+ # 3 Hz lightweight tick so the `stopping.`/`building..` etc dots
+ # animate smoothly between full state polls. Skips work entirely
+ # when no phase markers are active, so the steady-state cost is a
+ # 14-row dict lookup.
+ self.set_interval(0.33, self._tick_phase_animation)
+ # Kick the first poll immediately so the table populates fast.
+ self.call_later(self._tick_state)
+ self.call_later(self._tick_log)
+
+ # ── Updates ──
+ def _update_banner(self) -> None:
+ now = datetime.now().strftime("%H:%M:%S")
+ wt_tag = f"worktree: {self._worktree_name}" if self._is_worktree else f"checkout: {self._worktree_name}"
+ sub = f"{wt_tag} · branch: {self._branch} @ {self._sha} · {now}"
+ self.query_one("#banner-title", Static).update("Apache Texera — Local Dev")
+ self.query_one("#banner-sub", Static).update(sub)
+
+ def _update_status_bar(self) -> None:
+ running = sum(
+ 1 for s in SERVICES
+ if (s.type == "docker" and docker_state(*self.live.docker.get(s.name, ("", ""))) == "running")
+ or (s.type != "docker" and self.live.pids.get(s.name))
+ )
+ total = len(SERVICES)
+ dirty = sum(1 for d in self.live.dirty.values() if d)
+ active = self.active_cmd or "idle"
+ elapsed = ""
+ if self.active_cmd and self.cmd_started_at:
+ elapsed = f" ({int(time.monotonic() - self.cmd_started_at)}s)"
+ dirty_part = f" ★ {dirty} dirty" if dirty else ""
+ # Surface the frontend URL the moment ng serve is listening, so the
+ # user knows when the web app is actually clickable.
+ frontend_pid = self.live.pids.get("frontend")
+ url_part = " → http://localhost:4200" if frontend_pid else ""
+ self.query_one("#status-bar", Static).update(
+ f"{running}/{total} running{dirty_part}{url_part} last: {active}{elapsed}"
+ )
+
+ # State-cell renderer: when the shell has written a transitional phase
+ # (stopping / building / starting), we display that with cycling dots
+ # ("building." → "building.." → "building...") so the user sees motion
+ # while the per-service action is in flight. Cycles at 3 Hz off the
+ # main poll tick.
+ _PHASE_PALETTE = {
+ "stopping": ("◐", "yellow"),
+ "building": ("⚙", "cyan"),
+ "starting": ("⚠", "yellow"),
+ }
+
+ def _phase_text(self, phase: str) -> Text:
+ sym, style = self._PHASE_PALETTE.get(phase, ("⚠", "yellow"))
+ dots = "." * (int(time.monotonic() * 3) % 3 + 1)
+ return Text(f"{phase}{dots}", style=style)
+
+ def _refresh_table(self) -> None:
+ table = self.query_one(DataTable)
+ for svc in SERVICES:
+ phase = self.live.phases.get(svc.name, "")
+ if svc.type == "docker":
+ ds, dstatus = self.live.docker.get(svc.name, ("", ""))
+ state = docker_state(ds, dstatus)
+ pid = "—"
+ else:
+ pid = self.live.pids.get(svc.name) or "—"
+ state = "running" if self.live.pids.get(svc.name) else "stopped"
+ # If the shell signalled a transitional phase AND we haven't
+ # observed the service as fully running yet, render the phase
+ # instead. Once the poller confirms "running" we trust that
+ # over a stale stop/build/start marker.
+ if phase and state != "running":
+ sym, style = self._PHASE_PALETTE.get(phase, ("⚠", "yellow"))
+ state_cell_text = self._phase_text(phase)
+ else:
+ sym, style = STATE_STYLE.get(state, ("○", "grey50"))
+ state_cell_text = Text(state, style=style)
+ mtime = self.live.mtimes.get(svc.name) or "—"
+ dirty = self.live.dirty.get(svc.name, False)
+
+ # BUILD column tells the user whether they need to do anything to
+ # bring the running service in sync with the current source.
+ # ★ (yellow) — content/lock changed since last build; needs action.
+ # ↻ (cyan) — service auto-rebuilds on file change (ng serve /
+ # bun --watch). Reassurance that no manual step is
+ # needed for source edits.
+ # (blank) — built and up-to-date.
+ if dirty:
+ src_cell: Text | str = Text("★", style="bold yellow")
+ elif svc.type in WATCH_TYPES:
+ src_cell = Text("↻", style="cyan")
+ else:
+ src_cell = " "
+
+ uptime = self.live.uptimes.get(svc.name, "—")
+ cpu = self.live.cpu_pct.get(svc.name, "—")
+ mem = self.live.mem_use.get(svc.name, "—")
+ table.update_cell(svc.name, "sym", Text(sym, style=style))
+ table.update_cell(svc.name, "port", f":{svc.port}")
+ table.update_cell(svc.name, "pid", str(pid))
+ table.update_cell(svc.name, "uptime", uptime)
+ table.update_cell(svc.name, "cpu", cpu)
+ table.update_cell(svc.name, "mem", mem)
+ table.update_cell(svc.name, "mtime", mtime)
+ table.update_cell(svc.name, "src", src_cell)
+ table.update_cell(svc.name, "state", state_cell_text)
+
+ # ── Polling tasks (Textual will run them on the event loop) ──
+ @work(exclusive=True, group="state")
+ async def _tick_state(self) -> None:
+ # Polling cost ≈ 1 docker compose ps + lsof × N native services, run
+ # concurrently. ~200 ms total on this box.
+ docker_task = asyncio.create_task(docker_ps_all())
+ native_tasks = {
+ s.name: asyncio.create_task(lsof_port_pid(s.port))
+ for s in SERVICES if s.type != "docker"
+ }
+ docker_map = await docker_task
+ pids = {name: await task for name, task in native_tasks.items()}
+
+ # Dirty check is more expensive; only re-do every DIRTY_INTERVAL_S.
+ now = time.monotonic()
+ if now - self._last_dirty_check >= DIRTY_INTERVAL_S:
+ loop = asyncio.get_running_loop()
+ self._cached_dirty = {
+ s.name: await loop.run_in_executor(None, is_dirty, s) for s in SERVICES
+ }
+ self._cached_mtimes = {s.name: artifact_mtime_str(s) for s in SERVICES}
+ self._last_dirty_check = now
+
+ # Per-service uptime / cpu / mem.
+ # Native: cheap `ps -o etime,pcpu,rss` per pid (<5ms each).
+ # Docker: cached via `docker stats --no-stream` refreshed by a
+ # background worker every _DOCKER_STATS_TTL seconds (the
+ # call itself takes ~2s, too slow to run in-line each tick).
+ uptimes: dict[str, str] = {}
+ cpus: dict[str, str] = {}
+ mems: dict[str, str] = {}
+ for s in SERVICES:
+ if s.type == "docker":
+ state = docker_state(*docker_map.get(s.name, ("", "")))
+ if state == "running":
+ uptimes[s.name] = container_uptime(s.name)
+ container = _docker_container_for(s.name)
+ cpu_mem = _DOCKER_STATS_CACHE.get(container)
+ if cpu_mem:
+ cpus[s.name], mems[s.name] = cpu_mem
+ else:
+ cpus[s.name] = "…"; mems[s.name] = "…"
+ else:
+ uptimes[s.name] = "—"; cpus[s.name] = "—"; mems[s.name] = "—"
+ else:
+ pid = pids.get(s.name)
+ if pid:
+ up, cpu, mem = proc_uptime_cpu_mem(pid)
+ uptimes[s.name] = up; cpus[s.name] = cpu; mems[s.name] = mem
+ else:
+ uptimes[s.name] = "—"; cpus[s.name] = "—"; mems[s.name] = "—"
+ # Kick a docker-stats refresh if the cache is stale; runs in
+ # background so it never blocks the tick.
+ if time.time() - _DOCKER_STATS_TS >= _DOCKER_STATS_TTL:
+ self._refresh_docker_stats_worker()
+
+ phases = {s.name: (read_svc_phase(s.name) or "") for s in SERVICES}
+ new_state = LiveState(
+ docker=docker_map,
+ pids=pids,
+ dirty=dict(self._cached_dirty),
+ mtimes=dict(self._cached_mtimes),
+ uptimes=uptimes,
+ cpu_pct=cpus,
+ mem_use=mems,
+ phases=phases,
+ )
+ self.live = new_state
+ self._refresh_table()
+ self._update_status_bar()
+
+ @work(exclusive=True, group="docker_stats")
+ async def _refresh_docker_stats_worker(self) -> None:
+ loop = asyncio.get_running_loop()
+ await loop.run_in_executor(None, _refresh_docker_stats)
+
+ def _tick_banner(self) -> None:
+ self._update_banner()
+
+ def _tick_phase_animation(self) -> None:
+ """Refresh only the STATE cells whose phase is mid-transition, so
+ the trailing dots animate at 3 Hz without a full poll."""
+ active = {
+ name: phase for name, phase in self.live.phases.items() if phase
+ }
+ if not active:
+ return
+ table = self.query_one(DataTable)
+ for name, phase in active.items():
+ # Skip if the poller has already declared the service running —
+ # don't fight the steady-state render.
+ svc = SERVICES_BY_NAME.get(name)
+ if svc is None:
+ continue
+ if svc.type == "docker":
+ ds, dstatus = self.live.docker.get(name, ("", ""))
+ if docker_state(ds, dstatus) == "running":
+ continue
+ else:
+ if self.live.pids.get(name):
+ continue
+ with contextlib.suppress(Exception):
+ table.update_cell(name, "state", self._phase_text(phase))
+
+ @work(exclusive=True, group="log")
+ async def _tick_log(self) -> None:
+ if not REPL_LOG.exists():
+ return
+ try:
+ size = REPL_LOG.stat().st_size
+ except OSError:
+ return
+ if size < self.log_log_position:
+ # File was truncated (new command). Reset and reread tail.
+ self.log_log_position = 0
+ self.query_one("#log", RichLog).clear()
+ if size == self.log_log_position:
+ return
+ with REPL_LOG.open("rb") as f:
+ f.seek(self.log_log_position)
+ new_bytes = f.read()
+ self.log_log_position = size
+ text = new_bytes.decode(errors="replace")
+ log = self.query_one("#log", RichLog)
+ for raw_line in text.splitlines():
+ log.write(_strip_ansi_motion(raw_line))
+
+ # ── Command handling ──
+ def on_input_submitted(self, message: Input.Submitted) -> None:
+ cmd = message.value.strip()
+ message.input.value = ""
+ if isinstance(message.input, HistoricInput) and cmd:
+ message.input.push(cmd)
+ if not cmd:
+ return
+ if cmd in ("q", "quit", "exit"):
+ self.exit()
+ return
+ if cmd in ("h", "?", "help"):
+ self._show_help()
+ return
+ if cmd in ("r", "refresh"):
+ self.call_later(self._tick_state)
+ return
+ if cmd in ("clear",):
+ self.query_one("#log", RichLog).clear()
+ self._set_log_visible(False)
+ return
+ if cmd in ("log",):
+ # Toggle log pane visibility manually.
+ self._set_log_visible(not self._log_visible)
+ return
+ if cmd in ("banner",):
+ self.action_toggle_banner()
+ return
+ self._dispatch(cmd)
+
+ def _show_help(self) -> None:
+ log = self.query_one("#log", RichLog)
+ log.clear()
+ self._set_log_visible(True)
+ for line in [
+ "Commands:",
+ " r refresh state now",
+ " u build + start every service",
+ " u <svc> start one service (no rebuild)",
+ " d stop every service",
+ " d <svc> stop one service",
+ " b force incremental sbt + node deps",
+ " a / auto scan for dirty services and rebuild+bounce only those",
+ " <svc> rebuild that service and bounce it",
+ " l <svc> tail that service's log (Ctrl-C returns)",
+ " s <svc> stop one service",
+ " clear clear the log pane",
+ " log toggle log pane visibility",
+ " banner toggle banner (collapse the wordmark to save rows; Ctrl-B also works)",
+ " q quit",
+ "",
+ "Mouse: double-click a service row → tail its log.",
+ " double-click the banner → collapse / expand the wordmark.",
+ " Enter on a focused row does the same as double-click.",
+ "",
+ "BUILD column:",
+ " ★ (yellow) source or deps changed since last build — rebuild needed",
+ " ↻ (cyan) service auto-rebuilds on save (ng serve / bun --watch)",
+ " (blank) built and up-to-date",
+ "",
+ f"Known services: {', '.join(s.name for s in SERVICES)}",
+ ]:
+ log.write(line)
+
+ def _dispatch(self, cmd: str) -> None:
+ if self._cmd_proc and self._cmd_proc.returncode is None:
+ log = self.query_one("#log", RichLog)
+ log.write(Text(f"busy: '{self.active_cmd}' still running. Ctrl-C in the term to abort.", style="bold yellow"))
+ return
+
+ parts = cmd.split(None, 1)
+ verb = parts[0]
+ arg = parts[1] if len(parts) > 1 else ""
+
+ # Resolve to a bin/local-dev.sh invocation. Keeping the shell script
+ # as the canonical engine so behavior matches `bin/local-dev.sh up`
+ # from a terminal.
+ argv: Optional[list[str]] = None
+ if verb in ("u", "up"):
+ if arg:
+ if arg not in SERVICES_BY_NAME:
+ self._log_err(f"unknown service: {arg}")
+ return
+ argv = ["start", arg]
+ else:
+ argv = ["up"]
+ elif verb in ("d", "down"):
+ if arg:
+ if arg not in SERVICES_BY_NAME:
+ self._log_err(f"unknown service: {arg}")
+ return
+ argv = ["stop", arg]
+ else:
+ argv = ["down"]
+ elif verb in ("s", "stop"):
+ if not arg or arg not in SERVICES_BY_NAME:
+ self._log_err("usage: s <service>")
+ return
+ argv = ["stop", arg]
+ elif verb in ("b", "build"):
+ # Force an incremental build; the shell handles the "is this
+ # really needed" decision itself (it pre-bounces JVMs etc.).
+ argv = ["up", "--build"]
+ elif verb in ("a", "auto"):
+ # Scan for dirty services and rebuild + bounce only those.
+ argv = ["auto"]
+ elif verb in ("l", "logs", "tail"):
+ if not arg or arg not in SERVICES_BY_NAME:
+ self._log_err(f"usage: l <service> (known: {', '.join(s.name for s in SERVICES)})")
+ return
+ argv = ["logs", arg]
+ self._spawn_logs(arg)
+ return
+ elif verb in SERVICES_BY_NAME:
+ svc_obj = SERVICES_BY_NAME[verb]
+ if svc_obj.type in WATCH_TYPES:
+ # ng serve / bun --watch rebuild on save automatically. The
+ # shell's cmd_update_one already refuses this, but we get a
+ # nicer message by intercepting here.
+ self._log_msg(
+ f"{verb} runs in watch mode (↻) — source edits auto-reload. "
+ f"If you really need to bounce it, run `s {verb}` then `u {verb}`."
+ )
+ return
+ argv = [verb]
+ else:
+ self._log_err(f"unknown: {verb} (type 'h' for help)")
+ return
+
+ if argv is None:
+ return
+ self._spawn_action(verb if not arg else f"{verb} {arg}", argv)
+
+ def _log_err(self, msg: str) -> None:
+ self._set_log_visible(True)
+ self.query_one("#log", RichLog).write(Text("✗ " + msg, style="bold red"))
+
+ def _log_msg(self, msg: str) -> None:
+ self._set_log_visible(True)
+ self.query_one("#log", RichLog).write(Text("• " + msg, style="cyan"))
+ # Auto-dismiss the info pop after a few seconds.
+ if self._log_auto_hide_handle is not None:
+ with contextlib.suppress(Exception):
+ self._log_auto_hide_handle.stop()
+ self._log_auto_hide_handle = self.set_timer(4.0, self._auto_hide_log)
+
+ def _spawn_logs(self, svc: str) -> None:
+ log = self.query_one("#log", RichLog)
+ log.clear()
+ self._set_log_visible(True)
+ log.write(Text(f"── tailing {svc} (type any other command to return) ──", style="dim"))
+ self.active_cmd = f"logs {svc}"
+ self.cmd_started_at = time.monotonic()
+ self._tail_service_log(svc)
+
+ @work(exclusive=True, group="cmd")
+ async def _tail_service_log(self, svc: str) -> None:
+ log_widget = self.query_one("#log", RichLog)
+ svc_obj = SERVICES_BY_NAME[svc]
+ if svc_obj.type == "docker":
+ proc = await asyncio.create_subprocess_exec(
+ "docker", "compose", "-p", DOCKER_PROJECT, "logs", "-f", svc,
+ stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.STDOUT,
+ )
+ else:
+ log_path = LOG_DIR / f"{svc}.log"
+ if not log_path.exists():
+ log_path.touch()
+ proc = await asyncio.create_subprocess_exec(
+ "tail", "-n", "200", "-f", str(log_path),
+ stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.STDOUT,
+ )
+ self._cmd_proc = proc
+ try:
+ assert proc.stdout
+ async for raw in proc.stdout:
+ # If a newer action (e.g. `up`) has claimed _cmd_proc, our
+ # exclusive-group cancellation is in flight but hasn't hit
+ # the next await yet — stop writing to the widget *now* so
+ # we don't leak frontend.log lines into the up output.
+ if self._cmd_proc is not proc:
+ break
+ log_widget.write(_strip_ansi_motion(raw.decode(errors="replace").rstrip("\n")))
+ finally:
+ # ESC fires _cancel_active_cmd which terminates this proc AND
+ # immediately clears active_cmd/_cmd_proc so the user can run
+ # another command. By the time our async-for unwinds the proc
+ # we're holding, the user may have already kicked off `up`. Only
+ # clear the global state if it still belongs to us — otherwise
+ # we'd clobber a freshly-spawned command's state to idle, and
+ # any subsequent up/down would see a stale view.
+ if self._cmd_proc is proc:
+ self.active_cmd = None
+ self._cmd_proc = None
+
+ @work(exclusive=True, group="cmd")
+ async def _spawn_action(self, label: str, argv: list[str]) -> None:
+ log = self.query_one("#log", RichLog)
+ # Cancel any pending auto-hide from a previous command.
+ if self._log_auto_hide_handle is not None:
+ with contextlib.suppress(Exception):
+ self._log_auto_hide_handle.stop()
+ self._log_auto_hide_handle = None
+ # Trip _tail_service_log's ownership check (`self._cmd_proc is not
+ # proc → break`) immediately. Otherwise the tail keeps draining
+ # frontend.log into the log widget across our log.clear() + the
+ # async point at create_subprocess_exec below, and the user sees
+ # frontend output mixed into the up panel.
+ self._cmd_proc = None
+ # Truncate REPL_LOG so the log pane only shows this command's output.
+ REPL_LOG.write_text("")
+ log.clear()
+ self._set_log_visible(True)
+ log.write(Text(f"── {label} → bin/local-dev.sh {' '.join(shlex.quote(a) for a in argv)} ──", style="dim"))
+ self.active_cmd = label
+ self.cmd_started_at = time.monotonic()
+ self.log_log_position = 0
+
+ with REPL_LOG.open("wb") as out:
+ proc = await asyncio.create_subprocess_exec(
+ str(LOCAL_DEV_SH), *argv,
+ stdout=out, stderr=asyncio.subprocess.STDOUT,
+ cwd=str(REPO_ROOT),
+ )
+ self._cmd_proc = proc
+ await proc.wait()
+ rc = proc.returncode or 0
+ style = "bold green" if rc == 0 else "bold red"
+ log.write(Text(f"── {label}: done (exit {rc}) ──", style=style))
+ # Same guard as _tail_service_log: only release the global state if
+ # it still points at our proc. Otherwise a faster newer command has
+ # already taken over and we mustn't reset it to idle.
+ if self._cmd_proc is proc:
+ self.active_cmd = None
+ self._cmd_proc = None
+ # Right after a command, source state likely moved — force a state poll.
+ self._last_dirty_check = 0
+ self.call_later(self._tick_state)
+
+ # Successful command → auto-hide the log so the dashboard reclaims the
+ # space. Failure stays visible so the user can read the error.
+ if rc == 0:
+ self._log_auto_hide_handle = self.set_timer(3.0, self._auto_hide_log)
+
+ def _auto_hide_log(self) -> None:
+ # Only hide if no new command came in and the user hasn't engaged with
+ # the log (we keep it pinned during interactive log tailing).
+ if self.active_cmd is None:
+ self._set_log_visible(False)
+ self._log_auto_hide_handle = None
+
+ def action_toggle_banner(self) -> None:
+ """Collapse / expand the ASCII wordmark to reclaim ~7 rows."""
+ banner = self.query_one("#banner")
+ if banner.has_class("-collapsed"):
+ banner.remove_class("-collapsed")
+ else:
+ banner.add_class("-collapsed")
+
+ def action_clear_log(self) -> None:
+ self.query_one("#log", RichLog).clear()
+ self._set_log_visible(False)
+
+ def action_manual_refresh(self) -> None:
+ self.call_later(self._tick_state)
+
+ # ── ESC: leave whatever transient view we're in ──
+ def action_escape_view(self) -> None:
+ # If a log tail (or any running command) is up, cancel it.
+ if self.active_cmd is not None:
+ self._cancel_active_cmd()
+ # Hide the log pane regardless — ESC's main job is to give the
+ # dashboard the screen back.
+ if self._log_visible:
+ self._set_log_visible(False)
+ # Make sure the prompt has focus so the user can immediately type.
+ self.query_one("#prompt", HistoricInput).focus()
+
+ # ── Ctrl-C: cancel current work, or quit on a second tap ──
+ def action_soft_quit(self) -> None:
+ # 1. Active command (build / up / log tail …) → kill it, keep the
+ # REPL open.
+ if self.active_cmd is not None:
+ self._cancel_active_cmd()
+ self.notify("Ctrl-C — cancelled current command", timeout=2)
+ self._last_ctrl_c_ts = 0.0
+ return
+ # 2. Idle: require a second Ctrl-C within 2 s to actually exit. This
+ # prevents an accidental ⌃-C from killing a session you wanted to
+ # keep.
+ now = time.monotonic()
+ if now - self._last_ctrl_c_ts < 2.0:
+ self.exit()
+ return
+ self._last_ctrl_c_ts = now
+ self.notify("Press Ctrl-C again within 2 s to quit", timeout=2)
+
+ def _cancel_active_cmd(self) -> None:
+ proc = self._cmd_proc
+ if proc and proc.returncode is None:
+ with contextlib.suppress(ProcessLookupError, OSError):
+ proc.terminate()
+ # Cancel the worker too. proc.terminate() races with stdout draining
+ # — until the worker's `async for` actually returns, Textual still
+ # considers it alive in group "cmd", which can block the next
+ # `@work(exclusive=True, group="cmd")` from ever starting. That's
+ # the bug behind "tap into a log → ESC → `down` does nothing".
+ with contextlib.suppress(Exception):
+ self.workers.cancel_group(self, "cmd")
+ self.active_cmd = None
+ self._cmd_proc = None
+
+ # ── Mouse: double-click a service row to tail its log ──
+ def on_click(self, event: events.Click) -> None:
+ widget = getattr(event, "control", None) or getattr(event, "widget", None)
+
+ # Double-click on any banner element → toggle (collapse / expand).
+ # Lets the user reclaim the wordmark's rows without remembering the
+ # Ctrl-B / `banner` aliases.
+ if event.chain == 2 and self._widget_in_banner(widget):
+ self.action_toggle_banner()
+ return
+
+ # Double-click on a service row → tail that service's log.
+ if event.chain == 2:
+ table = self.query_one(DataTable)
+ if widget is table:
+ row = table.cursor_row
+ if 0 <= row < len(SERVICES):
+ self._spawn_logs(SERVICES[row].name)
+ self.query_one("#prompt", Input).focus()
+
+ def _widget_in_banner(self, widget) -> bool:
+ """True when `widget` is the banner container or any of its
+ children. We walk the parent chain rather than matching IDs so a
+ click on the wordmark / title / sub line all count."""
+ node = widget
+ while node is not None:
+ if getattr(node, "id", None) == "banner":
+ return True
+ node = getattr(node, "parent", None)
+ return False
+
+ # Enter on a focused row works too (keyboard equivalent of double-click).
+ def on_data_table_row_selected(self, event: DataTable.RowSelected) -> None:
+ svc = str(event.row_key.value) if event.row_key.value else ""
+ if svc and svc in SERVICES_BY_NAME:
+ self._spawn_logs(svc)
+
+
+# ─────────────────── ANSI motion stripper for log ───────────────────
+
+_CSI_NON_SGR = re.compile(r"\x1b\[[0-9;?]*[A-LN-Za-ln-z]") # everything except SGR (m)
+_CR = re.compile(r"\r+")
+
+
+def _strip_ansi_motion(s: str) -> str:
+ """Strip cursor-motion / erase-screen CSI sequences while keeping SGR colors,
+ and collapse \\r so spinner frames don't pile up in the log pane."""
+ s = _CSI_NON_SGR.sub("", s)
+ s = _CR.sub("", s)
+ return s
+
+
+# ─────────────────── Entry point ───────────────────
+
+def main() -> None:
+ if not LOCAL_DEV_SH.exists():
+ print(f"FATAL: {LOCAL_DEV_SH} not found", file=__import__("sys").stderr)
+ raise SystemExit(1)
+ if shutil.which("docker") is None:
+ print("FATAL: docker not on PATH", file=__import__("sys").stderr)
+ raise SystemExit(1)
+ LocalDevApp().run()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/bin/local-dev.sh b/bin/local-dev.sh
new file mode 100755
index 0000000000..a89eb29a4d
--- /dev/null
+++ b/bin/local-dev.sh
@@ -0,0 +1,2536 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# bin/local-dev.sh -- Manage the Texera local dev stack from a single script.
+#
+# Subcommands:
+# bin/local-dev.sh DEFAULT — one-shot text
+# dashboard (same as `status`).
+# Non-interactive, prints once and
+# exits — safe in scripts/CI.
+# bin/local-dev.sh -i | --interactive Launch the Textual TUI dashboard
+# (live states, SRC dirty
+# indicator, command prompt,
+# double-click for logs, ↑/↓
+# history, Ctrl-C twice to quit).
+# Requires Python + textual.
+# bin/local-dev.sh status same as no-arg invocation.
+# bin/local-dev.sh up [--fresh|--build|--no-build] [--skip=svc1,svc2]
+# Default: skip build if no source/lock
+# changes since last build. --build forces
+# incremental sbt dist + yarn/bun install.
+# --fresh runs `sbt clean dist`. --no-build
+# skips the build step entirely.
+# bin/local-dev.sh down [--skip=svc1,svc2] stop every non-skipped service.
+# bin/local-dev.sh start <service> start one service (no rebuild).
+# bin/local-dev.sh stop <service> stop one service.
+# bin/local-dev.sh <service> rebuild only that service incrementally
+# (sbt <Project>/dist), then bounce it.
+# frontend / agent-service are refused
+# (they have their own watch mode).
+# bin/local-dev.sh logs <service> tail this service's log.
+# bin/local-dev.sh w | watch [interval] Hands-off monitor: redraw the
+# dashboard every <interval>s
+# (default 2). No prompt; Ctrl-C
+# to exit.
+#
+# Managed services (start order):
+# config-service :9094 JVM (sbt ConfigService)
+# access-control-service :9096 JVM (sbt AccessControlService)
+# file-service :9092 JVM (sbt FileService)
+# workflow-compiling-service :9090 JVM (sbt WorkflowCompilingService)
+# computing-unit-managing-service :8082 JVM (sbt ComputingUnitManagingService)
+# texera-web :8080 JVM (sbt WorkflowExecutionService, amber)
+# computing-unit-master :8085 JVM (rides amber dist; no own sbt project)
+# agent-service :3001 Bun --watch (cd agent-service && bun run dev)
+# frontend :4200 ng serve via cd frontend && yarn start
+#
+# Docker infra (postgres / minio / lakefs / lakekeeper / litellm) is NOT managed
+# here -- bring those up yourself before `up`. The script will warn if expected
+# ports are unreachable.
+#
+# Logs and pid book-keeping live under: ${TEXERA_LOCAL_DEV_DIR:-/tmp/texera-local-dev}
+
+set -euo pipefail
+# Unmatched globs in bash default to the literal pattern (we handle that
+# in-place at every glob site). `failglob` / `nullglob` are opt-in per glob
+# via `( shopt -s nullglob; ... )` subshells where we need empty-on-no-match.
+
+REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+cd "$REPO_ROOT"
+
+STATE_DIR="${TEXERA_LOCAL_DEV_DIR:-/tmp/texera-local-dev}"
+LOG_DIR="$STATE_DIR/logs"
+BUILD_STAMP_DIR="$STATE_DIR/build-stamps"
+# Per-service phase markers: shell writes `<phase>\t<epoch>` here as it
+# walks each service through stop → build → start; the TUI reads them
+# every tick and renders an animated transitional state in the STATE
+# column. Removed once the service is up / on stale-after-90s.
+PHASE_DIR="$STATE_DIR/svc-phase"
+mkdir -p "$LOG_DIR" "$BUILD_STAMP_DIR" "$PHASE_DIR"
+
+# --------- associative-array shim for bash 3.2 ---------
+# Apple ships bash 3.2 at /bin/bash and we ship licensing as bash 3.2 too,
+# so we can't use `declare -A`. Every old `MAP[$key]=val` / `${MAP[$key]}`
+# is reshaped as a function call against a flat namespace of synthetic
+# variables `_amap__<MAP>__<key>`. Keys containing `-` get mangled to `_`
+# so they're valid identifier chars.
+amap_set() {
+ local _map="$1" _key="${2//-/_}" _val="$3"
+ eval "_amap__${_map}__${_key}=\$_val"
+}
+amap_get() {
+ local _map="$1" _key="${2//-/_}"
+ local _var="_amap__${_map}__${_key}"
+ eval "printf '%s' \"\${$_var:-}\""
+}
+amap_has() {
+ local _map="$1" _key="${2//-/_}"
+ local _var="_amap__${_map}__${_key}"
+ eval "[[ \${$_var+x} ]]"
+}
+amap_append() {
+ # amap_append MAP key suffix — append to existing value (or seed it).
+ local _map="$1" _key="${2//-/_}" _suffix="$3"
+ local _var="_amap__${_map}__${_key}"
+ eval "$_var=\"\${$_var:-}\$_suffix\""
+}
+
+# --------- toolchain (JDK 17 + node) ---------
+# Detect a JDK 17 installation rather than pinning one path. We try, in
+# order: (1) caller-set $JAVA_HOME if it really is 17, (2) macOS's official
+# locator `/usr/libexec/java_home -v 17`, (3) Homebrew on Apple Silicon +
+# Intel, (4) common Linux distro paths (openjdk / temurin / corretto / zulu),
+# (5) SDKMAN, (6) asdf, (7) the `java` on PATH if its `-version` says 17.
+# Fall through to a clear install hint if none match.
+_java_is_17() {
+ local home="$1"
+ [[ -x "$home/bin/java" ]] || return 1
+ "$home/bin/java" -version 2>&1 | head -1 | grep -q '"17[.]' || return 1
+ return 0
+}
+
+_find_jdk17() {
+ local cand=""
+ # 1. Respect $JAVA_HOME if the caller already set it AND it's 17.
+ if [[ -n "${JAVA_HOME:-}" ]] && _java_is_17 "$JAVA_HOME"; then
+ printf '%s\n' "$JAVA_HOME"; return 0
+ fi
+ # 2. macOS native locator (works for any vendor installed via /Library).
+ if command -v /usr/libexec/java_home >/dev/null 2>&1; then
+ cand=$(/usr/libexec/java_home -v 17 2>/dev/null) || cand=""
+ if [[ -n "$cand" ]] && _java_is_17 "$cand"; then
+ printf '%s\n' "$cand"; return 0
+ fi
+ fi
+ # 3. Homebrew — try `brew --prefix openjdk@17` first, then both well-
+ # known prefixes as a fallback (script may run without brew on PATH
+ # if /etc/zprofile didn't fire).
+ if command -v brew >/dev/null 2>&1; then
+ cand=$(brew --prefix openjdk@17 2>/dev/null) || cand=""
+ [[ -n "$cand" ]] && _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ fi
+ for cand in /opt/homebrew/opt/openjdk@17 /usr/local/opt/openjdk@17; do
+ _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ done
+ # 4. Linux distro layouts. Glob first match.
+ # bash equivalents of zsh's `*(N)` "null on no match" qualifier: enable
+ # `shopt -s nullglob` locally so an unmatched pattern expands to zero
+ # words. We localize it in a subshell so it doesn't leak out and break
+ # the unquoted globs elsewhere (e.g. unzip's zip glob).
+ local matched=""
+ matched=$(shopt -s nullglob; \
+ for cand in \
+ /usr/lib/jvm/java-17-openjdk* \
+ /usr/lib/jvm/temurin-17-jdk* \
+ /usr/lib/jvm/java-17-amazon-corretto* \
+ /usr/lib/jvm/zulu-17* \
+ /usr/lib/jvm/jdk-17*; do
+ printf '%s\n' "$cand"
+ done)
+ while IFS= read -r cand; do
+ [[ -z "$cand" ]] && continue
+ _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ done <<< "$matched"
+ # 5. SDKMAN (`sdk install java 17.x-...`) — pick the lex-largest 17.* dir.
+ if [[ -d "$HOME/.sdkman/candidates/java" ]]; then
+ matched=$(shopt -s nullglob; \
+ for cand in "$HOME"/.sdkman/candidates/java/17.*; do
+ printf '%s\n' "$cand"
+ done)
+ while IFS= read -r cand; do
+ [[ -z "$cand" ]] && continue
+ _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ done <<< "$matched"
+ fi
+ # 6. asdf.
+ if [[ -d "$HOME/.asdf/installs/java" ]]; then
+ matched=$(shopt -s nullglob; \
+ for cand in "$HOME"/.asdf/installs/java/*17*; do
+ printf '%s\n' "$cand"
+ done)
+ while IFS= read -r cand; do
+ [[ -z "$cand" ]] && continue
+ _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ done <<< "$matched"
+ fi
+ # 7. Whatever `java` is on PATH, IF it's 17 — covers cases like Docker
+ # images or distro-managed defaults.
+ cand=$(command -v java 2>/dev/null) || cand=""
+ if [[ -n "$cand" ]]; then
+ cand="$(dirname "$(dirname "$cand")")"
+ _java_is_17 "$cand" && { printf '%s\n' "$cand"; return 0; }
+ fi
+ return 1
+}
+
+# Print the current java environment alongside detection results so the
+# user can see *why* JDK 17 lookup failed (wrong version pinned, JAVA_HOME
+# pointing at JDK 21, etc) — not just "couldn't find one".
+_diagnose_jdk17() {
+ echo "" >&2
+ echo " current java environment:" >&2
+ if [[ -n "${JAVA_HOME:-}" ]]; then
+ local jhver=""
+ jhver=$("$JAVA_HOME/bin/java" -version 2>&1 | head -1 || echo "(unreadable)")
+ echo " \$JAVA_HOME = $JAVA_HOME" >&2
+ echo " ↳ $jhver" >&2
+ else
+ echo " \$JAVA_HOME (unset)" >&2
+ fi
+ local path_java=""
+ path_java=$(command -v java 2>/dev/null) || true
+ if [[ -n "$path_java" ]]; then
+ local pver=""
+ pver=$("$path_java" -version 2>&1 | head -1 || echo "(unreadable)")
+ echo " \`java\` on PATH → $path_java" >&2
+ echo " ↳ $pver" >&2
+ else
+ echo " \`java\` on PATH (not found)" >&2
+ fi
+ if command -v /usr/libexec/java_home >/dev/null 2>&1; then
+ local jh17=""
+ jh17=$(/usr/libexec/java_home -v 17 2>/dev/null) || jh17="(none registered)"
+ echo " /usr/libexec/java_home -v 17 → $jh17" >&2
+ fi
+ if command -v brew >/dev/null 2>&1; then
+ local brewp=""
+ brewp=$(brew --prefix openjdk@17 2>/dev/null) || brewp="(not installed)"
+ echo " brew --prefix openjdk@17 → $brewp" >&2
+ fi
+ echo "" >&2
+ echo " likely cause:" >&2
+ if [[ -n "${JAVA_HOME:-}" ]] && [[ -n "$path_java" ]]; then
+ echo " JAVA_HOME points at the wrong JDK (probably not 17)." >&2
+ echo " Either fix JAVA_HOME or unset it to let auto-detect try other paths." >&2
+ elif [[ -n "$path_java" ]]; then
+ echo " \`java\` on PATH is not JDK 17. Install a 17 sibling or set JAVA_HOME=/path/to/jdk-17." >&2
+ else
+ echo " No JDK is installed anywhere we know to look." >&2
+ fi
+ echo "" >&2
+}
+
+JAVA_HOME_DETECTED="$(_find_jdk17)" || {
+ echo "FATAL: could not find a JDK 17 install." >&2
+ _diagnose_jdk17
+ echo " fix:" >&2
+ echo " macOS: brew install openjdk@17" >&2
+ echo " Linux: apt install openjdk-17-jdk # or yum/dnf equivalent" >&2
+ echo " SDKMAN: sdk install java 17.0.13-tem" >&2
+ echo " or set JAVA_HOME=/path/to/jdk-17 explicitly" >&2
+ echo "" >&2
+ exit 1
+}
+export JAVA_HOME="$JAVA_HOME_DETECTED"
+export PATH="$JAVA_HOME/bin:$PATH"
+
+# Node: source the user's version manager (if any) so the right `node` is on
+# PATH for yarn/bun/ng. Try nvm, fnm, volta in that order; `command -v node`
+# remains the ultimate fallback.
+if [[ -z "${NVM_DIR:-}" && -d "$HOME/.nvm" ]]; then
+ export NVM_DIR="$HOME/.nvm"
+fi
+if [[ -n "${NVM_DIR:-}" && -s "$NVM_DIR/nvm.sh" ]]; then
+ # shellcheck disable=SC1091
+ \. "$NVM_DIR/nvm.sh" >/dev/null 2>&1 || true
+elif command -v fnm >/dev/null 2>&1; then
+ eval "$(fnm env --use-on-cd 2>/dev/null)" || true
+elif [[ -s "$HOME/.volta/load.sh" ]]; then
+ # shellcheck disable=SC1091
+ \. "$HOME/.volta/load.sh" >/dev/null 2>&1 || true
+fi
+
+# --------- runtime env for backend ---------
+# Detect the host's primary LAN IP so we can use it as the MinIO endpoint.
+# It has to be the same string from both directions:
+# • host-native JVMs need it to reach localhost-published port 9000
+# • the lakekeeper container needs it to do server-side S3 ops (validation,
+# compaction) AND to return URLs to clients that *they* can reach
+# `localhost` only works for the host. `texera-minio` only works inside the
+# docker network. The host's LAN IP works from BOTH (host loopback for the
+# host, docker NAT'd out-and-back for the container).
+_detect_host_lan_ip() {
+ local iface="" ip=""
+ # 1. The interface backing the default route — most reliable on a
+ # laptop that may have wifi + thunderbolt + tailscale all active.
+ iface=$(route get default 2>/dev/null | awk '/interface:/{print $2; exit}')
+ if [[ -n "$iface" ]]; then
+ ip=$(ipconfig getifaddr "$iface" 2>/dev/null)
+ [[ -n "$ip" && "$ip" != 127.* ]] && { printf '%s\n' "$ip"; return 0; }
+ fi
+ # 2. Fallback: linux `hostname -I`-equivalent walk over en*.
+ for iface in en0 en1 en2 en3 en4 en5 en6 en7 en8 en9 en10; do
+ ip=$(ipconfig getifaddr "$iface" 2>/dev/null)
+ [[ -n "$ip" && "$ip" != 127.* ]] && { printf '%s\n' "$ip"; return 0; }
+ done
+ return 1
+}
+# Lazy resolver — called from subcommands that actually need to publish a
+# host-reachable S3 endpoint (cmd_up, cmd_auto). Subcommands like
+# `version`, `status`, `--help`, or `-i` don't talk to MinIO and shouldn't
+# refuse to run just because the laptop is offline.
+_require_host_lan_ip() {
+ [[ -n "${HOST_LAN_IP:-}" ]] && return 0
+ HOST_LAN_IP="$(_detect_host_lan_ip)" || HOST_LAN_IP=""
+ if [[ -z "$HOST_LAN_IP" ]]; then
+ echo "FATAL: could not detect a host LAN IP." >&2
+ echo " MinIO needs an address reachable from both docker (lakekeeper" >&2
+ echo " does S3 ops) and the host (JVMs read signed URLs back); none" >&2
+ echo " of \`route get default\` / en0-en10 had a non-loopback IPv4." >&2
+ echo " Connect to a network or export HOST_LAN_IP=<your-IP> explicitly." >&2
+ exit 1
+ fi
+ export HOST_LAN_IP
+ # STORAGE_S3_ENDPOINT below uses the parameter-default form so it
+ # picks up HOST_LAN_IP once we set it here.
+ export STORAGE_S3_ENDPOINT="${STORAGE_S3_ENDPOINT:-http://$HOST_LAN_IP:9000}"
+}
+
+export STORAGE_JDBC_URL="${STORAGE_JDBC_URL:-jdbc:postgresql://localhost:5432/texera_db?currentSchema=texera_db,public}"
+export STORAGE_JDBC_USERNAME="${STORAGE_JDBC_USERNAME:-texera}"
+export STORAGE_JDBC_PASSWORD="${STORAGE_JDBC_PASSWORD:-password}"
+# STORAGE_S3_ENDPOINT is set lazily by _require_host_lan_ip — only the
+# subcommands that actually touch MinIO (infra_up + cmd_up + cmd_auto)
+# trigger that detection, so `version` / `status` / `-i` work offline.
+export STORAGE_S3_AUTH_USERNAME="${STORAGE_S3_AUTH_USERNAME:-texera_minio}"
+export STORAGE_S3_AUTH_PASSWORD="${STORAGE_S3_AUTH_PASSWORD:-password}"
+export STORAGE_S3_REGION="${STORAGE_S3_REGION:-us-west-2}"
+export STORAGE_ICEBERG_CATALOG_TYPE="${STORAGE_ICEBERG_CATALOG_TYPE:-rest}"
+export STORAGE_ICEBERG_CATALOG_REST_URI="${STORAGE_ICEBERG_CATALOG_REST_URI:-http://localhost:8181/catalog}"
+export STORAGE_ICEBERG_CATALOG_REST_WAREHOUSE_NAME="${STORAGE_ICEBERG_CATALOG_REST_WAREHOUSE_NAME:-texera}"
+export STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET="${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET:-texera-iceberg}"
+export STORAGE_ICEBERG_CATALOG_POSTGRES_USERNAME="${STORAGE_ICEBERG_CATALOG_POSTGRES_USERNAME:-texera}"
+export STORAGE_ICEBERG_CATALOG_POSTGRES_PASSWORD="${STORAGE_ICEBERG_CATALOG_POSTGRES_PASSWORD:-password}"
+export STORAGE_ICEBERG_CATALOG_POSTGRES_URI_WITHOUT_SCHEME="${STORAGE_ICEBERG_CATALOG_POSTGRES_URI_WITHOUT_SCHEME:-localhost:5432/texera_iceberg_catalog}"
+export STORAGE_LAKEFS_ENDPOINT="${STORAGE_LAKEFS_ENDPOINT:-http://localhost:8000/api/v1}"
+export STORAGE_LAKEFS_AUTH_USERNAME="${STORAGE_LAKEFS_AUTH_USERNAME:-AKIAIOSFOLKFSSAMPLES}"
+export STORAGE_LAKEFS_AUTH_PASSWORD="${STORAGE_LAKEFS_AUTH_PASSWORD:-wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY}"
+export STORAGE_LAKEFS_AUTH_API_SECRET="${STORAGE_LAKEFS_AUTH_API_SECRET:-random_string_for_lakefs}"
+export UDF_PYTHON_PATH="${UDF_PYTHON_PATH:-$(command -v python3 2>/dev/null || command -v python 2>/dev/null)}"
+export TEXERA_DASHBOARD_SERVICE_ENDPOINT="${TEXERA_DASHBOARD_SERVICE_ENDPOINT:-http://localhost:8080}"
+export WORKFLOW_COMPILING_SERVICE_ENDPOINT="${WORKFLOW_COMPILING_SERVICE_ENDPOINT:-http://localhost:9090}"
+export WORKFLOW_EXECUTION_SERVICE_ENDPOINT="${WORKFLOW_EXECUTION_SERVICE_ENDPOINT:-http://localhost:8085}"
+export FILE_SERVICE_GET_PRESIGNED_URL_ENDPOINT="${FILE_SERVICE_GET_PRESIGNED_URL_ENDPOINT:-http://localhost:9092/api/dataset/presign-download}"
+export FILE_SERVICE_UPLOAD_ONE_FILE_TO_DATASET_ENDPOINT="${FILE_SERVICE_UPLOAD_ONE_FILE_TO_DATASET_ENDPOINT:-http://localhost:9092/api/dataset/did/upload}"
+export LITELLM_BASE_URL="${LITELLM_BASE_URL:-http://localhost:4000}"
+export LITELLM_MASTER_KEY="${LITELLM_MASTER_KEY:-sk-texera-internal-do-not-share}"
+export LLM_ENDPOINT="${LLM_ENDPOINT:-http://localhost:8080}"
+export LLM_API_KEY="${LLM_API_KEY:-dummy}"
+
+# --------- texera version (dynamic) ---------
+# The sbt-native-packager dist directory and jar names embed the project
+# version (e.g. target/config-service-<VERSION>/...). That version moves
+# across branches (1.3.0-incubating-SNAPSHOT on main, 1.2.0-incubating on
+# release/v1.2, …) so resolve it from build.sbt at startup rather than
+# hardcoding. Override via the TEXERA_VERSION env var to target a sibling
+# tree or if the build.sbt parse fails.
+_texera_version() {
+ grep -E '^[[:space:]]*ThisBuild[[:space:]]*/[[:space:]]*version[[:space:]]*:=[[:space:]]*"' \
+ "$REPO_ROOT/build.sbt" 2>/dev/null \
+ | head -1 \
+ | sed -E 's/.*"([^"]+)".*/\1/'
+}
+TEXERA_VERSION="${TEXERA_VERSION:-$(_texera_version)}"
+if [[ -z "$TEXERA_VERSION" ]]; then
+ # tui_warn isn't defined yet at this point in the script; print raw.
+ printf "FATAL: could not detect texera version from %s/build.sbt\n" "$REPO_ROOT" >&2
+ printf " Set the TEXERA_VERSION env var to bypass.\n" >&2
+ exit 1
+fi
+
+# --------- sbt dependency graph (parsed from build.sbt) ---------
+# We parse the build.sbt's project + .dependsOn() declarations so that the
+# per-service "did anything that affects me change?" check can prune to
+# the actual transitive closure. Without this, every JVM service watches
+# every common/* dir and `cmd_auto` rebuilds half the stack whenever
+# common/workflow-operator changes, even though config-service has zero
+# dependency on it.
+#
+# Populates two amap maps keyed by sbt project name plus a parallel
+# indexed-array of keys so we can iterate without `${!MAP[@]}`:
+# amap_get SBT_PATH ConfigService → "config-service"
+# amap_get SBT_DEPS ConfigService → "Auth Config" (space-separated)
+# SBT_DEPS_KEYS=(ConfigService AccessControlService …)
+SBT_DEPS_KEYS=()
+
+_parse_sbt() {
+ local file="$REPO_ROOT/build.sbt"
+ [[ -f "$file" ]] || return 1
+ local current="" line=""
+ local decl_re='^lazy[[:space:]]+val[[:space:]]+([A-Z][A-Za-z0-9]*)[[:space:]]*=[[:space:]]*\(project[[:space:]]+in[[:space:]]+file\("([^"]+)"\)\)'
+ local deps_re='\.dependsOn\(([^)]*)\)'
+
+ # Helper: split a comma-list of dependsOn args, dropping test-scope
+ # and non-project tokens, then append unique main-scope refs.
+ _absorb() {
+ local args="$1"
+ local IFS_OLD="$IFS"
+ IFS=','
+ # shellcheck disable=SC2206
+ local parts=($args)
+ IFS="$IFS_OLD"
+ local arg="" existing=""
+ for arg in "${parts[@]}"; do
+ arg="${arg// /}"
+ arg="${arg//$'\t'/}"
+ # Drop ONLY test-scope deps. The previous catch-all `*"%"*`
+ # match silently swallowed any future `%`-scoped main dep
+ # (e.g. `X % "compile->compile"`, `X % Provided`) and would
+ # have broken dirty-detection on it.
+ if [[ "$arg" =~ %\"?test(-\>[^\"]*)?\"? ]] || [[ "$arg" =~ %Test$ ]]; then
+ continue
+ fi
+ [[ "$arg" =~ ^[A-Z] ]] || continue
+ existing=$(amap_get SBT_DEPS "$current")
+ case " $existing " in
+ *" $arg "*) ;;
+ *) amap_append SBT_DEPS "$current" " $arg" ;;
+ esac
+ done
+ }
+
+ while IFS= read -r line; do
+ local rest="$line"
+ if [[ "$line" =~ $decl_re ]]; then
+ current="${BASH_REMATCH[1]}"
+ amap_set SBT_PATH "$current" "${BASH_REMATCH[2]}"
+ amap_set SBT_DEPS "$current" ""
+ SBT_DEPS_KEYS+=("$current")
+ # Don't `continue` — `.dependsOn(...)` can chain on the SAME
+ # line as the declaration (WorkflowOperator's one-liner does
+ # this). Fall through so the loop below catches it.
+ elif [[ "$line" =~ ^lazy[[:space:]]+val ]]; then
+ # Other `lazy val` (settings, etc) breaks attribution.
+ current=""
+ continue
+ fi
+ [[ -z "$current" ]] && continue
+ # Scan ALL `.dependsOn(...)` matches on this line — some lines
+ # chain `.dependsOn(A).dependsOn(B % "test->test")` and we must
+ # see both to filter correctly. Snapshot BASH_REMATCH BEFORE
+ # calling `_absorb` — that helper does its own `[[ … =~ … ]]`
+ # for the test-scope filter, which clobbers `BASH_REMATCH[0]`
+ # under us. Without the snapshot the `#*…` trim referenced the
+ # wrong value, the strip silently no-op'd, and the outer while
+ # looped forever on the first `.dependsOn(…)`.
+ local _match="" _captured=""
+ while [[ "$rest" =~ $deps_re ]]; do
+ _match="${BASH_REMATCH[0]}"
+ _captured="${BASH_REMATCH[1]}"
+ _absorb "$_captured"
+ rest="${rest#*"$_match"}"
+ done
+ done < "$file"
+ # Trim leading space on every entry.
+ local p="" cur=""
+ for p in "${SBT_DEPS_KEYS[@]}"; do
+ cur=$(amap_get SBT_DEPS "$p")
+ amap_set SBT_DEPS "$p" "${cur# }"
+ done
+ return 0
+}
+_parse_sbt || true
+
+# BFS from $1 over SBT_DEPS, emit one `<path>/src` per visited project.
+# Caller can also pass an explicit fallback list for the unparseable case.
+_sbt_transitive_src_dirs() {
+ local root="$1"
+ [[ -z "$root" ]] && return 1
+ amap_has SBT_PATH "$root" || return 1
+ # `_visited_` namespace is a per-call amap. We can't easily clear all
+ # entries (bash 3.2 has no `${!_visited_*}` glob over set vars without
+ # `compgen`, but `compgen -v` IS available). We unset matching vars
+ # at function exit so successive calls start clean.
+ local queue=("$root")
+ local p="" path="" d="" key="" var=""
+ while ((${#queue[@]} > 0)); do
+ p="${queue[0]}"
+ queue=("${queue[@]:1}")
+ key="${p//-/_}"
+ var="_amap__visited__${key}"
+ eval "[[ \${$var+x} ]]" && continue
+ eval "$var=1"
+ path=$(amap_get SBT_PATH "$p")
+ [[ -n "$path" ]] && printf '%s/src\n' "$path"
+ local deps=""
+ deps=$(amap_get SBT_DEPS "$p")
+ for d in $deps; do
+ queue+=("$d")
+ done
+ done
+ # Clean up our visited markers so the next caller starts fresh.
+ local v=""
+ for v in $(compgen -v _amap__visited__ 2>/dev/null); do
+ unset "$v"
+ done
+ return 0
+}
+
+# --------- service catalog ---------
+SERVICES=(
+ postgres
+ minio
+ lakefs
+ lakekeeper
+ litellm
+ config-service
+ access-control-service
+ file-service
+ workflow-compiling-service
+ computing-unit-master
+ computing-unit-managing-service
+ texera-web
+ agent-service
+ frontend
+)
+
+# Service catalog. Under bash 3.2 this would normally be `typeset -A` maps;
+# we use the `amap_*` helpers defined above instead. Each amap_set call
+# stores into a synthetic var name (e.g. `_amap__SVC_TYPE__postgres`).
+
+# Each docker service is now its own row in the dashboard. start/stop still
+# batch through infra_up/infra_down because `docker compose up -d` and
+# `docker compose down` operate at the project level.
+amap_set SVC_TYPE postgres docker; amap_set SVC_PORT postgres 5432; amap_set SVC_CWD postgres "."
+amap_set SVC_TYPE minio docker; amap_set SVC_PORT minio 9000; amap_set SVC_CWD minio "."
+amap_set SVC_TYPE lakefs docker; amap_set SVC_PORT lakefs 8000; amap_set SVC_CWD lakefs "."
+amap_set SVC_TYPE lakekeeper docker; amap_set SVC_PORT lakekeeper 8181; amap_set SVC_CWD lakekeeper "."
+amap_set SVC_TYPE litellm docker; amap_set SVC_PORT litellm 4000; amap_set SVC_CWD litellm "."
+
+amap_set SVC_TYPE config-service jvm
+amap_set SVC_PORT config-service 9094
+amap_set SVC_SBT config-service ConfigService
+amap_set SVC_LAUNCHER config-service "target/config-service-${TEXERA_VERSION}/bin/config-service"
+amap_set SVC_CWD config-service "."
+amap_set SVC_ZIP_GLOB config-service "config-service/target/universal/config-service-*.zip"
+amap_set SVC_UNZIP_DEST config-service "target/"
+amap_set SVC_HEALTH config-service "/api/healthcheck"
+
+amap_set SVC_TYPE access-control-service jvm
+amap_set SVC_PORT access-control-service 9096
+amap_set SVC_SBT access-control-service AccessControlService
+amap_set SVC_LAUNCHER access-control-service "target/access-control-service-${TEXERA_VERSION}/bin/access-control-service"
+amap_set SVC_CWD access-control-service "."
+amap_set SVC_ZIP_GLOB access-control-service "access-control-service/target/universal/access-control-service-*.zip"
+amap_set SVC_UNZIP_DEST access-control-service "target/"
+amap_set SVC_HEALTH access-control-service "/api/healthcheck"
+
+amap_set SVC_TYPE file-service jvm
+amap_set SVC_PORT file-service 9092
+amap_set SVC_SBT file-service FileService
+amap_set SVC_LAUNCHER file-service "target/file-service-${TEXERA_VERSION}/bin/file-service"
+amap_set SVC_CWD file-service "."
+amap_set SVC_ZIP_GLOB file-service "file-service/target/universal/file-service-*.zip"
+amap_set SVC_UNZIP_DEST file-service "target/"
+amap_set SVC_HEALTH file-service "/api/healthcheck"
+
+amap_set SVC_TYPE workflow-compiling-service jvm
+amap_set SVC_PORT workflow-compiling-service 9090
+amap_set SVC_SBT workflow-compiling-service WorkflowCompilingService
+amap_set SVC_LAUNCHER workflow-compiling-service "target/workflow-compiling-service-${TEXERA_VERSION}/bin/workflow-compiling-service"
+amap_set SVC_CWD workflow-compiling-service "."
+amap_set SVC_ZIP_GLOB workflow-compiling-service "workflow-compiling-service/target/universal/workflow-compiling-service-*.zip"
+amap_set SVC_UNZIP_DEST workflow-compiling-service "target/"
+amap_set SVC_HEALTH workflow-compiling-service "/api/healthcheck"
+
+amap_set SVC_TYPE computing-unit-managing-service jvm
+amap_set SVC_PORT computing-unit-managing-service 8082
+amap_set SVC_SBT computing-unit-managing-service ComputingUnitManagingService
+amap_set SVC_LAUNCHER computing-unit-managing-service "target/computing-unit-managing-service-${TEXERA_VERSION}/bin/computing-unit-managing-service"
+amap_set SVC_CWD computing-unit-managing-service "."
+amap_set SVC_ZIP_GLOB computing-unit-managing-service "computing-unit-managing-service/target/universal/computing-unit-managing-service-*.zip"
+amap_set SVC_UNZIP_DEST computing-unit-managing-service "target/"
+amap_set SVC_HEALTH computing-unit-managing-service ""
+
+amap_set SVC_TYPE texera-web jvm
+amap_set SVC_PORT texera-web 8080
+amap_set SVC_SBT texera-web WorkflowExecutionService
+amap_set SVC_LAUNCHER texera-web "target/amber-${TEXERA_VERSION}/bin/texera-web-application"
+amap_set SVC_CWD texera-web "amber"
+amap_set SVC_ZIP_GLOB texera-web "amber/target/universal/amber-*.zip"
+amap_set SVC_UNZIP_DEST texera-web "amber/target/"
+amap_set SVC_HEALTH texera-web "/api/healthcheck"
+
+# computing-unit-master shares the amber dist with texera-web: sbt-native-
+# packager emits both `bin/texera-web-application` and `bin/computing-unit-master`
+# launchers under `amber/target/amber-<VERSION>/`. We register it as a separate
+# service for status/start/stop but leave SVC_SBT / SVC_ZIP_GLOB empty so the
+# build pipeline knows to skip it (the texera-web build path already produces
+# its launcher). Source-dir and canary-jar lookups treat it identically to
+# texera-web — see _svc_src_dirs / svc_src_changed / svc_artifact_mtime.
+amap_set SVC_TYPE computing-unit-master jvm
+amap_set SVC_PORT computing-unit-master 8085
+amap_set SVC_SBT computing-unit-master ""
+amap_set SVC_LAUNCHER computing-unit-master "target/amber-${TEXERA_VERSION}/bin/computing-unit-master"
+amap_set SVC_CWD computing-unit-master "amber"
+amap_set SVC_ZIP_GLOB computing-unit-master ""
+amap_set SVC_UNZIP_DEST computing-unit-master ""
+amap_set SVC_HEALTH computing-unit-master ""
+
+amap_set SVC_TYPE agent-service bun
+amap_set SVC_PORT agent-service 3001
+amap_set SVC_CWD agent-service "agent-service"
+amap_set SVC_HEALTH agent-service "/api/healthcheck"
+
+amap_set SVC_TYPE frontend yarn
+amap_set SVC_PORT frontend 4200
+amap_set SVC_CWD frontend "frontend"
+amap_set SVC_HEALTH frontend ""
+
+# --------- docker infra config ---------
+DOCKER_PROJECT="texera-local-dev"
+DOCKER_COMPOSE_FILE="$REPO_ROOT/bin/single-node/docker-compose.yml"
+DOCKER_OVERLAY_FILE="$REPO_ROOT/bin/local-dev/docker-compose.override.yml"
+DOCKER_ENV_FILE="$REPO_ROOT/bin/single-node/.env"
+DOCKER_INFRA_SERVICES=(postgres minio minio-init lakefs lakekeeper-migrate lakekeeper lakekeeper-init litellm)
+DOCKER_INFRA_LONGLIVED=(postgres minio lakefs lakekeeper litellm) # exclude one-shot init jobs
+
+# Build the array of -f flags: base single-node compose + local-dev overlay
+# (the overlay publishes infra ports to the host, which the upstream compose
+# intentionally does not do).
+docker_compose_files() {
+ local args=(-f "$DOCKER_COMPOSE_FILE")
+ [[ -f "$DOCKER_OVERLAY_FILE" ]] && args+=(-f "$DOCKER_OVERLAY_FILE")
+ printf '%s\n' "${args[@]}"
+}
+
+# --------- TUI helpers ---------
+if [[ -t 1 ]] && [[ "${TERM:-}" != "dumb" ]]; then
+ BOLD=$'\e[1m'; DIM=$'\e[2m'
+ RED=$'\e[31m'; GREEN=$'\e[32m'; YELLOW=$'\e[33m'
+ BLUE=$'\e[34m'; MAGENTA=$'\e[35m'; CYAN=$'\e[36m'
+ GRAY=$'\e[90m'; BRIGHT=$'\e[97m'
+ RESET=$'\e[0m'
+else
+ BOLD="" DIM="" RED="" GREEN="" YELLOW="" BLUE="" MAGENTA="" CYAN="" GRAY="" BRIGHT="" RESET=""
+fi
+
+TUI_WIDTH=$(tput cols 2>/dev/null || echo 80)
+[[ -z "$TUI_WIDTH" || "$TUI_WIDTH" -lt 60 ]] && TUI_WIDTH=80
+(( TUI_WIDTH > 100 )) && TUI_WIDTH=100 # cap for readability
+
+# Symbols
+SYM_RUN="●"; SYM_STOP="○"; SYM_WARN="⚠"; SYM_OK="✓"; SYM_ERR="✗"
+SYM_SECT="▸"; SYM_LIST="•"; SYM_PROG="→"
+
+tui_hline() {
+ local ch="${1:-─}" w="${2:-$TUI_WIDTH}"
+ printf "${ch}%.0s" $(seq 1 "$w")
+}
+
+tui_trunc() {
+ local s="$1" n="$2"
+ if (( ${#s} > n )); then
+ printf "%s…" "${s:0:$((n-1))}"
+ else
+ printf "%s" "$s"
+ fi
+}
+
+tui_banner() {
+ local title="$1" subtitle="${2:-}"
+ local w=$TUI_WIDTH
+ # Row layout: │ + 2 spaces + content + 2 spaces + │ = w → inner = w - 6
+ local inner=$((w - 6))
+ title="$(tui_trunc "$title" "$inner")"
+ subtitle="$(tui_trunc "$subtitle" "$inner")"
+ printf "${BLUE}╭"; tui_hline "─" $((w-2)); printf "╮${RESET}\n"
+ printf "${BLUE}│${RESET} ${BOLD}${BRIGHT}%-*s${RESET} ${BLUE}│${RESET}\n" "$inner" "$title"
+ if [[ -n "$subtitle" ]]; then
+ printf "${BLUE}│${RESET} ${DIM}%-*s${RESET} ${BLUE}│${RESET}\n" "$inner" "$subtitle"
+ fi
+ printf "${BLUE}╰"; tui_hline "─" $((w-2)); printf "╯${RESET}\n"
+}
+
+tui_section() {
+ printf "\n${BOLD}${MAGENTA}${SYM_SECT}${RESET} ${BOLD}%s${RESET}\n" "$1"
+}
+
+tui_ok() { printf " ${GREEN}${SYM_OK}${RESET} %s\n" "$*"; }
+tui_err() { printf " ${RED}${SYM_ERR}${RESET} %s\n" "$*"; }
+tui_warn() { printf " ${YELLOW}${SYM_WARN}${RESET} %s\n" "$*"; }
+tui_info() { printf " ${CYAN}${SYM_LIST}${RESET} %s\n" "$*"; }
+tui_step() { printf " ${DIM}${SYM_PROG}${RESET} ${DIM}%s${RESET}\n" "$*"; }
+tui_skip() { printf " ${GRAY}${SYM_STOP}${RESET} ${GRAY}%s${RESET}\n" "$*"; }
+
+tui_state_symbol() {
+ case "$1" in
+ running) printf "${GREEN}${SYM_RUN}${RESET}" ;;
+ starting) printf "${YELLOW}${SYM_WARN}${RESET}" ;;
+ unhealthy|failed) printf "${RED}${SYM_ERR}${RESET}" ;;
+ partial:*|external:*) printf "${YELLOW}${SYM_WARN}${RESET}" ;;
+ exited) printf "${GRAY}${SYM_OK}${RESET}" ;;
+ *) printf "${GRAY}${SYM_STOP}${RESET}" ;;
+ esac
+}
+
+tui_state_color() {
+ case "$1" in
+ running) echo "$GREEN" ;;
+ starting) echo "$YELLOW" ;;
+ unhealthy|failed) echo "$RED" ;;
+ partial:*|external:*) echo "$YELLOW" ;;
+ exited) echo "$GRAY" ;;
+ *) echo "$GRAY" ;;
+ esac
+}
+
+# Show a spinner next to $msg while $pid runs. Caller is responsible for
+# `wait $pid` afterwards to capture exit code.
+tui_spinner() {
+ local pid="$1" msg="$2"
+ if [[ ! -t 1 ]]; then
+ printf " ${BLUE}${SYM_PROG}${RESET} ${DIM}%s (no-TTY, no spinner)${RESET}\n" "$msg"
+ return
+ fi
+ # Use an array (vs a single multibyte string + byte indexing) because
+ # bash's `${str:i:1}` is byte-wise — each braille glyph is 3 UTF-8
+ # bytes, so a byte index would print broken half-chars.
+ local frames=(⠋ ⠙ ⠹ ⠸ ⠼ ⠴ ⠦ ⠧ ⠇ ⠏)
+ local n=${#frames[@]}
+ local i=0
+ local start_ts=$SECONDS
+ printf "\e[?25l" # hide cursor
+ while kill -0 "$pid" 2>/dev/null; do
+ local elapsed=$((SECONDS - start_ts))
+ local frame="${frames[i % n]}"
+ printf "\r ${BLUE}%s${RESET} ${DIM}%s${RESET} ${GRAY}(%ds)${RESET} " \
+ "$frame" "$msg" "$elapsed"
+ sleep 0.1
+ i=$((i+1))
+ done
+ printf "\r%-${TUI_WIDTH}s\r" " "
+ printf "\e[?25h" # show cursor
+}
+
+# Run a command in the background with output captured to $log, show spinner,
+# return command's exit code.
+tui_run_with_spinner() {
+ local log="$1" msg="$2"
+ shift 2
+ "$@" >"$log" 2>&1 &
+ local pid=$!
+ tui_spinner "$pid" "$msg"
+ wait "$pid"
+}
+
+# In-place panel that polls each non-skipped service's port and redraws the
+# whole panel until all are healthy or timed out. Sets a trap so Ctrl-C
+# restores the cursor.
+tui_wait_panel() {
+ local svcs=()
+ for svc in "${SERVICES[@]}"; do
+ is_skipped "$svc" && continue
+ svcs+=("$svc")
+ done
+ local n=${#svcs[@]}
+ (( n == 0 )) && return 0
+
+ local frames=(⠋ ⠙ ⠹ ⠸ ⠼ ⠴ ⠦ ⠧ ⠇ ⠏)
+ local n_frames=${#frames[@]}
+ local start_ts=$SECONDS
+ # JVM + docker services bind their port within ~10-30s; 90s leaves slack
+ # for cold-cache machines. The frontend's `ng serve` does a full Angular
+ # compile before listening — on a fresh checkout that's 90-180s easily,
+ # so give yarn-typed services 5 minutes. bun is in between.
+ local timeout_default=90
+ local timeout_yarn=300
+ local timeout_bun=120
+ local frame_idx=0
+ local first_render=true
+ local n_done=0
+ local n_failed=0
+ local svc="" i=0 state="" state_color="" state_sym="" port_str="" elapsed=0 spinner_frame=""
+
+ if [[ ! -t 1 ]]; then
+ # Non-TTY: redrawing would just spam lines. Poll each service to
+ # completion (or per-service timeout) and print one line per
+ # transition. Crucially we WAIT for docker containers to leave
+ # `starting`/`unhealthy` rather than treating an in-flight state as
+ # a failure — the previous one-shot check exited rc=1 within ~20s
+ # of `up` if e.g. lakefs hadn't reached `running` yet, which is its
+ # normal cold-start state.
+ local n_done=0 n_failed=0
+ local svc_timeout=0 waited=0 final_state=""
+ for svc in "${svcs[@]}"; do
+ svc_timeout=$timeout_default
+ case "$(amap_get SVC_TYPE "$svc")" in
+ yarn) svc_timeout=$timeout_yarn ;;
+ bun) svc_timeout=$timeout_bun ;;
+ esac
+ waited=0
+ final_state=""
+ while (( waited < svc_timeout )); do
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ case "$(docker_svc_state "$svc")" in
+ running|exited) final_state="ok"; break ;;
+ unhealthy|failed) final_state="bad"; break ;;
+ # starting / created / restarting / paused / "" → keep waiting
+ esac
+ else
+ [[ -n "$(listen_pid_for_port "$(amap_get SVC_PORT "$svc")")" ]] && { final_state="ok"; break; }
+ fi
+ sleep 1
+ waited=$((waited+1))
+ done
+ case "$final_state" in
+ ok)
+ printf " %s %-32s :%-6s %s\n" "$SYM_OK" "$svc" "$(amap_get SVC_PORT "$svc")" "healthy"
+ n_done=$((n_done+1)) ;;
+ bad)
+ printf " %s %-32s :%-6s %s\n" "$SYM_ERR" "$svc" "$(amap_get SVC_PORT "$svc")" "unhealthy"
+ n_failed=$((n_failed+1)) ;;
+ *) # timed out without ever reaching a terminal state
+ printf " %s %-32s :%-6s %s\n" "$SYM_ERR" "$svc" "$(amap_get SVC_PORT "$svc")" "timeout (${waited}s)"
+ n_failed=$((n_failed+1)) ;;
+ esac
+ done
+ return $((n_failed > 0))
+ fi
+
+ printf "\e[?25l" # hide cursor
+ trap 'printf "\e[?25h"' EXIT INT TERM
+
+ while true; do
+ elapsed=$((SECONDS - start_ts))
+ n_done=0
+ n_failed=0
+
+ if ! $first_render && [[ -t 1 ]]; then
+ printf "\e[${n}A" # move cursor to top of panel
+ fi
+ first_render=false
+
+ spinner_frame="${frames[frame_idx % n_frames]}"
+
+ for svc in "${svcs[@]}"; do
+ state_color="" state_sym=""
+ # Per-service wait budget — yarn (ng serve cold compile) gets the
+ # most slack, bun a moderate amount, everything else the default.
+ local svc_timeout=$timeout_default
+ case "$(amap_get SVC_TYPE "$svc")" in
+ yarn) svc_timeout=$timeout_yarn ;;
+ bun) svc_timeout=$timeout_bun ;;
+ esac
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ local dstate=""
+ dstate=$(docker_svc_state "$svc")
+ case "$dstate" in
+ running|exited)
+ state="$dstate"
+ state_color="$GREEN"; state_sym="${GREEN}${SYM_OK}${RESET}"
+ n_done=$((n_done+1)) ;;
+ starting)
+ state="starting (${elapsed}s)"
+ state_color="$YELLOW"; state_sym="${YELLOW}${spinner_frame}${RESET}" ;;
+ unhealthy|failed)
+ state="$dstate"
+ state_color="$RED"; state_sym="${RED}${SYM_ERR}${RESET}"
+ n_failed=$((n_failed+1)) ;;
+ *)
+ if (( elapsed >= svc_timeout )); then
+ state="timeout"
+ state_color="$RED"; state_sym="${RED}${SYM_ERR}${RESET}"
+ n_failed=$((n_failed+1))
+ else
+ state="${dstate} (${elapsed}s)"
+ state_color="$YELLOW"; state_sym="${YELLOW}${spinner_frame}${RESET}"
+ fi ;;
+ esac
+ port_str=":$(amap_get SVC_PORT "$svc")"
+ else
+ if [[ -n "$(listen_pid_for_port "$(amap_get SVC_PORT "$svc")")" ]]; then
+ state="healthy"
+ state_color="$GREEN"; state_sym="${GREEN}${SYM_OK}${RESET}"
+ n_done=$((n_done+1))
+ elif (( elapsed >= svc_timeout )); then
+ state="timeout — see bin/local-dev.sh logs $svc"
+ state_color="$RED"; state_sym="${RED}${SYM_ERR}${RESET}"
+ n_failed=$((n_failed+1))
+ else
+ state="starting (${elapsed}s)"
+ state_color="$YELLOW"; state_sym="${YELLOW}${spinner_frame}${RESET}"
+ fi
+ port_str=":$(amap_get SVC_PORT "$svc")"
+ fi
+
+ [[ -t 1 ]] && printf "\e[2K"
+ printf " %s %-32s ${DIM}%-7s${RESET} ${state_color}%s${RESET}\n" \
+ "$state_sym" "$svc" "$port_str" "$state"
+ done
+
+ if (( n_done + n_failed == n )); then
+ break
+ fi
+
+ frame_idx=$((frame_idx + 1))
+ sleep 0.2
+ done
+
+ if [[ -t 1 ]]; then
+ printf "\e[?25h"
+ trap - EXIT INT TERM
+ fi
+
+ return $((n_failed > 0))
+}
+
+# --------- toolchain install hints ---------
+# Print install instructions for a missing toolchain. Used by both startup
+# detection failures and runtime "command not found" surfaces. Keeps the
+# guidance in one place so every failure mode looks the same.
+_install_hint() {
+ local tool="$1"
+ case "$tool" in
+ java)
+ printf " ${BOLD}install JDK 17:${RESET}\n"
+ printf " macOS: brew install openjdk@17\n"
+ printf " Linux: apt install openjdk-17-jdk ${DIM}# or yum/dnf equivalent${RESET}\n"
+ printf " SDKMAN: sdk install java 17.0.13-tem\n"
+ printf " or set JAVA_HOME=/path/to/jdk-17 explicitly\n"
+ ;;
+ python)
+ printf " ${BOLD}install Python 3.10+ and the TUI deps:${RESET}\n"
+ printf " macOS: brew install python@3.12\n"
+ printf " Linux: apt install python3 python3-pip\n"
+ printf " then: python3 -m pip install -r %s/amber/dev-requirements.txt\n" "$REPO_ROOT"
+ printf " or set TEXERA_PYTHON=/path/to/python explicitly (must have textual installed)\n"
+ ;;
+ node)
+ printf " ${BOLD}install Node 20+ (needed for frontend & agent-service):${RESET}\n"
+ printf " macOS: brew install node\n"
+ printf " nvm: nvm install --lts && nvm use --lts\n"
+ printf " fnm: fnm install --lts\n"
+ printf " volta: volta install node\n"
+ ;;
+ yarn)
+ printf " ${BOLD}install yarn (needed for the frontend):${RESET}\n"
+ printf " macOS: brew install yarn\n"
+ printf " npm: npm install -g yarn\n"
+ printf " corepack: corepack enable\n"
+ ;;
+ bun)
+ printf " ${BOLD}install bun (needed for agent-service):${RESET}\n"
+ printf " macOS: brew install oven-sh/bun/bun\n"
+ printf " curl: curl -fsSL https://bun.sh/install | bash\n"
+ ;;
+ sbt)
+ printf " ${BOLD}install sbt (needed to build the JVM services):${RESET}\n"
+ printf " macOS: brew install sbt\n"
+ printf " Linux: see https://www.scala-sbt.org/download.html\n"
+ ;;
+ docker)
+ printf " ${BOLD}install Docker (needed for postgres/minio/lakefs/lakekeeper/litellm):${RESET}\n"
+ printf " macOS: download Docker Desktop from https://docker.com/products/docker-desktop\n"
+ printf " Linux: apt install docker.io docker-compose-plugin\n"
+ ;;
+ *)
+ printf " ${DIM}no install hint for: %s${RESET}\n" "$tool"
+ ;;
+ esac
+}
+
+# Inspect the user's Node/yarn/bun setup and print what we actually find,
+# so "yarn not found" reads as "you have node 18 but no yarn" instead of
+# the generic install hint. Called from start_one's frontend/agent paths.
+_diagnose_node() {
+ printf "\n ${BOLD}node environment:${RESET}\n"
+ for tool in node npm yarn bun corepack; do
+ local p=""
+ p=$(command -v "$tool" 2>/dev/null) || true
+ if [[ -n "$p" ]]; then
+ local v=""
+ v=$("$tool" --version 2>&1 | head -1) || v="(no --version)"
+ printf " ${GREEN}✓${RESET} %-8s → %s ${DIM}(%s)${RESET}\n" "$tool" "$p" "$v"
+ else
+ printf " ${RED}✗${RESET} %-8s ${DIM}not on PATH${RESET}\n" "$tool"
+ fi
+ done
+ printf "\n ${BOLD}version manager:${RESET}\n"
+ if [[ -n "${NVM_DIR:-}" && -s "${NVM_DIR}/nvm.sh" ]]; then
+ printf " nvm → ${DIM}%s${RESET}\n" "$NVM_DIR"
+ elif command -v fnm >/dev/null 2>&1; then
+ printf " fnm → ${DIM}%s${RESET}\n" "$(command -v fnm)"
+ elif [[ -s "$HOME/.volta/load.sh" ]]; then
+ printf " volta → ${DIM}%s${RESET}\n" "$HOME/.volta"
+ else
+ printf " ${DIM}none detected (using whatever \`node\` is on PATH)${RESET}\n"
+ fi
+ printf "\n"
+}
+
+# --------- helpers ---------
+listen_pid_for_port() {
+ # || true so pipefail doesn't kill us when nothing is listening
+ lsof -nP -iTCP:"$1" -sTCP:LISTEN -t 2>/dev/null | head -1 || true
+}
+
+# Returns the count of long-lived infra services currently running under our project.
+infra_running_count() {
+ docker compose -p "$DOCKER_PROJECT" ps --services --filter status=running 2>/dev/null | grep -cxE "$(IFS=\|; echo "${DOCKER_INFRA_LONGLIVED[*]}")" || true
+}
+
+# Returns one of: stopped | running | partial:N | external — aggregate view
+# across all infra containers, used by infra_up/infra_down and the wait panel.
+infra_state() {
+ local running=""
+ running=$(infra_running_count)
+ if [[ "$running" -eq ${#DOCKER_INFRA_LONGLIVED[@]} ]]; then
+ echo "running"
+ elif [[ "$running" -gt 0 ]]; then
+ echo "partial:$running"
+ else
+ local taken=0
+ for port in 5432 9000 8000 8181 4000; do
+ [[ -n "$(listen_pid_for_port "$port")" ]] && taken=$((taken+1))
+ done
+ if [[ "$taken" -gt 0 ]]; then
+ echo "external:$taken/5"
+ else
+ echo "stopped"
+ fi
+ fi
+}
+
+# Cached snapshot of `docker compose ps -a`. Refreshed at most once per second
+# (5 docker services × per-row ~200 ms `docker ps` call would otherwise tank
+# the render).
+_docker_states_cache=""
+_docker_states_cache_ts=-1
+
+_refresh_docker_states_cache() {
+ if (( _docker_states_cache_ts < 0 || SECONDS - _docker_states_cache_ts >= 1 )); then
+ _docker_states_cache=$(docker compose -p "$DOCKER_PROJECT" ps -a \
+ --format '{{.Service}}|{{.State}}|{{.Status}}' 2>/dev/null)
+ _docker_states_cache_ts=$SECONDS
+ fi
+}
+
+# Per-service state for any of postgres/minio/lakefs/lakekeeper/litellm.
+# Returns one of: running | starting | unhealthy | exited | failed | stopped
+docker_svc_state() {
+ local svc="$1"
+ _refresh_docker_states_cache
+ local line=""
+ line=$(printf '%s\n' "$_docker_states_cache" | grep "^${svc}|" | head -1 || true)
+ if [[ -z "$line" ]]; then
+ echo "stopped"
+ return
+ fi
+ local rest="${line#*|}"
+ local dstate="${rest%%|*}" # NB: not `status` — zsh reserves $status
+ local dstatus="${rest#*|}"
+ case "$dstate" in
+ running)
+ if [[ "$dstatus" == *'(healthy)'* ]]; then echo "running"
+ elif [[ "$dstatus" == *'(health: starting)'* ]]; then echo "starting"
+ elif [[ "$dstatus" == *'(unhealthy)'* ]]; then echo "unhealthy"
+ else echo "running"
+ fi ;;
+ exited)
+ if [[ "$dstatus" == 'Exited (0)'* ]]; then echo "exited"
+ else echo "failed"
+ fi ;;
+ created|restarting|paused|removing) echo "starting" ;;
+ *) echo "stopped" ;;
+ esac
+}
+
+infra_up() {
+ # Resolve the host LAN IP now (lazy) — both the docker compose stack
+ # (lakekeeper-init reads STORAGE_S3_ENDPOINT) and the host JVMs about
+ # to start need it pointing at a host-reachable MinIO.
+ _require_host_lan_ip
+ if [[ "$(infra_state)" == external:* ]]; then
+ tui_err "infra: ports already taken by non-script containers"
+ printf " ${DIM}Likely an old project (e.g. \`texera-dev\`) is running. Stop it first:${RESET}\n"
+ printf " ${DIM} docker compose -p texera-dev down${RESET}\n"
+ return 1
+ fi
+ local files=($(docker_compose_files))
+ tui_step "infra: docker compose up -d ${DIM}(in-place TTY progress)${RESET}"
+ # No stdout redirect → docker compose detects TTY and renders an in-place
+ # progress panel that overwrites itself instead of appending event lines.
+ # --progress=tty forces it even if stdout looks like a pipe.
+ docker compose --progress auto -p "$DOCKER_PROJECT" --env-file "$DOCKER_ENV_FILE" "${files[@]}" \
+ up -d "${DOCKER_INFRA_SERVICES[@]}"
+ tui_ok "infra: 5 containers up"
+}
+
+infra_down() {
+ tui_step "infra: docker compose -p $DOCKER_PROJECT down ${DIM}(in-place TTY progress)${RESET}"
+ docker compose --progress auto -p "$DOCKER_PROJECT" down || true
+ tui_ok "infra: stopped"
+}
+
+# Ensure the texera_db schema exists in the postgres container. The compose
+# file mounts sql/*.sql to /docker-entrypoint-initdb.d, but Postgres only
+# runs those on first init (empty data dir). If the volume was carried over
+# from an older texera version (e.g. before the `feedback` table was added)
+# the schema will be missing relations that current code references, the
+# jOOQ codegen produces an incomplete Tables.java, and sbt compile fails on
+# `not found: value FEEDBACK`. Probe for a canonical table and re-run
+# texera_ddl.sql if it's absent.
+infra_ensure_db_schema() {
+ local pg="texera-postgres"
+ # Wait briefly for postgres to be ready — `up -d` returned but the
+ # container may still be running its own init sequence.
+ local i=0
+ while (( i < 30 )); do
+ if docker exec "$pg" pg_isready -U texera -d texera_db -q 2>/dev/null; then
+ break
+ fi
+ sleep 1
+ i=$((i+1))
+ done
+ if (( i >= 30 )); then
+ tui_warn "postgres: not ready after 30s -- skipping schema check"
+ return 0
+ fi
+ # `feedback` is one of the newer tables; if it exists we assume the
+ # whole schema is current. (texera_ddl.sql is idempotent with
+ # CREATE TABLE IF NOT EXISTS, so re-applying it is safe even if some
+ # tables already exist, but skipping the copy + exec is faster.)
+ local has_feedback=""
+ has_feedback=$(docker exec "$pg" psql -U texera -d texera_db -tAc \
+ "SELECT 1 FROM pg_tables WHERE schemaname='texera_db' AND tablename='feedback'" \
+ 2>/dev/null || true)
+ if [[ "$has_feedback" == "1" ]]; then
+ tui_skip "postgres: schema already current"
+ return 0
+ fi
+ tui_step "postgres: applying sql/texera_ddl.sql (one-time bootstrap)"
+ local ddl="$REPO_ROOT/sql/texera_ddl.sql"
+ if [[ ! -f "$ddl" ]]; then
+ tui_warn "postgres: $ddl not found -- skipping (jOOQ codegen may fail)"
+ return 0
+ fi
+ docker cp "$ddl" "$pg":/tmp/texera_ddl.sql >/dev/null
+ if docker exec -u postgres "$pg" psql -U texera -f /tmp/texera_ddl.sql >/dev/null 2>&1; then
+ tui_ok "postgres: schema bootstrapped"
+ else
+ tui_warn "postgres: ddl exec returned non-zero (check container logs)"
+ fi
+}
+
+svc_running_pid() {
+ listen_pid_for_port "$(amap_get SVC_PORT "$1")"
+}
+
+# Compact-format a duration in seconds:
+# "—" for <0, "12s", "5m 23s", "2h 14m", "3d 4h"
+_format_uptime() {
+ local s="${1:-0}"
+ if (( s < 0 )); then printf "—"; return; fi
+ if (( s < 60 )); then printf "%ds" "$s"; return; fi
+ if (( s < 3600 )); then printf "%dm %ds" $((s/60)) $((s%60)); return; fi
+ if (( s < 86400 )); then printf "%dh %dm" $((s/3600)) $(((s%3600)/60)); return; fi
+ printf "%dd %dh" $((s/86400)) $(((s%86400)/3600))
+}
+
+# Translate ps -o etime output (`[[DD-]hh:]mm:ss`) to whole seconds.
+_etime_to_seconds() {
+ local et="$1"
+ # bash quirk: in `local a=$x b=$a`, the right-hand `$a` resolves
+ # against the *outer* scope (before this `local` declares either).
+ # Putting `rest` on its own line ensures it sees the fresh `et`.
+ local rest="$et" days=0 h=0 m=0 s=0
+ if [[ "$rest" == *-* ]]; then
+ days="${rest%%-*}"
+ rest="${rest#*-}"
+ fi
+ # `IFS=: read -ra` reliably splits on `:` without the local-IFS-vs-
+ # array-literal trap.
+ local parts=()
+ IFS=: read -ra parts <<< "$rest"
+ case "${#parts[@]}" in
+ 3) h=${parts[0]}; m=${parts[1]}; s=${parts[2]} ;;
+ 2) m=${parts[0]}; s=${parts[1]} ;;
+ *) s=${parts[0]} ;;
+ esac
+ # Force base-10 so "08" / "09" don't blow up under bash's octal default.
+ printf '%d' "$((10#${days:-0}*86400 + 10#${h:-0}*3600 + 10#${m:-0}*60 + 10#${s:-0}))"
+}
+
+# Translate a number of bytes to a 4-char-ish human string ("85M", "1.2G").
+_format_bytes() {
+ local b="${1:-0}"
+ if (( b < 1024 )); then printf "%dB" "$b"; return; fi
+ if (( b < 1048576 )); then printf "%dK" $((b/1024)); return; fi
+ if (( b < 1073741824 )); then printf "%dM" $((b/1048576)); return; fi
+ # GB → 1 decimal place
+ local g_int=$((b / 1073741824)) g_frac=$(( (b % 1073741824) * 10 / 1073741824 ))
+ printf "%d.%dG" "$g_int" "$g_frac"
+}
+
+# Compact "8.2%" formatter from a string like "0.50" or "8.2".
+_format_pct() {
+ local p="${1:-0}"
+ # Strip optional trailing %, normalise weird inputs
+ p="${p%%%}"
+ [[ -z "$p" || "$p" == "-" ]] && { printf "—"; return; }
+ # Round to 1 decimal place using awk to avoid bash's lack of float math
+ printf '%s%%' "$(awk -v v="$p" 'BEGIN { printf "%.1f", v }' 2>/dev/null)"
+}
+
+# Container name for a docker-typed service. Most use `texera-<svc>`, but
+# `litellm` ships unprefixed in upstream's compose.
+_docker_container_for() {
+ case "$1" in
+ litellm) printf "litellm" ;;
+ *) printf "texera-%s" "$1" ;;
+ esac
+}
+
+# Uptime for a service. Native: `ps -o etime` (POSIX, works on mac+linux).
+# Docker: parse the container's ISO-8601 .State.StartedAt against `date -u`.
+# Echoes "—" when not running.
+svc_uptime() {
+ local svc="$1"
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ local container="" since=""
+ container=$(_docker_container_for "$svc")
+ since=$(docker inspect -f '{{.State.StartedAt}}' "$container" 2>/dev/null) || { printf "—"; return; }
+ [[ -z "$since" || "$since" == "0001-01-01T00:00:00Z" ]] && { printf "—"; return; }
+ local now=0 started=0
+ now=$(date -u +%s 2>/dev/null) || { printf "—"; return; }
+ # macOS `date -j` parses ISO without fractional seconds — trim them.
+ started=$(date -u -j -f "%Y-%m-%dT%H:%M:%S" "${since%%.*}" +%s 2>/dev/null) || {
+ # GNU date fallback
+ started=$(date -u -d "${since%%.*}" +%s 2>/dev/null) || { printf "—"; return; }
+ }
+ local elapsed=$((now - started))
+ (( elapsed < 0 )) && elapsed=0
+ _format_uptime "$elapsed"
+ return
+ fi
+ local pid=""
+ pid=$(svc_running_pid "$svc") || true
+ [[ -z "$pid" ]] && { printf "—"; return; }
+ local et=""
+ et=$(ps -p "$pid" -o etime= 2>/dev/null | tr -d ' ')
+ [[ -z "$et" ]] && { printf "—"; return; }
+ _format_uptime "$(_etime_to_seconds "$et")"
+}
+
+# CPU% + RSS for a running native process. Returns "cpu%|rss_bytes".
+# Cheap enough (<5ms) that we can call it per service per tick.
+svc_proc_cpu_mem() {
+ local pid=""
+ pid=$(svc_running_pid "$1") || true
+ [[ -z "$pid" ]] && { printf "—|—"; return; }
+ local cpu="" rss=""
+ # macOS ps emits RSS in KB. Linux's `ps -o rss=` is also KB.
+ read -r cpu rss <<< "$(ps -p "$pid" -o pcpu=,rss= 2>/dev/null | tr -s ' ')"
+ [[ -z "$cpu" || -z "$rss" ]] && { printf "—|—"; return; }
+ printf "%s|%s" "$(_format_pct "$cpu")" "$(_format_bytes $((rss * 1024)))"
+}
+
+# Bulk docker stats — one slow `docker stats --no-stream` call returns CPU
+# + memory for every running container. Callers can grep their service out
+# of the result instead of paying the 1-2 second cost N times. Format:
+# <container>|<cpu%>|<used-bytes>
+_docker_stats_snapshot() {
+ docker stats --no-stream --format '{{.Name}}|{{.CPUPerc}}|{{.MemUsage}}' 2>/dev/null \
+ | while IFS='|' read -r name cpu memu; do
+ # MemUsage is like "85.2MiB / 1.95GiB" — keep only the first
+ # half and translate IEC units to bytes.
+ local used="${memu%% *}" # "85.2MiB"
+ local val="${used%[A-Za-z]*}"
+ local unit="${used##*[0-9.]}"
+ local bytes=0
+ case "$unit" in
+ KB|KiB) bytes=$(awk -v v="$val" 'BEGIN { printf "%d", v * 1024 }') ;;
+ MB|MiB) bytes=$(awk -v v="$val" 'BEGIN { printf "%d", v * 1048576 }') ;;
+ GB|GiB) bytes=$(awk -v v="$val" 'BEGIN { printf "%d", v * 1073741824 }') ;;
+ *) bytes=$(awk -v v="$val" 'BEGIN { printf "%d", v }') ;;
+ esac
+ printf "%s|%s|%s\n" "$name" "$(_format_pct "${cpu%%%}")" "$(_format_bytes "$bytes")"
+ done
+}
+
+svc_artifact_mtime() {
+ local svc="$1" type=""
+ type=$(amap_get SVC_TYPE "$svc")
+ case "$type" in
+ jvm)
+ local launcher="$(amap_get SVC_CWD "$svc")/$(amap_get SVC_LAUNCHER "$svc")"
+ launcher="${launcher#./}"
+ local jar_dir=""
+ jar_dir="$(dirname "$(dirname "$launcher")")/lib"
+ if [[ -d "$jar_dir" ]]; then
+ # `shopt -s nullglob` localised in a subshell so an
+ # unmatched glob yields no words instead of the literal
+ # pattern. Capture into a string then split into an array.
+ local globbed=""
+ globbed=$(shopt -s nullglob; \
+ printf '%s\n' "$jar_dir"/org.apache.texera."${svc}"-*.jar)
+ local main_jars=()
+ while IFS= read -r _f; do
+ [[ -n "$_f" ]] && main_jars+=("$_f")
+ done <<< "$globbed"
+ if [[ ${#main_jars[@]} -eq 0 && ( "$svc" == "texera-web" || "$svc" == "computing-unit-master" ) ]]; then
+ globbed=$(shopt -s nullglob; \
+ printf '%s\n' "$jar_dir"/org.apache.texera.amber-*.jar)
+ while IFS= read -r _f; do
+ [[ -n "$_f" ]] && main_jars+=("$_f")
+ done <<< "$globbed"
+ fi
+ if [[ ${#main_jars[@]} -gt 0 ]]; then
+ stat -f "%Sm" -t "%Y-%m-%d %H:%M" "${main_jars[0]}"
+ return
+ fi
+ fi
+ echo "—"
+ ;;
+ bun) stat -f "%Sm" -t "%Y-%m-%d %H:%M" "$(amap_get SVC_CWD "$svc")/bun.lock" 2>/dev/null || echo "—" ;;
+ yarn) stat -f "%Sm" -t "%Y-%m-%d %H:%M" "$(amap_get SVC_CWD "$svc")/yarn.lock" 2>/dev/null || echo "—" ;;
+ docker) echo "—" ;;
+ esac
+}
+
+is_skipped() {
+ [[ ",${SKIP_LIST:-}," == *",$1,"* ]]
+}
+
+wait_for_port() {
+ local port="$1" timeout="${2:-90}" i=0
+ while (( i < timeout )); do
+ [[ -n "$(listen_pid_for_port "$port")" ]] && return 0
+ sleep 1
+ i=$((i+1))
+ done
+ return 1
+}
+
+# Write a transitional phase for $svc so the TUI dashboard can render it
+# in the STATE column with animated dots. Cleared by phase_clear or after
+# 90s (TUI side stale check).
+phase_set() {
+ local svc="$1" phase="$2"
+ printf '%s\t%s\n' "$phase" "$(date +%s)" > "$PHASE_DIR/$svc" 2>/dev/null || true
+}
+phase_clear() {
+ local svc="$1"
+ rm -f "$PHASE_DIR/$svc" 2>/dev/null || true
+}
+
+stop_one() {
+ local svc="$1"
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ phase_set "$svc" stopping
+ tui_step "$svc: docker compose stop $svc"
+ docker compose -p "$DOCKER_PROJECT" stop "$svc" >/dev/null 2>&1 || true
+ phase_clear "$svc"
+ tui_ok "$svc: stopped"
+ return
+ fi
+ local pid=""
+ pid=$(svc_running_pid "$svc")
+ if [[ -z "$pid" ]]; then
+ tui_skip "$svc: already stopped"
+ return 0
+ fi
+ phase_set "$svc" stopping
+ tui_step "$svc: stopping PID $pid"
+ kill "$pid" 2>/dev/null || true
+ local i=0
+ while (( i < 30 )) && kill -0 "$pid" 2>/dev/null; do
+ sleep 0.5
+ i=$((i+1))
+ done
+ if kill -0 "$pid" 2>/dev/null; then
+ tui_warn "$svc: SIGKILL $pid"
+ kill -9 "$pid" 2>/dev/null || true
+ fi
+ phase_clear "$svc"
+ tui_ok "$svc: stopped"
+}
+
+start_one() {
+ local svc="$1"
+ local type=""
+ type=$(amap_get SVC_TYPE "$svc")
+ # JVMs and docker services need STORAGE_S3_ENDPOINT exported before
+ # launch (JVM clients dial MinIO; lakekeeper bakes the URL into the
+ # warehouse storage profile). yarn/bun watch services don't.
+ if [[ "$type" == "jvm" || "$type" == "docker" ]]; then
+ _require_host_lan_ip
+ fi
+ if [[ "$type" == "docker" ]]; then
+ local dstate=""
+ dstate=$(docker_svc_state "$svc")
+ if [[ "$dstate" == "running" ]]; then
+ tui_ok "$svc: already running"
+ return 0
+ fi
+ phase_set "$svc" starting
+ tui_step "$svc: docker compose up -d $svc"
+ local files=()
+ while read -r f; do files+=("$f"); done < <(docker_compose_files | tr ' ' '\n')
+ docker compose --progress auto -p "$DOCKER_PROJECT" --env-file "$DOCKER_ENV_FILE" \
+ "${files[@]}" up -d "$svc" >/dev/null 2>&1
+ # Don't clear the phase yet — docker `up -d` returns before the
+ # container is healthy. The TUI's docker_state poller will flip
+ # the row to "running" once the container reports healthy; the
+ # stale-after-90s rule covers cleanup if that never happens.
+ tui_ok "$svc: started"
+ return
+ fi
+ if [[ -n "$(svc_running_pid "$svc")" ]]; then
+ tui_ok "$svc: already running ${DIM}(PID $(svc_running_pid "$svc"))${RESET}"
+ return 0
+ fi
+ local cwd="" log="$LOG_DIR/$svc.log"
+ cwd=$(amap_get SVC_CWD "$svc")
+ phase_set "$svc" starting
+ tui_step "$svc: starting ${DIM}(log: $log)${RESET}"
+ case "$type" in
+ jvm)
+ local launcher=""
+ launcher=$(amap_get SVC_LAUNCHER "$svc")
+ if [[ ! -x "$cwd/$launcher" ]]; then
+ phase_clear "$svc"
+ tui_err "$svc: launcher missing at $cwd/$launcher -- run \`bin/local-dev.sh up\` to build first"
+ return 1
+ fi
+ ( cd "$cwd" && nohup "./$launcher" >"$log" 2>&1 </dev/null & )
+ ;;
+ bun)
+ if ! command -v bun >/dev/null 2>&1; then
+ phase_clear "$svc"
+ tui_err "$svc: \`bun\` not found on PATH"
+ _diagnose_node
+ _install_hint bun
+ return 1
+ fi
+ ( cd "$cwd" && nohup bun run dev >"$log" 2>&1 </dev/null & )
+ ;;
+ yarn)
+ if ! command -v yarn >/dev/null 2>&1; then
+ phase_clear "$svc"
+ tui_err "$svc: \`yarn\` not found on PATH"
+ _diagnose_node
+ if ! command -v node >/dev/null 2>&1; then
+ _install_hint node
+ else
+ _install_hint yarn
+ fi
+ return 1
+ fi
+ ( cd "$cwd" && nohup yarn start >"$log" 2>&1 </dev/null & )
+ ;;
+ esac
+}
+
+build_one_jvm() {
+ local svc="$1" proj=""
+ proj=$(amap_get SVC_SBT "$svc")
+ local log="$LOG_DIR/sbt-${svc}.log"
+ # Empty SVC_SBT means this service rides another service's dist (e.g.
+ # computing-unit-master shares amber's). Nothing to build directly — the
+ # launcher is produced when its sibling builds. Stamp `svc` so the dirty
+ # indicator can clear if amber/src actually matches.
+ if [[ -z "$proj" ]]; then
+ tui_skip "$svc: no own sbt project (built with its sibling)"
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc" 2>/dev/null || true
+ return 0
+ fi
+ phase_set "$svc" building
+ if tui_run_with_spinner "$log" "sbt $proj/dist ${DIM}(log: $log)${RESET}" \
+ sbt -no-colors "$proj/dist"; then
+ local zip_glob="" unzip_dest=""
+ zip_glob=$(amap_get SVC_ZIP_GLOB "$svc")
+ unzip_dest=$(amap_get SVC_UNZIP_DEST "$svc")
+ tui_step "unzip ${zip_glob} → ${unzip_dest}"
+ # shellcheck disable=SC2086
+ unzip -oq ${zip_glob} -d "${unzip_dest}"
+ # Stamp = SHA-1 of the source we just built from. Clears the `*` and
+ # lets us tell content-vs-mtime apart on the next dirty check.
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc"
+ tui_ok "$svc: build done"
+ # Don't clear the phase yet — the caller (cmd_update_one) will
+ # transition us through stop_one/start_one which overwrite it.
+ # If something else is the caller, the TUI's "phase cleared once
+ # poller sees running" rule covers us.
+ else
+ phase_clear "$svc"
+ tui_err "$svc: sbt $proj/dist FAILED ${DIM}(tail -f $log)${RESET}"
+ return 1
+ fi
+}
+
+# True if ANY JVM service's source changed (content-hash) since its last build.
+# This is what `up`/`build` use to decide whether to skip the sbt step. It
+# is the same check the dashboard's SRC `*` indicator uses, just OR'd across
+# all JVM services.
+any_jvm_src_changed() {
+ local svc=""
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "jvm" ]] || continue
+ if svc_src_changed "$svc" 2>/dev/null; then
+ return 0
+ fi
+ done
+ return 1
+}
+
+needs_yarn_install() {
+ [[ ! -f frontend/node_modules/.yarn-state.yml ]] && return 0
+ [[ frontend/yarn.lock -nt frontend/node_modules/.yarn-state.yml ]] && return 0
+ [[ frontend/package.json -nt frontend/node_modules/.yarn-state.yml ]] && return 0
+ return 1
+}
+
+needs_bun_install() {
+ [[ ! -d agent-service/node_modules ]] && return 0
+ [[ agent-service/bun.lock -nt agent-service/node_modules ]] && return 0
+ [[ agent-service/package.json -nt agent-service/node_modules ]] && return 0
+ return 1
+}
+
+# Cache of per-service transitive src dirs, populated once at startup by
+# _precompute_src_dirs(). Each entry (in the SVC_SRC_DIRS amap) is a
+# newline-separated list — callers consume via `while read` (same shape
+# as the old _svc_src_dirs output).
+
+_precompute_src_dirs() {
+ local svc="" entry="" dirs=""
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "jvm" ]] || continue
+ entry=$(amap_get SVC_SBT "$svc")
+ # computing-unit-master rides amber's dist (no SVC_SBT of its
+ # own); enter the graph at the producing project explicitly.
+ if [[ -z "$entry" && "$svc" == "computing-unit-master" ]]; then
+ entry="WorkflowExecutionService"
+ fi
+ if [[ -z "$entry" ]]; then
+ echo "FATAL: service '$svc' has no SVC_SBT entry and no" >&2
+ echo " SHARED_SBT_PROJECT mapping — can't compute its" >&2
+ echo " transitive source-dir closure." >&2
+ exit 1
+ fi
+ if ! amap_has SBT_PATH "$entry"; then
+ echo "FATAL: sbt project '$entry' (for service '$svc') is not" >&2
+ echo " in the parsed build.sbt graph. Either build.sbt" >&2
+ echo " parsing failed or the project was renamed/removed." >&2
+ exit 1
+ fi
+ dirs="$(_sbt_transitive_src_dirs "$entry")"
+ if [[ -z "$dirs" ]]; then
+ echo "FATAL: transitive source-dir closure for '$svc' is empty." >&2
+ exit 1
+ fi
+ amap_set SVC_SRC_DIRS "$svc" "$dirs"
+ done
+}
+
+# Look up the pre-populated cache.
+_svc_src_dirs() {
+ local svc="$1"
+ local cached=""
+ cached=$(amap_get SVC_SRC_DIRS "$svc")
+ if [[ -z "$cached" ]]; then
+ echo "FATAL: _svc_src_dirs: '$svc' missing from SVC_SRC_DIRS" >&2
+ echo " (did _precompute_src_dirs run?)" >&2
+ exit 1
+ fi
+ printf '%s\n' "$cached"
+}
+
+# Compute a SHA-1 over the content of every .scala/.java/.proto file that
+# matters for this service. ~100 ms; called only on the slow path of
+# svc_src_changed and from the post-build stamp write.
+svc_source_hash() {
+ local svc="$1"
+ local dirs=()
+ local d=""
+ while IFS= read -r d; do
+ [[ -n "$d" ]] && dirs+=("$d")
+ done < <(_svc_src_dirs "$svc")
+ find "${dirs[@]}" \
+ \( -name "*.scala" -o -name "*.java" -o -name "*.proto" \) \
+ -type f -print0 2>/dev/null \
+ | sort -z \
+ | xargs -0 cat 2>/dev/null \
+ | shasum -a 1 \
+ | awk '{print $1}'
+}
+
+# Per-service dirty check (the SRC * indicator). Two-stage:
+# Fast path (~22 ms): is any tracked source newer than the stamp file's
+# mtime? If not, definitely clean.
+# Slow path (~100 ms): compute current source hash and compare to the hash
+# we stored at last build time. If they match, the
+# mtime moved without content moving (typical for git
+# checkout) — refresh the stamp mtime so we skip the
+# slow path next tick. If they differ, dirty.
+svc_src_changed() {
+ local svc="$1"
+ case "$(amap_get SVC_TYPE "$svc")" in
+ jvm)
+ local stamp="$BUILD_STAMP_DIR/$svc"
+ # Lazy seed: if we have a jar but no stamp, assume the jar matches
+ # current sources and seed with the hash. First REPL after a fresh
+ # checkout pays this once (~100 ms) and is clean afterwards.
+ if [[ ! -s "$stamp" ]]; then
+ local jar=""
+ if [[ "$svc" == "texera-web" || "$svc" == "computing-unit-master" ]]; then
+ jar="amber/target/amber-${TEXERA_VERSION}/lib/org.apache.texera.amber-${TEXERA_VERSION}.jar"
+ else
+ jar="target/${svc}-${TEXERA_VERSION}/lib/org.apache.texera.${svc}-${TEXERA_VERSION}.jar"
+ fi
+ [[ ! -f "$jar" ]] && return 0 # no jar, definitely dirty
+ svc_source_hash "$svc" > "$stamp"
+ return 1
+ fi
+
+ # Fast path: any tracked source mtime newer than stamp's mtime?
+ local dirs=() d=""
+ while IFS= read -r d; do
+ [[ -n "$d" ]] && dirs+=("$d")
+ done < <(_svc_src_dirs "$svc")
+ local newer=""
+ newer=$(find "${dirs[@]}" \
+ \( -name "*.scala" -o -name "*.java" -o -name "*.proto" \) \
+ -newer "$stamp" -type f -print 2>/dev/null | head -1)
+ if [[ -z "$newer" ]]; then
+ return 1 # nothing changed since last stamp → clean
+ fi
+
+ # Slow path: did the content actually change, or just mtimes?
+ local current_hash="" stored_hash=""
+ current_hash=$(svc_source_hash "$svc")
+ stored_hash=$(cat "$stamp" 2>/dev/null)
+ if [[ "$current_hash" == "$stored_hash" ]]; then
+ # Same content, just newer mtimes (git checkout, touch, etc.).
+ # Refresh stamp mtime to skip the slow path next tick.
+ touch "$stamp"
+ return 1
+ fi
+ return 0 # content really changed → dirty
+ ;;
+ yarn) needs_yarn_install ;;
+ bun) needs_bun_install ;;
+ docker) return 1 ;;
+ esac
+}
+
+build_all() {
+ BUILD_DID_RUN=false
+ local log="$LOG_DIR/sbt-dist.log"
+ if [[ "${FRESH:-false}" == "true" ]]; then
+ if tui_run_with_spinner "$log" "sbt clean dist ${DIM}(log: $log)${RESET}" \
+ sbt -no-colors clean dist; then
+ tui_ok "sbt: clean dist done"
+ BUILD_DID_RUN=true
+ else
+ tui_err "sbt: clean dist FAILED ${DIM}(tail -f $log)${RESET}"
+ return 1
+ fi
+ elif [[ "${BUILD:-auto}" == "auto" ]] && ! any_jvm_src_changed; then
+ tui_skip "sbt dist: skipped (no source changes since last build)"
+ return 0
+ else
+ if tui_run_with_spinner "$log" "sbt dist ${DIM}(log: $log)${RESET}" \
+ sbt -no-colors dist; then
+ tui_ok "sbt: dist done"
+ BUILD_DID_RUN=true
+ else
+ tui_err "sbt: dist FAILED ${DIM}(tail -f $log)${RESET}"
+ return 1
+ fi
+ fi
+ # Stop any running JVMs BEFORE unzip — overwriting jars under a live JVM
+ # corrupts its lazy class loads and the service silently dies later.
+ if [[ "$BUILD_DID_RUN" == "true" ]]; then
+ local svc="" pid=""
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "jvm" ]] || continue
+ pid=$(svc_running_pid "$svc")
+ [[ -z "$pid" ]] && continue
+ tui_step "$svc: pre-bouncing PID $pid (jars about to change)"
+ kill "$pid" 2>/dev/null || true
+ done
+ # Wait briefly for them to actually exit
+ local waited=0
+ while (( waited < 10 )); do
+ local still_up=0
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "jvm" ]] || continue
+ [[ -n "$(svc_running_pid "$svc")" ]] && still_up=$((still_up+1))
+ done
+ (( still_up == 0 )) && break
+ sleep 0.5
+ waited=$((waited+1))
+ done
+ fi
+ tui_step "unzipping dist artifacts"
+ local zip_glob="" unzip_dest=""
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "jvm" ]] || continue
+ zip_glob=$(amap_get SVC_ZIP_GLOB "$svc")
+ unzip_dest=$(amap_get SVC_UNZIP_DEST "$svc")
+ # Sibling services (empty ZIP_GLOB) share another service's dist —
+ # just stamp them as clean since the unzip already happened for the
+ # twin holding the build.
+ if [[ -z "$zip_glob" ]]; then
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc" 2>/dev/null || true
+ continue
+ fi
+ # shellcheck disable=SC2086
+ if unzip -oq ${zip_glob} -d "${unzip_dest}" 2>/dev/null; then
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc"
+ else
+ tui_warn "${zip_glob} not produced"
+ fi
+ done
+ tui_ok "artifacts unzipped"
+}
+
+refresh_node_deps() {
+ if ! is_skipped frontend; then
+ if [[ "${BUILD:-auto}" == "auto" ]] && ! needs_yarn_install; then
+ tui_skip "yarn install: skipped (lock unchanged)"
+ else
+ local log="$LOG_DIR/yarn-install.log"
+ if tui_run_with_spinner "$log" "yarn install (frontend) ${DIM}(log: $log)${RESET}" \
+ bash -c "cd frontend && yarn install"; then
+ tui_ok "yarn: deps refreshed"
+ else
+ tui_warn "yarn install failed ${DIM}(tail -f $log)${RESET}"
+ fi
+ fi
+ fi
+ if ! is_skipped agent-service; then
+ if [[ "${BUILD:-auto}" == "auto" ]] && ! needs_bun_install; then
+ tui_skip "bun install: skipped (lock unchanged)"
+ else
+ local log="$LOG_DIR/bun-install.log"
+ if tui_run_with_spinner "$log" "bun install (agent-service) ${DIM}(log: $log)${RESET}" \
+ bash -c "cd agent-service && bun install"; then
+ tui_ok "bun: deps refreshed"
+ else
+ tui_warn "bun install failed ${DIM}(tail -f $log)${RESET}"
+ fi
+ fi
+ fi
+}
+
+# --------- subcommands ---------
+cmd_status() {
+ local branch="" sha=""
+ branch=$(git -C "$REPO_ROOT" rev-parse --abbrev-ref HEAD 2>/dev/null || echo "?")
+ sha=$(git -C "$REPO_ROOT" rev-parse --short HEAD 2>/dev/null || echo "?")
+ tui_banner "Texera Local Dev" "branch: $branch @ $sha"
+
+ # One docker stats call up front — paying the ~2s docker-API cost once
+ # is cheaper than running it per docker service. Indexed by container
+ # name → "cpu%|mem" so the per-row formatting is just a lookup.
+ # `_DSTATS` is an amap (bash 3.2: no associative arrays).
+ while IFS='|' read -r name cpu mem; do
+ [[ -n "$name" ]] && amap_set _DSTATS "$name" "$cpu|$mem"
+ done < <(_docker_stats_snapshot)
+
+ printf "\n"
+ printf " ${BOLD}%-32s %-6s %-9s %-10s %-7s %-7s %s${RESET}\n" \
+ "SERVICE" "PORT" "PID" "UPTIME" "CPU%" "MEM" "STATE"
+ printf " ${GRAY}"
+ tui_hline "─" 32; printf " "
+ tui_hline "─" 6; printf " "
+ tui_hline "─" 9; printf " "
+ tui_hline "─" 10; printf " "
+ tui_hline "─" 7; printf " "
+ tui_hline "─" 7; printf " "
+ tui_hline "─" 12; printf "${RESET}\n"
+
+ local n_running=0 n_total=0 frontend_up=false
+ for svc in "${SERVICES[@]}"; do
+ n_total=$((n_total+1))
+ local pid="—" state="stopped" uptime="—" cpu="—" mem="—"
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ state=$(docker_svc_state "$svc")
+ if [[ "$state" == "running" ]]; then
+ local container="" dstat=""
+ container=$(_docker_container_for "$svc")
+ dstat=$(amap_get _DSTATS "$container")
+ if [[ -n "$dstat" ]]; then
+ cpu="${dstat%|*}"
+ mem="${dstat#*|}"
+ fi
+ uptime=$(svc_uptime "$svc")
+ fi
+ else
+ local found_pid=""
+ found_pid=$(svc_running_pid "$svc")
+ if [[ -n "$found_pid" ]]; then
+ state="running"; pid="$found_pid"
+ uptime=$(svc_uptime "$svc")
+ local cm=""
+ cm=$(svc_proc_cpu_mem "$svc")
+ cpu="${cm%|*}"
+ mem="${cm#*|}"
+ fi
+ fi
+ [[ "$state" == "running" ]] && n_running=$((n_running+1))
+ [[ "$svc" == "frontend" && "$state" == "running" ]] && frontend_up=true
+
+ local sym="" color=""
+ sym=$(tui_state_symbol "$state")
+ color=$(tui_state_color "$state")
+ printf " %s " "$sym"
+ printf "${color}%-32s${RESET} %-6s ${DIM}%-9s${RESET} %-10s %-7s %-7s ${color}%s${RESET}\n" \
+ "$svc" "$(amap_get SVC_PORT "$svc")" "$pid" "$uptime" "$cpu" "$mem" "$state"
+ done
+ printf "\n"
+
+ local summary_color="$YELLOW"
+ (( n_running == n_total )) && summary_color="$GREEN"
+ (( n_running == 0 )) && summary_color="$GRAY"
+ printf " ${BOLD}Status${RESET}: ${summary_color}%d of %d services running${RESET}\n" "$n_running" "$n_total"
+
+ printf "\n"
+ printf " ${CYAN}${SYM_LIST}${RESET} Logs: ${DIM}%s${RESET}\n" "$LOG_DIR/<service>.log"
+ printf " ${CYAN}${SYM_LIST}${RESET} Docker: ${DIM}docker compose -p %s ps${RESET}\n" "$DOCKER_PROJECT"
+ if $frontend_up; then
+ printf " ${CYAN}${SYM_LIST}${RESET} Open: ${BOLD}${GREEN}http://localhost:4200${RESET} ${GREEN}(frontend live)${RESET}\n"
+ else
+ printf " ${CYAN}${SYM_LIST}${RESET} Open: ${DIM}http://localhost:4200 (frontend not running yet)${RESET}\n"
+ fi
+
+ printf "\n ${BOLD}Common operations${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh up${RESET} ${DIM}# bring up the whole stack (build + start)${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh down${RESET} ${DIM}# stop every service${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh auto${RESET} ${DIM}# rebuild + bounce only the services whose source changed${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh <svc>${RESET} ${DIM}# rebuild that one JVM service and bounce it${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh start <svc>${RESET} ${DIM}# start one service without rebuilding${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh stop <svc>${RESET} ${DIM}# stop one service${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh logs <svc>${RESET} ${DIM}# tail a service's log${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh -i${RESET} ${DIM}# open the live TUI (needs Python + textual)${RESET}\n"
+ printf " ${BOLD}bin/local-dev.sh --help${RESET} ${DIM}# full reference${RESET}\n"
+ printf "\n"
+}
+
+cmd_up() {
+ SKIP_LIST=""
+ FRESH=false
+ BUILD=auto # auto (skip if no source change) | force | no
+ while [[ $# -gt 0 ]]; do
+ case "$1" in
+ --skip=*) SKIP_LIST="${1#--skip=}" ;;
+ --fresh) FRESH=true; BUILD=force ;;
+ --build) BUILD=force ;;
+ --no-build) BUILD=no ;;
+ *) tui_err "unknown flag: $1" >&2; exit 2 ;;
+ esac
+ shift
+ done
+
+ local n_skip=0
+ [[ -n "$SKIP_LIST" ]] && n_skip=$(echo "$SKIP_LIST" | tr ',' '\n' | wc -l | tr -d ' ')
+ local skip_label="none"
+ (( n_skip > 0 )) && skip_label="$n_skip service(s)"
+ tui_banner "Texera Local Dev — bringing stack up" "JDK 17 · skip=$skip_label · build=$BUILD"
+
+ # ── Pre-flight short-circuit ───────────────────────────────────────────
+ # If nothing's changed AND every service is already running, just say so
+ # and exit. Saves the user from scrolling through 30+ "already running"
+ # lines for the common "I just want to check" case.
+ if [[ "$BUILD" == "auto" && -z "$SKIP_LIST" ]]; then
+ local nothing_to_build=true
+ any_jvm_src_changed && nothing_to_build=false
+ needs_yarn_install && nothing_to_build=false
+ needs_bun_install && nothing_to_build=false
+
+ local all_running=true svc=""
+ for svc in "${SERVICES[@]}"; do
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ [[ "$(docker_svc_state "$svc")" == "running" ]] || { all_running=false; break; }
+ else
+ [[ -n "$(svc_running_pid "$svc")" ]] || { all_running=false; break; }
+ fi
+ done
+
+ if $nothing_to_build && $all_running; then
+ tui_section "Pre-flight"
+ tui_ok "no source/lock changes since last build"
+ tui_ok "all ${#SERVICES[@]} services already running"
+ printf "\n ${BOLD}${GREEN}${SYM_OK} nothing to do${RESET} ${DIM}(use \`u --build\` to force a rebuild, or \`<svc>\` to bounce just one)${RESET}\n\n"
+ return 0
+ fi
+ fi
+
+ if [[ "$BUILD" != "no" ]]; then
+ tui_section "Build"
+ build_all
+ refresh_node_deps
+ else
+ tui_section "Build"
+ tui_skip "build: --no-build (using existing artifacts)"
+ fi
+
+ # ▸ Services -- docker compose has its own TTY panel; native services we
+ # kick off silently in the background, then a single redrawing panel below
+ # shows progress for ALL of them.
+ tui_section "Services ${DIM}(launching)${RESET}"
+ local svc="" cwd="" log="" type="" launcher=""
+
+ # One project-level `docker compose up -d` for every non-skipped docker
+ # row — much faster than five separate calls.
+ local has_docker_targets=false
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]] || continue
+ is_skipped "$svc" && continue
+ has_docker_targets=true
+ break
+ done
+ if $has_docker_targets; then
+ case "$(infra_state)" in
+ running) tui_ok "infra: already running" ;;
+ external:*) tui_err "infra: ports taken by non-script containers"
+ printf " ${DIM}docker compose -p texera-dev down${RESET}\n" ;;
+ *) infra_up || true ;;
+ esac
+ # Whether infra is fresh or already up, make sure the texera_db
+ # schema is current — JVMs about to start expect it (jOOQ + Iceberg
+ # both need real tables).
+ infra_ensure_db_schema
+ fi
+
+ for svc in "${SERVICES[@]}"; do
+ is_skipped "$svc" && { tui_skip "$svc: --skip"; continue; }
+ type=$(amap_get SVC_TYPE "$svc")
+ [[ "$type" == "docker" ]] && continue # already handled by infra_up
+ if [[ -n "$(svc_running_pid "$svc")" ]]; then
+ tui_ok "$svc: already running ${DIM}(PID $(svc_running_pid "$svc"))${RESET}"
+ continue
+ fi
+ cwd=$(amap_get SVC_CWD "$svc")
+ log="$LOG_DIR/$svc.log"
+ tui_step "$svc: launching → ${DIM}$log${RESET}"
+ case "$type" in
+ jvm)
+ launcher=$(amap_get SVC_LAUNCHER "$svc")
+ if [[ ! -x "$cwd/$launcher" ]]; then
+ tui_err "$svc: launcher missing at $cwd/$launcher"
+ continue
+ fi
+ ( cd "$cwd" && nohup "./$launcher" >"$log" 2>&1 </dev/null & ) ;;
+ bun) ( cd "$cwd" && nohup bun run dev >"$log" 2>&1 </dev/null & ) ;;
+ yarn) ( cd "$cwd" && nohup yarn start >"$log" 2>&1 </dev/null & ) ;;
+ esac
+ done
+
+ tui_section "Health ${DIM}(refreshing in place)${RESET}"
+ local ec=0
+ tui_wait_panel || ec=$?
+
+ printf "\n"
+ local ok=0 total=0 failed=0
+ for svc in "${SERVICES[@]}"; do
+ is_skipped "$svc" && continue
+ total=$((total+1))
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ case "$(docker_svc_state "$svc")" in
+ running|exited) ok=$((ok+1)) ;;
+ *) failed=$((failed+1)) ;;
+ esac
+ else
+ [[ -n "$(svc_running_pid "$svc")" ]] && ok=$((ok+1)) || failed=$((failed+1))
+ fi
+ done
+ if (( failed == 0 )); then
+ printf " ${BOLD}${GREEN}${SYM_OK} %d of %d services healthy${RESET}\n" "$ok" "$total"
+ else
+ printf " ${BOLD}${YELLOW}${SYM_WARN} %d of %d services healthy${RESET} ${RED}(%d failed)${RESET}\n" \
+ "$ok" "$total" "$failed"
+ fi
+ printf "\n"
+
+ cmd_status
+ [[ $ec -eq 0 ]]
+}
+
+# `auto`: the minimal "make the running services match my current source" path.
+# Walks every service, identifies what's actually dirty (content-hash for JVM,
+# lock mtime for yarn/bun, never for docker), and only touches those:
+# - dirty JVM, currently running → rebuild + bounce
+# - dirty JVM, currently stopped → rebuild only (don't auto-start)
+# - dirty yarn (frontend lock) → yarn install + bounce frontend if up
+# - dirty bun (agent-service lock) → bun install (bun --watch reloads itself)
+# Clean services are left alone — no pre-bounce, no restart.
+cmd_auto() {
+ SKIP_LIST=""
+ while [[ $# -gt 0 ]]; do
+ case "$1" in
+ --skip=*) SKIP_LIST="${1#--skip=}" ;;
+ *) tui_err "unknown flag: $1" >&2; exit 2 ;;
+ esac
+ shift
+ done
+
+ tui_banner "Texera Local Dev — auto bounce" \
+ "rebuild + bounce only what changed since last build"
+
+ # ── Scan ──────────────────────────────────────────────────────────────
+ tui_section "Scan"
+ local svc=""
+ local dirty_jvms=()
+ local need_yarn=false
+ local need_bun=false
+ for svc in "${SERVICES[@]}"; do
+ is_skipped "$svc" && continue
+ case "$(amap_get SVC_TYPE "$svc")" in
+ jvm)
+ if svc_src_changed "$svc"; then
+ dirty_jvms+=("$svc")
+ tui_warn "$svc: source changed since last build"
+ fi ;;
+ yarn)
+ if needs_yarn_install; then
+ need_yarn=true
+ tui_warn "frontend: yarn.lock newer than node_modules — needs install"
+ fi ;;
+ bun)
+ if needs_bun_install; then
+ need_bun=true
+ tui_warn "agent-service: bun.lock newer than node_modules — needs install"
+ fi ;;
+ esac
+ done
+
+ if (( ${#dirty_jvms[@]} == 0 )) && ! $need_yarn && ! $need_bun; then
+ tui_ok "everything up-to-date — nothing to bounce"
+ printf "\n"
+ return 0
+ fi
+
+ # ── Build ─────────────────────────────────────────────────────────────
+ # One `sbt dist` covers every dirty JVM in a single sbt invocation; sbt's
+ # own incremental compiler only recompiles the subprojects that need it,
+ # and we only unzip + bounce the dirty ones below — clean services don't
+ # get pre-bounced just because the build ran.
+ if (( ${#dirty_jvms[@]} > 0 )); then
+ tui_section "Build ${DIM}(${#dirty_jvms[@]} JVM service(s) dirty)${RESET}"
+ # Mark each dirty service as "building" so the TUI shows the
+ # animation across the whole sbt window (~30s+). Without this the
+ # dashboard stays on the prior STATE during the slow build.
+ local _s=""
+ for _s in "${dirty_jvms[@]}"; do
+ phase_set "$_s" building
+ done
+ local log="$LOG_DIR/sbt-dist.log"
+ if ! tui_run_with_spinner "$log" "sbt dist ${DIM}(log: $log)${RESET}" \
+ sbt -no-colors dist; then
+ for _s in "${dirty_jvms[@]}"; do
+ phase_clear "$_s"
+ done
+ tui_err "sbt dist failed ${DIM}(tail -f $log)${RESET}"
+ return 1
+ fi
+ tui_ok "sbt: dist done"
+ fi
+
+ # ── Bounce dirty JVMs ────────────────────────────────────────────────
+ # Two passes so siblings can coexist:
+ # 1) stop + (maybe-)unzip + stamp every dirty service
+ # 2) start the ones that were running when we entered
+ #
+ # Pass 1 needs to finish before pass 2 because computing-unit-master
+ # shares amber's dist with texera-web: cu-master has empty SVC_ZIP_GLOB
+ # / SVC_UNZIP_DEST, so its "rebuild" is just waiting for texera-web's
+ # iteration to unzip amber. If we kept the previous single-pass loop
+ # we'd `continue` past start_one whenever ZIP_GLOB was empty and the
+ # sibling never came back up.
+ local n_bounced=0 n_rebuilt=0
+ # `_was_running` is an amap (bash 3.2: no associative arrays). Keyed by
+ # svc → the PID we observed at the start of pass 1, "" if not running.
+ # Cleared with `unset` at the top so a re-invocation (e.g. interactive
+ # REPL would have, if we still had one) starts fresh.
+ local _wr_var=""
+ for _wr_var in $(compgen -v _amap___was_running__ 2>/dev/null); do
+ unset "$_wr_var"
+ done
+ if (( ${#dirty_jvms[@]} > 0 )); then
+ tui_section "Bounce"
+ # Pass 1: stop running pids; unzip own dist if we have one.
+ for svc in "${dirty_jvms[@]}"; do
+ local pid=""
+ pid=$(svc_running_pid "$svc")
+ amap_set _was_running "$svc" "$pid"
+ if [[ -n "$pid" ]]; then
+ # Flip the dashboard from `building` to `stopping` while we
+ # SIGTERM/SIGKILL the JVM, then back to `building` for the
+ # unzip step below. Without these the user only sees one
+ # state during the whole bounce.
+ phase_set "$svc" stopping
+ tui_step "$svc: stopping PID $pid before unzip"
+ kill "$pid" 2>/dev/null || true
+ local i=0
+ while (( i < 30 )) && kill -0 "$pid" 2>/dev/null; do
+ sleep 0.5
+ i=$((i+1))
+ done
+ kill -0 "$pid" 2>/dev/null && kill -9 "$pid" 2>/dev/null
+ fi
+ local zip_glob="" unzip_dest=""
+ zip_glob=$(amap_get SVC_ZIP_GLOB "$svc")
+ unzip_dest=$(amap_get SVC_UNZIP_DEST "$svc")
+ if [[ -z "$zip_glob" ]]; then
+ # Sibling service (e.g. computing-unit-master): no own dist
+ # to unzip — its launcher comes from the twin's unzip later
+ # in this pass. Just stamp it clean.
+ phase_set "$svc" building
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc" 2>/dev/null || true
+ n_rebuilt=$((n_rebuilt+1))
+ continue
+ fi
+ phase_set "$svc" building
+ # shellcheck disable=SC2086
+ if unzip -oq ${zip_glob} -d "${unzip_dest}" 2>/dev/null; then
+ svc_source_hash "$svc" > "$BUILD_STAMP_DIR/$svc"
+ n_rebuilt=$((n_rebuilt+1))
+ else
+ tui_warn "$svc: ${zip_glob} not produced — skipping"
+ fi
+ done
+ # amber's two siblings (texera-web + computing-unit-master) are
+ # *intended* to run together — they share the dist and they're
+ # both required for a working stack. If either was running before
+ # the auto pass, treat the whole group as "should be running" so
+ # the other doesn't end up silently dead just because it had
+ # already crashed / been left stopped from an earlier session.
+ local AMBER_SIBLINGS=(texera-web computing-unit-master)
+ local amber_group_active=false
+ local s=""
+ for s in "${AMBER_SIBLINGS[@]}"; do
+ [[ -n "$(amap_get _was_running "$s")" ]] && amber_group_active=true
+ done
+
+ # Pass 2: start the ones that had been running, plus any amber
+ # sibling that is dirty when the group is "active". By now every
+ # sibling's launcher is on disk (the twin's unzip in pass 1
+ # populated `amber/target/amber-<VERSION>/bin/*`).
+ for svc in "${dirty_jvms[@]}"; do
+ local should_start=false
+ if [[ -n "$(amap_get _was_running "$svc")" ]]; then
+ should_start=true
+ elif $amber_group_active && { [[ "$svc" == "texera-web" ]] || [[ "$svc" == "computing-unit-master" ]]; }; then
+ should_start=true
+ tui_step "$svc: was stopped but its sibling is active — starting too"
+ fi
+ if $should_start; then
+ start_one "$svc"
+ n_bounced=$((n_bounced+1))
+ else
+ # Pass 1 left phase=building set so the dashboard animates
+ # during sbt + unzip; clear it now since we won't be
+ # starting (otherwise the row spins building… until the
+ # 90s stale rule kicks in).
+ phase_clear "$svc"
+ tui_skip "$svc: was stopped — rebuilt but not started"
+ fi
+ done
+ fi
+
+ # ── Node deps ────────────────────────────────────────────────────────
+ if $need_yarn; then
+ tui_section "Frontend deps"
+ local log="$LOG_DIR/yarn-install.log"
+ if tui_run_with_spinner "$log" "yarn install ${DIM}(log: $log)${RESET}" \
+ bash -c "cd frontend && yarn install"; then
+ tui_ok "yarn: deps refreshed"
+ # ng serve doesn't pick up dependency-tree changes from a running
+ # process; bounce if it was up.
+ if [[ -n "$(svc_running_pid frontend)" ]]; then
+ stop_one frontend
+ start_one frontend
+ n_bounced=$((n_bounced+1))
+ else
+ tui_skip "frontend: was stopped — deps refreshed but not started"
+ fi
+ else
+ tui_err "yarn install failed ${DIM}(tail -f $log)${RESET}"
+ fi
+ fi
+
+ if $need_bun; then
+ tui_section "Agent-service deps"
+ local log="$LOG_DIR/bun-install.log"
+ if tui_run_with_spinner "$log" "bun install ${DIM}(log: $log)${RESET}" \
+ bash -c "cd agent-service && bun install"; then
+ tui_ok "bun: deps refreshed"
+ # bun --watch reloads itself when node_modules changes; no manual
+ # bounce needed.
+ if [[ -n "$(svc_running_pid agent-service)" ]]; then
+ tui_skip "agent-service: bun --watch will reload"
+ else
+ tui_skip "agent-service: was stopped — deps refreshed but not started"
+ fi
+ else
+ tui_err "bun install failed ${DIM}(tail -f $log)${RESET}"
+ fi
+ fi
+
+ # ── Summary + final dashboard ────────────────────────────────────────
+ printf "\n"
+ printf " ${BOLD}${GREEN}${SYM_OK} auto bounce done${RESET}: %d rebuilt, %d bounced\n\n" \
+ "$n_rebuilt" "$n_bounced"
+ cmd_status
+}
+
+cmd_down() {
+ SKIP_LIST=""
+ while [[ $# -gt 0 ]]; do
+ case "$1" in
+ --skip=*) SKIP_LIST="${1#--skip=}" ;;
+ *) tui_err "unknown flag: $1" >&2; exit 2 ;;
+ esac
+ shift
+ done
+ tui_banner "Texera Local Dev — stopping stack" "skip=${SKIP_LIST:-none}"
+ tui_section "Stopping (reverse order)"
+ local svc=""
+ # Stop native services first (reverse declaration order)
+ # bash arrays are 0-indexed: last element is at N-1.
+ for (( i=${#SERVICES[@]} - 1; i>=0; i-- )); do
+ svc="${SERVICES[i]}"
+ [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]] && continue
+ if is_skipped "$svc"; then
+ tui_skip "$svc: --skip"
+ continue
+ fi
+ stop_one "$svc"
+ done
+ # Then one project-level docker compose down for any docker target.
+ local has_docker_targets=false
+ for svc in "${SERVICES[@]}"; do
+ [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]] || continue
+ if is_skipped "$svc"; then
+ tui_skip "$svc: --skip"
+ continue
+ fi
+ has_docker_targets=true
+ done
+ $has_docker_targets && infra_down
+ printf "\n"
+}
+
+cmd_update_one() {
+ local svc="$1"
+ if ! amap_has SVC_TYPE "$svc"; then
+ tui_err "unknown service: ${BOLD}$svc${RESET}"
+ printf " ${DIM}Known:${RESET} ${SERVICES[*]}\n"
+ exit 1
+ fi
+ local type=""
+ type=$(amap_get SVC_TYPE "$svc")
+ case "$type" in
+ docker)
+ tui_banner "Restarting ${svc}" "docker compose restart $svc"
+ tui_step "$svc: docker compose restart $svc"
+ docker compose -p "$DOCKER_PROJECT" restart "$svc" >/dev/null 2>&1 \
+ && tui_ok "$svc: restarted" \
+ || { tui_err "$svc: restart failed"; exit 1; }
+ exit 0
+ ;;
+ yarn)
+ tui_warn "frontend uses ng's watch -- source changes hot-reload automatically."
+ printf " ${DIM}For dep changes: kill PID ${RESET}$(svc_running_pid frontend)${DIM}; then bin/local-dev.sh up${RESET}\n"
+ exit 0
+ ;;
+ bun)
+ tui_banner "Updating ${svc}" "bun install + bounce"
+ tui_section "Deps"
+ ( cd "$(amap_get SVC_CWD "$svc")" && bun install )
+ tui_section "Bounce"
+ stop_one "$svc"
+ start_one "$svc"
+ ;;
+ jvm)
+ local _sbt_proj=""
+ _sbt_proj=$(amap_get SVC_SBT "$svc")
+ if [[ -n "$_sbt_proj" ]]; then
+ tui_banner "Updating ${svc}" "sbt ${_sbt_proj}/dist + bounce"
+ else
+ tui_banner "Updating ${svc}" "bounce only (shares dist with its sibling)"
+ fi
+ tui_section "Build"
+ build_one_jvm "$svc"
+ tui_section "Bounce"
+ # amber's two siblings share a dist — rebuilding one moves the
+ # jar bytes underneath the other. Always bounce them together
+ # so neither ends up running stale code (or silently dead).
+ local sibling=""
+ case "$svc" in
+ texera-web) sibling="computing-unit-master" ;;
+ computing-unit-master) sibling="texera-web" ;;
+ esac
+ local sibling_was_running=false
+ if [[ -n "$sibling" ]] && [[ -n "$(svc_running_pid "$sibling")" ]]; then
+ sibling_was_running=true
+ tui_step "$sibling: stopping (shares amber dist with $svc)"
+ stop_one "$sibling"
+ fi
+ stop_one "$svc"
+ start_one "$svc"
+ if $sibling_was_running; then
+ start_one "$sibling"
+ fi
+ ;;
+ esac
+ tui_section "Health"
+ local _port=""
+ _port=$(amap_get SVC_PORT "$svc")
+ if wait_for_port "$_port" 60; then
+ printf " ${GREEN}${SYM_OK}${RESET} %-32s ${DIM}:%s${RESET}\n" "$svc" "$_port"
+ else
+ printf " ${RED}${SYM_ERR}${RESET} %-32s ${DIM}:%s${RESET} ${RED}timeout${RESET} ${DIM}(bin/local-dev.sh logs %s)${RESET}\n" \
+ "$svc" "$_port" "$svc"
+ exit 1
+ fi
+ printf "\n"
+}
+
+cmd_logs() {
+ local svc="${1:?usage: bin/local-dev.sh logs <service>}"
+ if ! amap_has SVC_TYPE "$svc"; then
+ echo "Unknown service: $svc" >&2
+ exit 1
+ fi
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ exec docker compose -p "$DOCKER_PROJECT" logs -f "$svc"
+ fi
+ exec tail -f "$LOG_DIR/$svc.log"
+}
+
+# Render the interactive dashboard panel (banner + service table + hint + summary).
+tui_render_dashboard() {
+ printf "\e[2J\e[H" # clear screen + home cursor (scrollback preserved)
+ local branch="" sha=""
+ branch=$(git -C "$REPO_ROOT" rev-parse --abbrev-ref HEAD 2>/dev/null || echo "?")
+ sha=$(git -C "$REPO_ROOT" rev-parse --short HEAD 2>/dev/null || echo "?")
+ tui_banner "Texera Local Dev — interactive" "branch: $branch @ $sha · $(date '+%H:%M:%S') · type ? for help"
+ printf "\n"
+
+ printf " ${BOLD}%-32s %-7s %-9s %-18s %-3s %s${RESET}\n" \
+ "SERVICE" "PORT" "PID" "ARTIFACT MTIME" "SRC" "STATE"
+ printf " ${GRAY}"; tui_hline "─" 32; printf " "
+ tui_hline "─" 7; printf " "; tui_hline "─" 9; printf " "
+ tui_hline "─" 18; printf " "; tui_hline "─" 3; printf " "
+ tui_hline "─" 12; printf "${RESET}\n"
+
+ local n_run=0 n_total=0 n_dirty=0
+ local svc=""
+ for svc in "${SERVICES[@]}"; do
+ n_total=$((n_total+1))
+ local pid="—" state="stopped" mtime="—" port_str="—" src_disp=" "
+ if [[ "$(amap_get SVC_TYPE "$svc")" == "docker" ]]; then
+ state=$(docker_svc_state "$svc")
+ mtime="docker"
+ port_str=":$(amap_get SVC_PORT "$svc")"
+ else
+ local found_pid=""
+ found_pid=$(svc_running_pid "$svc")
+ if [[ -n "$found_pid" ]]; then state="running"; pid="$found_pid"; fi
+ mtime=$(svc_artifact_mtime "$svc")
+ port_str=":$(amap_get SVC_PORT "$svc")"
+ fi
+ [[ "$state" == "running" ]] && n_run=$((n_run+1))
+
+ if svc_src_changed "$svc" 2>/dev/null; then
+ src_disp="${YELLOW}${BOLD}*${RESET} "
+ n_dirty=$((n_dirty+1))
+ fi
+
+ local sym="" color=""
+ sym=$(tui_state_symbol "$state")
+ color=$(tui_state_color "$state")
+ printf " %s " "$sym"
+ printf "${color}%-32s${RESET} %-7s ${DIM}%-9s${RESET} %-18s %s ${color}%s${RESET}\n" \
+ "$svc" "$port_str" "$pid" "$mtime" "$src_disp" "$state"
+ done
+
+ printf "\n"
+ local sum_color="$YELLOW"
+ (( n_run == n_total )) && sum_color="$GREEN"
+ (( n_run == 0 )) && sum_color="$GRAY"
+ printf " ${BOLD}${sum_color}%d of %d running${RESET}" "$n_run" "$n_total"
+ if (( n_dirty > 0 )); then
+ printf " ${YELLOW}${BOLD}*${RESET} ${DIM}%d with source changes${RESET}" "$n_dirty"
+ fi
+ printf "\n\n"
+ printf " ${DIM}Commands:${RESET} "
+ printf "${BOLD}r${RESET}efresh${DIM} (or just ↩)${RESET} · "
+ printf "${BOLD}u${RESET}p · ${BOLD}d${RESET}own · "
+ printf "${BOLD}b${RESET}uild · "
+ printf "${BOLD}<svc>${RESET}${DIM}=rebuild+bounce${RESET} · "
+ printf "${BOLD}l${RESET}ogs ${DIM}<svc>${RESET} · "
+ printf "${BOLD}s${RESET}top ${DIM}<svc>${RESET} · "
+ printf "${BOLD}q${RESET}uit\n\n"
+}
+
+# Pure monitoring mode: redraw the dashboard every $1 seconds, no prompt.
+# Ctrl-C to exit. Useful when watching a build/restart from another terminal.
+cmd_watch() {
+ if [[ ! -t 1 ]]; then
+ tui_err "watch mode requires a TTY"
+ exit 1
+ fi
+ local interval="${1:-2}"
+ trap 'printf "\e[?25h\n${DIM}bye${RESET}\n"; exit 0' EXIT INT TERM
+ printf "\e[?25l" # hide cursor
+ while true; do
+ tui_render_dashboard
+ printf " ${DIM}watch: refreshing every %ss · Ctrl-C to exit${RESET}\n" "$interval"
+ sleep "$interval"
+ done
+}
+
+# Pause and let the user read command output before re-rendering the dashboard.
+
+# Print the ordered list of Python interpreters we consider for launching the
+# Textual TUI: an explicit override, then any active venv, then the canonical
+# texera dev venv, then whatever `python3`/`python` happen to resolve to. We
+# de-duplicate as we go so the diagnostic doesn't show the same path twice.
+_probed_pythons() {
+ local seen=""
+ local cand=""
+ local raw=(
+ "${TEXERA_PYTHON:-}"
+ "${VIRTUAL_ENV:+$VIRTUAL_ENV/bin/python}"
+ "$(command -v python3 2>/dev/null)"
+ "$(command -v python 2>/dev/null)"
+ )
+ for cand in "${raw[@]}"; do
+ [[ -z "$cand" ]] && continue
+ case ":$seen:" in *":$cand:"*) continue ;; esac
+ seen="$seen:$cand"
+ printf '%s\n' "$cand"
+ done
+}
+
+# Walk `_probed_pythons` and return the first interpreter where `import
+# textual` succeeds, or empty string if none.
+_find_python_with_textual() {
+ local cand=""
+ while IFS= read -r cand; do
+ [[ -x "$cand" ]] || continue
+ if "$cand" -c "import textual" >/dev/null 2>&1; then
+ printf '%s\n' "$cand"
+ return 0
+ fi
+ done < <(_probed_pythons)
+ return 1
+}
+
+# Show every candidate Python the picker considered, its version, and
+# whether it can `import textual`. Pinpoints "I have the right python on
+# PATH but textual isn't installed THERE" vs "no python found at all".
+_diagnose_python() {
+ printf "\n ${BOLD}candidate interpreters:${RESET}\n"
+ local any=0
+ while IFS= read -r cand; do
+ any=1
+ if [[ ! -x "$cand" ]]; then
+ printf " ${RED}✗${RESET} %s ${DIM}(not executable)${RESET}\n" "$cand"
+ continue
+ fi
+ local ver=""
+ ver=$("$cand" --version 2>&1 | head -1) || ver="(unreadable)"
+ if "$cand" -c "import textual; print(textual.__version__)" >/dev/null 2>&1; then
+ local tv=""
+ tv=$("$cand" -c "import textual; print(textual.__version__)" 2>/dev/null) || tv="?"
+ printf " ${GREEN}✓${RESET} %s ${DIM}(%s, textual %s)${RESET}\n" "$cand" "$ver" "$tv"
+ else
+ printf " ${YELLOW}!${RESET} %s ${DIM}(%s, textual MISSING)${RESET}\n" "$cand" "$ver"
+ fi
+ done < <(_probed_pythons)
+ if (( any == 0 )); then
+ printf " ${RED}✗${RESET} ${DIM}no candidate python on PATH or in TEXERA_PYTHON / VIRTUAL_ENV${RESET}\n"
+ fi
+ printf "\n ${BOLD}env:${RESET}\n"
+ printf " \$TEXERA_PYTHON = ${DIM}%s${RESET}\n" "${TEXERA_PYTHON:-(unset)}"
+ printf " \$VIRTUAL_ENV = ${DIM}%s${RESET}\n" "${VIRTUAL_ENV:-(unset)}"
+ printf "\n"
+}
+
+# Hand off to the Python + Textual TUI. Hard requirement now (no more zsh
+# REPL fallback) — if we can't find a Python with `textual` installed,
+# print install instructions and exit non-zero. Use the non-interactive
+# `status` (or any other subcommand) when you don't have Python set up.
+cmd_interactive() {
+ if [[ ! -t 0 || ! -t 1 ]]; then
+ tui_err "interactive mode requires a TTY"
+ exit 1
+ fi
+ # zsh + `set -e` kills the script when a command substitution's command
+ # exits non-zero (`var=$(false)` → silent abort). Suppress with || true
+ # so the error path below actually gets to print the install hint.
+ local py=""
+ py="$(_find_python_with_textual)" || true
+ if [[ -z "$py" ]]; then
+ tui_err "interactive mode requires Python with the ${BOLD}textual${RESET} package"
+ _diagnose_python
+ _install_hint python
+ exit 1
+ fi
+ exec "$py" "$REPO_ROOT/bin/local-dev-tui.py"
+}
+
+# --------- main ---------
+# Warm the per-service source-dir cache so dirty checks below skip the
+# repeated BFS over the sbt graph. ~5ms of one-time cost; saves N×BFS
+# work per `cmd_up` / `cmd_auto` / status loop.
+_precompute_src_dirs
+
+case "${1:-}" in
+ ""|status) cmd_status ;; # default: one-shot dashboard (safe in scripts/CI)
+ -i|--interactive) cmd_interactive ;; # opt in to the live TUI
+ up) shift; cmd_up "$@" ;;
+ auto) shift; cmd_auto "$@" ;;
+ down) shift; cmd_down "$@" ;;
+ start) shift; start_one "${1:?need service name}" ;;
+ stop) shift; stop_one "${1:?need service name}" ;;
+ logs) shift; cmd_logs "${1:-}" ;;
+ w|watch) shift; cmd_watch "${1:-2}" ;;
+ version) printf "%s\n" "$TEXERA_VERSION" ;;
+ -h|--help) sed -n '18,67p' "$0" ;;
+ *) cmd_update_one "$1" ;;
+esac
diff --git a/bin/local-dev/docker-compose.override.yml b/bin/local-dev/docker-compose.override.yml
new file mode 100644
index 0000000000..94156e9e56
--- /dev/null
+++ b/bin/local-dev/docker-compose.override.yml
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Overlay for bin/local-dev.sh on top of bin/single-node/docker-compose.yml.
+# bin/single-node was designed so the texera microservices run inside the same
+# compose network as the infra — postgres/lakefs/lakekeeper/litellm therefore
+# don't publish their ports to the host. The local-dev flow runs the JVM and
+# Node services *natively* on the host, so we expose each backend port here.
+
+services:
+ postgres:
+ ports:
+ - "5432:5432"
+
+ lakefs:
+ ports:
+ - "8000:8000"
+
+ lakekeeper:
+ ports:
+ - "8181:8181"
+
+ litellm:
+ ports:
+ - "4000:4000"
+
+ # The default .env hardcodes STORAGE_S3_ENDPOINT=http://texera-minio:9000
+ # — only routable inside the docker compose network. lakekeeper-init bakes
+ # that string into the warehouse's storage profile, which Lakekeeper then
+ # both dials itself (for s3 ops) AND returns to clients. The host-native
+ # JVMs in local-dev mode can't resolve `texera-minio`.
+ #
+ # Use whatever STORAGE_S3_ENDPOINT the parent process exported. The
+ # wrapper script (bin/local-dev.sh) detects the host's LAN IP at startup
+ # and exports e.g. http://192.168.1.x:9000 — reachable from both sides:
+ # • lakekeeper container: out via docker bridge → host's published :9000
+ # • host JVM: straight to host's published :9000
+ # If the env wasn't exported (script not used), fall back to localhost
+ # which at least keeps the warehouse creation from blowing up.
+ lakekeeper-init:
+ environment:
+ STORAGE_S3_ENDPOINT: ${STORAGE_S3_ENDPOINT:-http://localhost:9000}
diff --git a/amber/dev-requirements.txt b/bin/local-dev/tests/__init__.py
similarity index 56%
copy from amber/dev-requirements.txt
copy to bin/local-dev/tests/__init__.py
index 4ab14960cb..13a83393a9 100644
--- a/amber/dev-requirements.txt
+++ b/bin/local-dev/tests/__init__.py
@@ -14,21 +14,3 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-
-# Test- and dev-only Python dependencies. Installed in CI after the
-# LICENSE-binary snapshot is taken, so packages here never appear in
-# pip-licenses output and never need to be tracked in LICENSE-binary
-# / NOTICE-binary. Not installed by packaging.
-
-# pytest test runner and plugins.
-pytest==9.0.3
-pytest-reraise==2.1.2
-pytest-timeout==2.4.0
-
-# Coverage instrumentation for pytest; emits coverage.xml consumed by
-# Codecov's Phase 1 upload.
-pytest-cov==7.1.0
-
-# protoc plugin for bin/python-proto-gen.sh; runtime needs only the
-# base `betterproto` in requirements.txt.
-betterproto[compiler]==2.0.0b7
diff --git a/bin/local-dev/tests/conftest.py b/bin/local-dev/tests/conftest.py
new file mode 100644
index 0000000000..7525cf3a87
--- /dev/null
+++ b/bin/local-dev/tests/conftest.py
@@ -0,0 +1,44 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""pytest fixtures for the local-dev TUI tests.
+
+The TUI lives as `bin/local-dev-tui.py` rather than a package, so we load it
+once here with `importlib.util` and expose the resulting module via the
+`tui` fixture. `sys.modules` registration is required so the module's
+`@dataclass` decorators can resolve `__module__` lookups."""
+
+from __future__ import annotations
+
+import importlib.util
+import sys
+from pathlib import Path
+
+import pytest
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+TUI_PATH = REPO_ROOT / "bin" / "local-dev-tui.py"
+
+
+@pytest.fixture(scope="session")
+def tui():
+ spec = importlib.util.spec_from_file_location("local_dev_tui", TUI_PATH)
+ assert spec is not None and spec.loader is not None
+ module = importlib.util.module_from_spec(spec)
+ sys.modules["local_dev_tui"] = module
+ spec.loader.exec_module(module)
+ return module
diff --git a/bin/local-dev/tests/test_local_dev_sh.sh b/bin/local-dev/tests/test_local_dev_sh.sh
new file mode 100755
index 0000000000..e467259b13
--- /dev/null
+++ b/bin/local-dev/tests/test_local_dev_sh.sh
@@ -0,0 +1,167 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Smoke tests for bin/local-dev.sh. Run from the repo root:
+# bash bin/local-dev/tests/test_local_dev_sh.sh
+# Exits 0 if every check passes, 1 otherwise.
+#
+# Kept deliberately small: bringing up the actual stack needs Docker /
+# sbt / a Mac and is out of scope for CI here. We cover the things that
+# regress quietly — script syntax, version-detection, the subcommand
+# dispatch, and graceful failure on garbage input.
+
+set -u
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
+SCRIPT="$REPO_ROOT/bin/local-dev.sh"
+
+PASS=0
+FAIL=0
+
+_pass() { printf " \e[32m✓\e[0m %s\n" "$1"; PASS=$((PASS+1)); }
+_fail() {
+ printf " \e[31m✗\e[0m %s\n" "$1"
+ [[ $# -ge 2 ]] && printf " %s\n" "$2"
+ FAIL=$((FAIL+1))
+}
+
+# 1) bash -n: syntax check. Catches everything from typos to unbalanced
+# heredocs without executing a line of the script.
+if bash -n "$SCRIPT" 2>/tmp/.local-dev-syntax.err; then
+ _pass "bash -n bin/local-dev.sh"
+else
+ _fail "bash -n bin/local-dev.sh" "$(cat /tmp/.local-dev-syntax.err)"
+fi
+rm -f /tmp/.local-dev-syntax.err
+
+# 2) `version` subcommand returns the same string we'd extract by hand
+# from build.sbt. This is the single source of truth that all the
+# dist / launcher / canary-jar paths in the script and the TUI build
+# off of, so we'd rather catch a regression here.
+script_version=$("$SCRIPT" version 2>/dev/null | head -1 | tr -d '[:space:]')
+sbt_version=$(
+ grep -E '^[[:space:]]*ThisBuild[[:space:]]*/[[:space:]]*version[[:space:]]*:=[[:space:]]*"' \
+ "$REPO_ROOT/build.sbt" 2>/dev/null \
+ | head -1 \
+ | sed -E 's/.*"([^"]+)".*/\1/' \
+ | tr -d '[:space:]'
+)
+if [[ -n "$script_version" && "$script_version" == "$sbt_version" ]]; then
+ _pass "version matches build.sbt ($script_version)"
+else
+ _fail "version mismatch" "script=$script_version build.sbt=$sbt_version"
+fi
+
+# 3) TEXERA_VERSION env var should override.
+override=$(TEXERA_VERSION="9.9.9-TEST" "$SCRIPT" version 2>/dev/null | head -1 | tr -d '[:space:]')
+if [[ "$override" == "9.9.9-TEST" ]]; then
+ _pass "TEXERA_VERSION env var overrides build.sbt"
+else
+ _fail "env override didn't take" "got: $override"
+fi
+
+# 4) `--help` prints usage.
+help_out=$("$SCRIPT" --help 2>&1 | head -20)
+if [[ "$help_out" == *"local-dev.sh"* && "$help_out" == *"Subcommands"* ]]; then
+ _pass "--help shows usage"
+else
+ _fail "--help didn't show usage" "$(echo "$help_out" | head -3)"
+fi
+
+# 5) An unknown service name routes through cmd_update_one and exits
+# non-zero rather than silently doing nothing.
+out=$("$SCRIPT" definitely-not-a-real-service 2>&1)
+rc=$?
+if (( rc != 0 )) && [[ "$out" == *"unknown service"* || "$out" == *"Unknown service"* ]]; then
+ _pass "unknown service exits non-zero with clear error"
+else
+ _fail "unknown service didn't error properly" "rc=$rc out=$out"
+fi
+
+# 6) `start` with no service name fails immediately (zsh parameter expansion
+# `${1:?...}` exits with the message).
+out=$("$SCRIPT" start 2>&1)
+rc=$?
+if (( rc != 0 )) && [[ "$out" == *"need service name"* ]]; then
+ _pass "start without arg refuses cleanly"
+else
+ _fail "start without arg should refuse" "rc=$rc out=$out"
+fi
+
+# 7) No-arg invocation must be non-interactive (= `status`). Previously the
+# default launched the TUI, which made the script unsafe to drop into
+# cron jobs or CI smoke tests. Anything that prints the banner without
+# hanging counts.
+out=$("$SCRIPT" 2>&1 | head -5)
+rc=$?
+if (( rc == 0 )) && [[ "$out" == *"Texera Local Dev"* ]]; then
+ _pass "no-arg invocation prints status (non-interactive)"
+else
+ _fail "no-arg invocation didn't print status" "rc=$rc out=$(echo "$out" | head -1)"
+fi
+
+# 8) `-i` without a TTY must refuse cleanly, not crash or hang. We pipe
+# stdin from /dev/null so the TTY check fires. Avoid piping into `head`
+# here — that masks the script's exit code under zsh.
+out=$("$SCRIPT" -i </dev/null 2>&1)
+rc=$?
+if (( rc != 0 )) && [[ "$out" == *"requires a TTY"* || "$out" == *"requires Python"* ]]; then
+ _pass "-i refuses cleanly without a TTY or Python"
+else
+ _fail "-i didn't refuse cleanly" "rc=$rc out=$(echo "$out" | head -1)"
+fi
+
+# 9) Regression: when no Python on the candidate list has textual, the
+# error message + install hint must actually print. The bug it
+# guards against: zsh's `set -e` aborts the script silently when a
+# command substitution's command exits non-zero (`var=$(returns 1)`),
+# so the install-hint code below the assignment never ran. `script`
+# allocates a pty so the TTY check passes and we get to the python
+# check.
+#
+# `script` has two incompatible invocation styles:
+# macOS BSD: script -q OUT_FILE CMD ARGS...
+# util-linux: script -qc "CMD ARGS..." OUT_FILE
+# Probe with `script --version`: util-linux supports it, BSD doesn't.
+if command -v script >/dev/null 2>&1; then
+ bad_py="/usr/bin/python3"
+ if [[ -x "$bad_py" ]] && ! "$bad_py" -c "import textual" >/dev/null 2>&1; then
+ if script --version >/dev/null 2>&1; then
+ # util-linux dialect
+ out=$(env -i HOME="$HOME" PATH=/usr/bin:/bin TERM="${TERM:-xterm}" \
+ script -qc "$SCRIPT -i; echo __rc=\$?" /dev/null </dev/null 2>&1)
+ else
+ # macOS BSD dialect
+ out=$(env -i HOME="$HOME" PATH=/usr/bin:/bin TERM="${TERM:-xterm}" \
+ script -q /dev/null sh -c "$SCRIPT -i; echo __rc=\$?" </dev/null 2>&1)
+ fi
+ if [[ "$out" == *"requires Python"* && "$out" == *"install Python"* && "$out" == *"__rc=1"* ]]; then
+ _pass "-i with no textual prints the install hint (regression for zsh set -e bug)"
+ else
+ _fail "-i with no textual didn't print install hint" \
+ "got: $(echo "$out" | head -3 | tr '\n' '|')"
+ fi
+ else
+ _pass "skip: no textual-less python available to test against"
+ fi
+else
+ _pass "skip: 'script' not on PATH"
+fi
+
+printf "\n%d passed, %d failed\n" "$PASS" "$FAIL"
+(( FAIL == 0 ))
diff --git a/bin/local-dev/tests/test_local_dev_tui.py b/bin/local-dev/tests/test_local_dev_tui.py
new file mode 100644
index 0000000000..6d61531a32
--- /dev/null
+++ b/bin/local-dev/tests/test_local_dev_tui.py
@@ -0,0 +1,376 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Unit tests for `bin/local-dev-tui.py`.
+
+We cover the pure helper functions that don't need a running Textual app:
+version detection, docker state mapping, ANSI scrubbing, source-hash
+fingerprinting, and the service catalog's structural invariants. The
+interactive widgets are exercised manually."""
+
+from __future__ import annotations
+
+import os
+import textwrap
+from pathlib import Path
+
+import pytest
+
+
+# ─────────────────── texera_version() ───────────────────
+
+def test_texera_version_parses_build_sbt(tmp_path, monkeypatch, tui):
+ sbt = tmp_path / "build.sbt"
+ sbt.write_text(textwrap.dedent("""\
+ ThisBuild / scalaVersion := "2.13.18"
+ ThisBuild / version := "9.9.9-FIXTURE"
+ ThisBuild / organization := "org.apache.texera"
+ """))
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.delenv("TEXERA_VERSION", raising=False)
+ assert tui.texera_version() == "9.9.9-FIXTURE"
+
+
+def test_texera_version_env_var_wins(tmp_path, monkeypatch, tui):
+ (tmp_path / "build.sbt").write_text('ThisBuild / version := "ignored-by-env"\n')
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.setenv("TEXERA_VERSION", "from-env")
+ assert tui.texera_version() == "from-env"
+
+
+def test_texera_version_raises_when_build_sbt_missing(tmp_path, monkeypatch, tui):
+ # No build.sbt in tmp_path. No fallback — must raise so a stale or
+ # broken checkout doesn't silently pick the wrong version.
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.delenv("TEXERA_VERSION", raising=False)
+ with pytest.raises(RuntimeError, match="build.sbt not found"):
+ tui.texera_version()
+
+
+def test_texera_version_raises_when_version_unparseable(tmp_path, monkeypatch, tui):
+ # build.sbt exists but the `ThisBuild / version := "…"` line isn't there.
+ (tmp_path / "build.sbt").write_text("name := \"texera\"\n")
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.delenv("TEXERA_VERSION", raising=False)
+ with pytest.raises(RuntimeError, match="could not find.*version"):
+ tui.texera_version()
+
+
+# ─────────────────── docker_state() ───────────────────
+
+@pytest.mark.parametrize("state,status,expected", [
+ ("running", "Up 5 minutes (healthy)", "running"),
+ ("running", "Up 1s (health: starting)", "starting"),
+ ("running", "Up 30s (unhealthy)", "unhealthy"),
+ ("running", "Up 2 minutes", "running"),
+ ("exited", "Exited (0) 4 minutes ago", "exited"),
+ ("exited", "Exited (137) 1 minute ago", "failed"),
+ ("created", "Created", "starting"),
+ ("restarting", "Restarting (1) 2 seconds ago", "starting"),
+ ("paused", "Up 10 minutes (Paused)", "starting"),
+ ("", "", "stopped"),
+ ("dead", "Dead", "stopped"),
+])
+def test_docker_state_mapping(state, status, expected, tui):
+ assert tui.docker_state(state, status) == expected
+
+
+# ─────────────────── _strip_ansi_motion() ───────────────────
+
+def test_strip_ansi_keeps_sgr_drops_motion(tui):
+ raw = "\x1b[32m✓\x1b[0m hello \x1b[2K\x1b[Hworld\r\r"
+ out = tui._strip_ansi_motion(raw)
+ # SGR (\e[32m, \e[0m) must survive so colours render
+ assert "\x1b[32m" in out
+ assert "\x1b[0m" in out
+ # cursor-positioning / erase / CR must be gone
+ assert "\x1b[2K" not in out
+ assert "\x1b[H" not in out
+ assert "\r" not in out
+
+
+def test_strip_ansi_idempotent(tui):
+ plain = "no escape codes here"
+ assert tui._strip_ansi_motion(plain) == plain
+
+
+# ─────────────────── source_hash() / is_dirty() ───────────────────
+
+def _seed_jvm_layout(repo: Path, svc_own_src: str, version: str = "9.9.9-T") -> None:
+ """Drop the directory layout source_hash() / is_dirty() walk.
+
+ Six shared `common/*/src` dirs + the per-service src dir + the dist
+ artifact jar at `target/<svc>-<version>/lib/...` so the artifact-mtime
+ lookup works."""
+ for d in ["dao", "config", "auth", "workflow-core", "workflow-operator", "pybuilder"]:
+ p = repo / "common" / d / "src"
+ p.mkdir(parents=True, exist_ok=True)
+ (p / "Stub.scala").write_text(f"// stub for common/{d}\n")
+ own = repo / svc_own_src
+ own.mkdir(parents=True, exist_ok=True)
+ (own / "Main.scala").write_text("object Main\n")
+
+
+def test_source_hash_stable_across_calls(tmp_path, monkeypatch, tui):
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ _seed_jvm_layout(tmp_path, "config-service/src")
+ svc = tui.SERVICES_BY_NAME["config-service"]
+ a = tui.source_hash(svc)
+ b = tui.source_hash(svc)
+ assert a == b
+ assert len(a) == 40 # SHA-1 hex
+
+
+def test_source_hash_changes_with_content(tmp_path, monkeypatch, tui):
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ _seed_jvm_layout(tmp_path, "config-service/src")
+ svc = tui.SERVICES_BY_NAME["config-service"]
+ before = tui.source_hash(svc)
+ (tmp_path / "config-service/src/Main.scala").write_text("object Main { def x = 1 }\n")
+ after = tui.source_hash(svc)
+ assert before != after
+
+
+def test_is_dirty_docker_always_clean(tui):
+ svc = tui.SERVICES_BY_NAME["postgres"]
+ assert tui.is_dirty(svc) is False
+
+
+def test_is_dirty_after_seed_then_edit(tmp_path, monkeypatch, tui):
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.setattr(tui, "BUILD_STAMP_DIR", tmp_path / "stamps")
+ (tmp_path / "stamps").mkdir()
+ _seed_jvm_layout(tmp_path, "config-service/src")
+
+ # Drop in a "jar" so the lazy seed path kicks in (it just needs to exist;
+ # the file's content isn't read by the dirty check, only the stamp's).
+ svc = tui.SERVICES_BY_NAME["config-service"]
+ jar = tmp_path / svc.artifact_jar
+ jar.parent.mkdir(parents=True, exist_ok=True)
+ jar.write_bytes(b"fake-jar-bytes")
+
+ # First call seeds the stamp and reports clean.
+ assert tui.is_dirty(svc) is False
+ stamp = tmp_path / "stamps" / svc.name
+ assert stamp.exists()
+ seeded_hash = stamp.read_text().strip()
+ assert len(seeded_hash) == 40
+
+ # Edit source → next dirty check sees a hash mismatch.
+ (tmp_path / "config-service/src/Main.scala").write_text("object Main { def y = 2 }\n")
+ assert tui.is_dirty(svc) is True
+
+
+def test_is_dirty_mtime_bump_without_content_change_stays_clean(tmp_path, monkeypatch, tui):
+ """Robustness against `git checkout` touching mtimes — the whole reason we
+ moved off pure-mtime detection. After seeding the stamp, simulating a
+ checkout (re-touching source files without changing content) must NOT
+ flash dirty."""
+ monkeypatch.setattr(tui, "REPO_ROOT", tmp_path)
+ monkeypatch.setattr(tui, "BUILD_STAMP_DIR", tmp_path / "stamps")
+ (tmp_path / "stamps").mkdir()
+ _seed_jvm_layout(tmp_path, "config-service/src")
+ svc = tui.SERVICES_BY_NAME["config-service"]
+ jar = tmp_path / svc.artifact_jar
+ jar.parent.mkdir(parents=True, exist_ok=True)
+ jar.touch()
+
+ tui.is_dirty(svc) # seed
+
+ # Simulate `git checkout` by touching all source files to "now" without
+ # changing content.
+ for f in (tmp_path / "config-service/src").rglob("*"):
+ if f.is_file():
+ os.utime(f, None)
+
+ assert tui.is_dirty(svc) is False
+
+
+# ─────────────────── Service catalog invariants ───────────────────
+
+def test_service_names_unique(tui):
+ names = [s.name for s in tui.SERVICES]
+ assert len(names) == len(set(names)), "duplicate service name in catalog"
+
+
+def test_service_ports_unique(tui):
+ ports = [s.port for s in tui.SERVICES]
+ assert len(ports) == len(set(ports)), "two services claim the same port"
+
+
+def test_jvm_services_have_sbt_project_and_src(tui):
+ for s in tui.SERVICES:
+ if s.type == "jvm":
+ # sbt_project is optional: sibling services (e.g.
+ # computing-unit-master) ride another service's dist and have
+ # no separate sbt invocation. own_src + artifact_jar are still
+ # required so dirty-detection and start_one work.
+ assert s.own_src, f"{s.name} missing own_src"
+ assert s.artifact_jar, f"{s.name} missing artifact_jar"
+ assert tui.TEXERA_VERSION in s.artifact_jar, (
+ f"{s.name}'s jar path should embed the dynamic version, got {s.artifact_jar}"
+ )
+
+
+def test_jvm_siblings_share_artifact_jar(tui):
+ """computing-unit-master is a launcher shipped inside amber's dist; it must
+ point at the same canary jar as texera-web so dirty-state stays in sync."""
+ cum = tui.SERVICES_BY_NAME["computing-unit-master"]
+ web = tui.SERVICES_BY_NAME["texera-web"]
+ assert cum.sbt_project is None, "computing-unit-master must not own an sbt project"
+ assert cum.artifact_jar == web.artifact_jar
+ assert cum.own_src == web.own_src
+
+
+def test_docker_services_have_no_jar(tui):
+ for s in tui.SERVICES:
+ if s.type == "docker":
+ assert s.artifact_jar is None
+
+
+def test_watch_types_constant(tui):
+ assert tui.WATCH_TYPES == {"yarn", "bun"}
+ # Every watch-type service must show up in the catalog
+ for t in tui.WATCH_TYPES:
+ assert any(s.type == t for s in tui.SERVICES), f"no service of type {t}"
+
+
+# ─────────────────── CommandHistory navigation ───────────────────
+
+def test_command_history_back_and_forward(tmp_path, tui):
+ h = tui.CommandHistory(history_file=tmp_path / "h")
+ for cmd in ["u", "d", "config-service"]:
+ h.push(cmd)
+ assert h._history == ["u", "d", "config-service"]
+
+ # ↑ from a fresh draft walks back from newest
+ assert h.back("draft-typing") == "config-service"
+ assert h.back("config-service") == "d"
+ assert h.back("d") == "u"
+ # At oldest, further ↑ returns None (caller keeps current value)
+ assert h.back("u") is None
+
+ # ↓ walks forward; once past newest it restores the draft
+ assert h.forward() == "d"
+ assert h.forward() == "config-service"
+ assert h.forward() == "draft-typing"
+ # At the live draft, ↓ is a no-op
+ assert h.forward() is None
+
+
+def test_command_history_dedup_consecutive(tmp_path, tui):
+ h = tui.CommandHistory(history_file=tmp_path / "h")
+ h.push("u")
+ h.push("u") # same as previous → not appended
+ h.push("d")
+ h.push("u") # not consecutive, gets appended
+ assert h._history == ["u", "d", "u"]
+
+
+def test_command_history_persists(tmp_path, tui):
+ f = tmp_path / "h"
+ a = tui.CommandHistory(history_file=f)
+ a.push("u")
+ a.push("d")
+
+ b = tui.CommandHistory(history_file=f)
+ assert b._history == ["u", "d"]
+
+
+def test_command_history_max_size(tmp_path, tui):
+ h = tui.CommandHistory(history_file=tmp_path / "h", max_size=3)
+ for cmd in ["a", "b", "c", "d", "e"]:
+ h.push(cmd)
+ reloaded = tui.CommandHistory(history_file=tmp_path / "h", max_size=3)
+ # On-disk cap: only the most recent 3 survive a reload.
+ assert reloaded._history == ["c", "d", "e"]
+
+
+# ─────────────────── build.sbt dependency parsing ───────────────────
+
+def test_sbt_graph_parsed_from_real_build_sbt(tui):
+ """The parser must extract at least the well-known subset of projects
+ (DAO, Config, Auth, the seven JVM services) and their direct deps from
+ the live repo's build.sbt. If any of these go missing we've drifted
+ from sbt's view of the world and dirty-detection is silently wrong."""
+ g = tui._SBT_GRAPH
+ assert "DAO" in g and "Config" in g and "Auth" in g
+ assert "ConfigService" in g
+ assert "WorkflowExecutionService" in g
+ # Auth depends on DAO + Config — anchor invariant from build.sbt.
+ assert set(g["Auth"]["deps"]) >= {"DAO", "Config"}
+ # WorkflowCore depends on DAO/Config/PyBuilder.
+ assert set(g["WorkflowCore"]["deps"]) >= {"DAO", "Config", "PyBuilder"}
+
+
+def test_sbt_transitive_closure_skips_unrelated(tui):
+ """config-service should NOT pull workflow-operator/workflow-core into
+ its dep set. That's the whole point of using build.sbt as source of
+ truth instead of a shared common/* list."""
+ dirs = tui._transitive_src_dirs("ConfigService", tui._SBT_GRAPH)
+ assert "common/workflow-operator/src" not in dirs
+ assert "common/workflow-core/src" not in dirs
+ # Should still include its own + Auth's reach (DAO/Config).
+ assert "config-service/src" in dirs
+ assert "common/auth/src" in dirs
+ assert "common/dao/src" in dirs
+ assert "common/config/src" in dirs
+
+
+def test_sbt_transitive_for_amber_includes_workflow_chain(tui):
+ """texera-web / computing-unit-master ride WorkflowExecutionService;
+ its closure must include WorkflowOperator → WorkflowCore → DAO/Config/PyBuilder."""
+ dirs = tui._transitive_src_dirs("WorkflowExecutionService", tui._SBT_GRAPH)
+ expected = {
+ "amber/src",
+ "common/workflow-operator/src",
+ "common/workflow-core/src",
+ "common/dao/src",
+ "common/config/src",
+ "common/pybuilder/src",
+ "common/auth/src",
+ }
+ assert expected <= set(dirs), f"missing: {expected - set(dirs)}"
+
+
+def test_sbt_unknown_project_raises(tui):
+ """Asking the graph for an unknown sbt project name raises rather
+ than silently falling back to a hardcoded list — that fallback would
+ mask a real build.sbt drift / typo."""
+ with pytest.raises(RuntimeError, match="not found in build.sbt graph"):
+ tui._transitive_src_dirs("NoSuchProject", tui._SBT_GRAPH)
+
+
+def test_sbt_empty_graph_raises(tui):
+ """A parse that returned no projects (very old / mangled build.sbt)
+ must raise. Silently using the prior pre-parse list would let dirty
+ detection report wrong results without anyone noticing."""
+ with pytest.raises(RuntimeError, match="graph is empty"):
+ tui._transitive_src_dirs("ConfigService", {})
+
+
+def test_sbt_test_scope_deps_ignored(tui):
+ """`X % "test->test"` is test-only — must not pollute the runtime dep
+ list (services would otherwise rebuild on test-only edits, defeating
+ the build skip)."""
+ # Auth has a `.dependsOn(DAO % "test->test")` next to its main
+ # `.dependsOn(DAO, Config)`. The test-scope reference should not
+ # appear as a second entry.
+ auth_deps = tui._SBT_GRAPH["Auth"]["deps"]
+ # DAO appears once (from the main-scope dependsOn) — not twice from
+ # the test-scope one.
+ assert auth_deps.count("DAO") == 1