[GH] (texera/backport/6011-order-worker-state-by-version-not-timest-v1.2): Workflow run "Required Checks" failed!

[GitBox <gi...@apache.org>]

The GitHub Actions job "Required Checks" on texera.git/backport/6011-order-worker-state-by-version-not-timest-v1.2 has failed.
Run started by GitHub user Yicong-Huang (triggered by Yicong-Huang).

Head commit for run:
48de98359d7a2af004ac11286f4b7f590c8faef1 / Yicong Huang <17...@users.noreply.github.com>
fix(amber): order worker state by version, not timestamp (#6011)

### What changes were proposed in this PR?

A fast **source** operator stayed **orange (RUNNING)** in the editor
after the run finished (issue #6010).

**Why it's a bug.** The controller reconstructs each worker's state from
several unordered channels (source `RUNNING` via the `startWorker`
response, non-source `RUNNING` via `workerStateUpdated`,
`COMPLETED`/`PAUSED` via `queryStatistics`/`pauseWorker` responses) and
reconciled them with **last-`System.nanoTime()`-wins** in
`WorkerExecution`. Worker state, however, is single-writer and strictly
ordered causally. For a tiny source the run finishes almost instantly,
so the `startWorker` response — carrying the stale `RUNNING` it sampled
at launch — can reach the controller *after* `COMPLETED` was recorded;
its later receipt timestamp wins and clobbers `COMPLETED`. Results
render fine (separate path), so only the border is stuck.

This PR orders worker state **causally** instead of by wall clock:

| Before | After |
| --- | --- |
| `WorkerExecution.update(nanoTime, state)` — last-write-wins by receipt
time | `updateState(version, state)` — newest **logical version** wins |
| stale late RUNNING clobbers COMPLETED | RUNNING (v2) < COMPLETED (v3)
⇒ COMPLETED kept |
| — | terminal states (COMPLETED/TERMINATED) are absorbing |

- `WorkerStateManager` bumps a monotonic `stateVersion` on every
`transitTo` (its state-machine logical clock).
- The version rides on every state report: `WorkerStateResponse`,
`WorkerStateUpdatedRequest`, `WorkerMetrics` (3 new proto fields).
- The controller applies a state only when its version is newer. Stats
stay timestamp-ordered (monotonic snapshots can share a state version).
- A single per-worker counter ⇒ no cross-process clock-sync assumptions
(why not worker timestamps).
- **pyamber parity:** the Python worker also reports state
(start/pause/resume + query-statistics). Its `StateManager` now bumps
the same monotonic version and the four handlers include it; otherwise
version-0 reports would be dropped after the first and reconfiguration
(RUNNING→PAUSED→RUNNING) would hang.

### Any related issues, documentation, discussions?

Closes #6010. The timestamp-based `update` was introduced in #3557.

### How was this PR tested?

JDK 17. Scala unit + Scala/Python integration + Python unit:

```
sbt "WorkflowExecutionService/testOnly \
  *WorkerExecutionSpec *WorkerStateManagerSpec *OperatorExecutionSpec \
  *RegionExecutionCoordinatorSpec *WorkflowExecutionCoordinatorSpec \
  *RegionExecutionSpec *WorkflowExecutionSpec"

# Scala-Python reconfiguration end-to-end (spawns Python UDF workers):
sbt "WorkflowExecutionService/testOnly *ReconfigurationIntegrationSpec"   # 3 passed

# Python unit:
cd amber && pytest src/test/python/core/architecture/managers/test_state_manager.py   # 9 passed
```

- `WorkerExecutionSpec`: version ordering (positive +
stale/equal-version negatives), terminal-state absorption,
`forceTerminate`, independent stats-vs-state ordering, and a named
regression for #6010 (`COMPLETED` survives a late `RUNNING`). Verified
the regression goes **red** when `updateState` is reverted to
last-write-wins.
- `WorkerStateManagerSpec` / `test_state_manager.py`: version starts at
0, bumps per successful transition, no bump on no-op self-transition or
rejected transition.
- `ReconfigurationIntegrationSpec` reproduced the failure (timeout)
before the pyamber fix and passes after.

### Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 4.8 (1M context), via Claude Code

---------

(backported from commit f5217504c3cfd8ac1d56f6617d7849895af17f59)

Co-authored-by: Claude Fable 5 <no...@anthropic.com>

Report URL: https://github.com/apache/texera/actions/runs/30512223015

With regards,
GitHub Actions via GitBox