You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by da...@apache.org on 2026/09/17 11:07:14 UTC
(camel) branch tui-ollama-tab created (now c5259a0393de)
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a change to branch tui-ollama-tab
in repository https://gitbox.apache.org/repos/asf/camel.git
at c5259a0393de CAMEL-24794: camel-jbang - TUI manual explains working with a local Ollama model
This branch includes the following new commits:
new ad704f3280d5 CAMEL-24794: camel-jbang - TUI Ollama tab showing local model performance
new 8dbf082cfb19 CAMEL-24794: camel-jbang - TUI Ollama tab F1 help explains the phases, panels and columns
new a3ed5ea24f16 CAMEL-24794: camel-jbang - TUI Ollama tab shows context fill per turn and compactions
new 3f195298bffc CAMEL-24794: camel-jbang - TUI Ollama tab counts cached prompt tokens once
new f9e59142a0bc camel-jbang-tui: F1 help files load under the camel launcher
new 98b4b1adff64 CAMEL-24794: camel-jbang - TUI lists the Ollama tab only while an Ollama server answers
new c58cdae15641 CAMEL-24794: camel-jbang - TUI Ollama tab Context panel reads the cache hit from the last request when idle
new e5a0c17cb599 CAMEL-24794: camel-jbang - TUI Ollama tab groups the request log per question
new c5259a0393de CAMEL-24794: camel-jbang - TUI manual explains working with a local Ollama model
The 9 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(camel) 05/09: camel-jbang-tui: F1 help files load under the camel launcher
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch tui-ollama-tab
in repository https://gitbox.apache.org/repos/asf/camel.git
commit f9e59142a0bceb739370db258dee28ba2fa1a8b7
Author: Claus Ibsen <cl...@gmail.com>
AuthorDate: Thu Sep 17 12:37:33 2026 +0200
camel-jbang-tui: F1 help files load under the camel launcher
DocHelper read tui/help/*.md through the classloader of a camel-jbang-core
class. The camel launcher loads plugin jars in a child URLClassLoader with
core in the parent, so the parent could not see the plugin's resources and
every tab's F1 help came back null (no F1 hint in the footer, F1 did nothing)
even though the files are in the jar. Resolve from this plugin's own class
first, like Theme does for its stylesheets, with the context classloader and
the core loader as fallbacks for a flat classpath.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Signed-off-by: Claus Ibsen <cl...@gmail.com>
---
.../dsl/jbang/core/commands/tui/DocHelper.java | 32 ++++++++++++++++++----
1 file changed, 27 insertions(+), 5 deletions(-)
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocHelper.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocHelper.java
index 2e1c92174b78..73a1425c6888 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocHelper.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocHelper.java
@@ -51,15 +51,37 @@ final class DocHelper {
return cached;
}
+ /**
+ * Reads a resource of this plugin. Under the {@code camel} launcher the plugin jar lives in its own classloader
+ * with camel-jbang-core in the parent, so the lookup must start from a class of this jar; the context classloader
+ * and the core's loader are fallbacks for other setups (tests, a flat classpath).
+ */
static String loadResourceContent(String resourcePath) {
- try (InputStream is = ExampleHelper.class.getClassLoader().getResourceAsStream(resourcePath)) {
- if (is != null) {
- return new String(is.readAllBytes(), StandardCharsets.UTF_8);
+ InputStream is = null;
+ ClassLoader own = DocHelper.class.getClassLoader();
+ if (own != null) {
+ is = own.getResourceAsStream(resourcePath);
+ }
+ if (is == null) {
+ ClassLoader tccl = Thread.currentThread().getContextClassLoader();
+ if (tccl != null) {
+ is = tccl.getResourceAsStream(resourcePath);
+ }
+ }
+ if (is == null) {
+ ClassLoader core = ExampleHelper.class.getClassLoader();
+ if (core != null) {
+ is = core.getResourceAsStream(resourcePath);
}
+ }
+ if (is == null) {
+ return null;
+ }
+ try (InputStream in = is) {
+ return new String(in.readAllBytes(), StandardCharsets.UTF_8);
} catch (IOException e) {
- // ignore
+ return null;
}
- return null;
}
static String downloadContent(String url) {
(camel) 09/09: CAMEL-24794: camel-jbang - TUI manual explains working with a local Ollama model
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch tui-ollama-tab
in repository https://gitbox.apache.org/repos/asf/camel.git
commit c5259a0393def1d69efe5b1051868f2bfd73b989
Author: Claus Ibsen <cl...@gmail.com>
AuthorDate: Thu Sep 17 13:05:08 2026 +0200
CAMEL-24794: camel-jbang - TUI manual explains working with a local Ollama model
New subsection under the AI provider docs: how a question is spent with
Ollama (load, prefill, decode and time to first token, with figures
measured on an M4 Pro with qwen3.6:35b-a3b), why one question costs several
requests and how the prompt cache keeps that affordable, the 32k context
window, compaction and its re-prefill cost, the rule to keep every Ollama
client on the same context size, how the model is chosen, where to look
(Ollama tab, /context, /usage, Run Doctor), remote and containerised
Ollama, and links to the local model benchmark and GenAI observability
blog posts. The Ollama tab section points to it.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Signed-off-by: Claus Ibsen <cl...@gmail.com>
---
.../modules/ROOT/pages/camel-jbang-tui.adoc | 69 +++++++++++++++++++++-
1 file changed, 68 insertions(+), 1 deletion(-)
diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
index 2d3f8a0373a6..64db50b05aa8 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
@@ -451,7 +451,8 @@ second. Against a remote or containerised Ollama the tab keeps the model, per-re
and says which panels are unavailable.
Press *r* to reset the request log and the session totals, *F5* to refresh immediately. The same data
-is available to AI agents through the `tui_get_ollama` MCP tool.
+is available to AI agents through the `tui_get_ollama` MCP tool. For what the figures mean for the AI
+panel and which knobs to turn, see <<_working_with_a_local_ollama_model>>.
== Process Information
@@ -904,6 +905,72 @@ all tools to a local model too, `/tools core` to trim the set for a hosted one,
model loaded for 30 minutes and use a 32k context window (`OLLAMA_CONTEXT_LENGTH` overrides it),
so follow-up questions reuse the cached prompt instead of reloading the model.
+==== Working with a local Ollama model
+
+A local model is not a slower version of a hosted one; it spends its time differently, and the TUI
+shows you where. This section explains what a question costs with Ollama and which knobs matter,
+with figures measured on an Apple M4 Pro (64 GB) running `qwen3.6:35b-a3b`.
+
+*How a question is spent.* Ollama answers in three phases, and every timing the TUI shows maps to one
+of them:
+
+1. *Load*: if the model is not in memory (first question, the keep-alive expired, or a request asked for
+ a different context size) Ollama starts a runner and loads the weights: 10 to 20 seconds for a 22 GB
+ model. The TUI asks Ollama to keep the model loaded for 30 minutes after each request.
+2. *Prefill*: the prompt (system prompt, tool definitions, conversation history, your question) is
+ processed in one batch, at roughly 600 to 700 tokens per second when nothing is cached.
+3. *Decode*: the answer is generated token by token, at 50 to 60 tokens per second for this model.
+
+The wait before anything appears, the time to first token, is load plus prefill. A cold first question
+therefore takes 20 seconds before the first word; a warm one under a second.
+
+*One question, many requests.* The panel answers by calling the `tui_*` tools, and every tool call the
+model makes costs a new request that sends the whole prompt again. A simple question can take 3 to 13
+requests. This is affordable only because Ollama caches the prompt prefix: the system prompt, the tool
+definitions and the history are identical from one step to the next, so each step prefills only the new
+tokens and takes about half a second. Across a session the cache hit is typically above 90%. The
+Ollama tab shows the request count per question as `×N` and the cache hit per question; a question
+with a high count and a short answer is the model exploring, which a smaller, sharper tool set reduces
+(see <<_tool_set_for_local_models>>).
+
+*The context window.* The TUI asks Ollama for a 32k context window; `OLLAMA_CONTEXT_LENGTH` overrides
+it. The static prefix of system prompt and core tools is about 4.5k tokens, 14% of the window, and each
+question with tool calls adds another 2k to 4k of history. The panel compacts the history once it
+grows past roughly 16k tokens (see `/compact` under <<_ai_panel_slash_commands>>). Compacting rewrites
+the history, which invalidates Ollama's prompt cache, so the request after a compaction prefills the
+whole prompt again: about 40 seconds for a 21k-token prompt. That is why the panel compacts local
+history late and rarely rather than a little on every turn. Two practical rules follow:
+
+* Keep every Ollama client on the same context size. A request with a different `num_ctx` makes
+ Ollama reload the model, which costs a cold start. The TUI's AI panel, `camel ask` and your own routes
+ should agree on the value.
+* A bigger window is cheap in memory for mixture-of-experts models with hybrid attention (this model
+ grows from 22.5 GB at 32k to 23.6 GB at 262k) but every token in it is prefill time after a
+ compaction or a reload, so raise it with a purpose.
+
+*Choosing the model.* With no `camel.tui.ai.model` set the panel takes `llama3.2` when it is installed,
+otherwise the first installed model; run `/model <name>` in the panel or set *AI Model* in
+*F2 -> Settings* to pin one. The model must support tool calling and should have at least 14B
+parameters; a mixture-of-experts model such as `qwen3.6:35b-a3b` prefills several times faster than a
+dense model of similar quality, which is what matters for a tool-heavy prompt. *F2 -> Run Doctor* shows
+whether Ollama was found, which models are installed and whether they are large enough.
+
+*Where to look.* The <<_ollama>> tab is the instrument for all of the above: tokens per second live and
+per request, time to first token with cold starts marked, cache hit, how full the context window is
+and how it grows per question, GPU and process load, and one line per question with the requests it
+took. In the AI panel, `/context` prints what the next request will cost, `/usage` and *Ctrl+U* the
+session totals per question.
+
+*Remote and containerised Ollama.* Everything above applies to an Ollama on another host or inside
+`camel infra run ollama` as well, with two differences: the container runs without GPU acceleration,
+and the live runner state and host load on the Ollama tab need the server on the same machine.
+
+For the wider picture see the blog posts
+https://camel.apache.org/blog/2026/09/camel-local-model-benchmark/index.html[We had a frontier AI coach a small local model through Camel]
+on what a local model can do with Camel and what was changed to help it, and
+https://camel.apache.org/blog/2026/09/camel-genai-observability-jbang/index.html[Observe Your Camel AI Routes with GenAI OpenTelemetry]
+on observing routes that call Ollama.
+
==== Using an OpenAI-compatible local server
Set `LLM_API_KEY` and `LLM_BASE_URL` to connect to any OpenAI-compatible server
(camel) 01/09: CAMEL-24794: camel-jbang - TUI Ollama tab showing local model performance
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch tui-ollama-tab
in repository https://gitbox.apache.org/repos/asf/camel.git
commit ad704f3280d5b448e39b8f69a0dfa4f3e0222e78
Author: Claus Ibsen <cl...@gmail.com>
AuthorDate: Thu Sep 17 11:55:44 2026 +0200
CAMEL-24794: camel-jbang - TUI Ollama tab showing local model performance
Adds an Ollama tab (More menu, AI group) that shows how the model served by
Ollama is performing, in the spirit of the llm-visuals dashboard but built on
what Ollama exposes:
- Ollama API: version, loaded models (size, GPU share, allocated context,
unload time), installed models and the model shape from /api/show
(architecture, layers, experts and active experts, max context,
capabilities).
- Per-request timings Ollama returns: prompt and generation token counts and
durations, load time, total time and cached prompt tokens, giving prefill
and decode tokens per second, time to first token, cache hit ratio and
cold versus warm starts. The TUI AI panel feeds these for its own requests;
calls made by Camel routes arrive through the GenAI observability spans
(tokens and duration, tagged with the route id).
- On the same machine: Ollama's llama-server runner is found by its command
line and its /slots endpoint polled for live state (working or idle,
prompt tokens processed, cache tokens, tokens decoded, context size,
speculative decoding), which yields a live decode rate and context fill for
every caller. GPU utilization and memory come from ioreg on macOS and
nvidia-smi on Linux; CPU and memory of the server and runner processes from
ps. Remote or containerised Ollama keeps the API and per-request data and
says which panels are unavailable.
The LLM client in camel-jbang-core now keeps Ollama's load_duration,
total_duration and prompt_eval_cached_count in TokenUsage (additive, the
existing constructors remain).
The same data is exposed to AI agents as the tui_get_ollama MCP tool and
through tui_get_table on the Ollama tab. Documented in the TUI user manual.
Out of scope, since Ollama does not expose them: speculative decoding
acceptance rate and expert routing.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Signed-off-by: Claus Ibsen <cl...@gmail.com>
---
.../modules/ROOT/pages/camel-jbang-tui.adoc | 40 +-
.../camel/dsl/jbang/core/commands/LlmClient.java | 43 +-
.../camel/dsl/jbang/core/commands/tui/AiPanel.java | 5 +
.../dsl/jbang/core/commands/tui/CamelMonitor.java | 12 +
.../dsl/jbang/core/commands/tui/McpFacade.java | 12 +
.../jbang/core/commands/tui/MonitorContext.java | 2 +
.../dsl/jbang/core/commands/tui/OllamaMonitor.java | 927 +++++++++++++++++++++
.../dsl/jbang/core/commands/tui/OllamaParsers.java | 439 ++++++++++
.../dsl/jbang/core/commands/tui/OllamaTab.java | 692 +++++++++++++++
.../dsl/jbang/core/commands/tui/TabRegistry.java | 9 +-
.../jbang/core/commands/tui/TokenRateWindow.java | 80 ++
.../dsl/jbang/core/commands/tui/TuiIcons.java | 1 +
.../core/commands/tui/TuiToolDefinitions.java | 9 +
.../jbang/core/commands/tui/TuiToolRegistry.java | 12 +-
.../jbang/core/commands/tui/OllamaMonitorTest.java | 271 ++++++
.../jbang/core/commands/tui/OllamaParsersTest.java | 222 +++++
.../core/commands/tui/OllamaTabRenderTest.java | 240 ++++++
.../core/commands/tui/TokenRateWindowTest.java | 77 ++
18 files changed, 3076 insertions(+), 17 deletions(-)
diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
index ccdebce532f5..f8c97d5ced6f 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
@@ -150,6 +150,7 @@ The *More* menu (key *0*) opens a popup with tabs organized into groups:
* *Routing* -- Browse Endpoints, Consumers, HTTP, Inflight, Producers, Route Controller
* *Observability* -- Circuit Breaker, Health, JFR, Metrics, Network Services, Exchange Events, Recovery Tasks, OpenTelemetry Spans
+* *AI* -- Ollama
* *Data* -- JDBC DataSource, Kafka, SQL Query, SQL Trace
* *JVM* -- Classpath, Heap Memory Histogram, Memory Usage, Memory Leak, Process, Startup, Threads
* *Project* -- Beans, Catalog, Configuration, CVE Audit, Maven Dependencies, Type Converters, Data Type Transformers
@@ -411,6 +412,42 @@ Features:
This is especially useful for understanding latency in multi-route integrations and for
correlating Camel processing with external service calls (HTTP, database, messaging).
+== Ollama
+
+The Ollama tab (under More, in the *AI* group) shows how the model served by a local
+https://ollama.com[Ollama] is performing, in the spirit of an LLM dashboard. It works with or without a
+running integration: it finds Ollama at `localhost:11434`, at the address of `camel infra run ollama`,
+or at the endpoint the AI panel (*F8*) is using.
+
+* *Model* -- the loaded model with its family, parameters, quantization, layers, experts (and how many
+ are active per token for a mixture-of-experts model), how much of it sits in GPU memory, the
+ allocated context length and when Ollama will unload it. With no model loaded, the installed models
+ are listed instead.
+* *Throughput* -- decode and prefill tokens per second, *live* while the model is generating and
+ otherwise from the last request; time to first token and load time (a load of a second or more
+ is a cold start); session averages and a sparkline of the decode rate.
+* *Context* -- how full the context window is, the share of the prompt served from Ollama's cache,
+ whether the model is working or idle, and the speculative decoding method in use.
+* *Host* -- GPU utilization and memory (Apple silicon through `ioreg`, NVIDIA through `nvidia-smi`),
+ and CPU and memory of the Ollama server and its model runner.
+* *Requests* -- one line per request with input, output and cached tokens, prefill and decode
+ tokens per second, time to first token, total time and the stop reason.
+
+Two kinds of requests appear in the log. Questions asked in the AI panel with Ollama as the provider
+come with the timings Ollama reports for each request (prompt evaluation, generation, model load,
+total). Calls made by Camel routes through `camel-langchain4j-chat`, `camel-openai` or
+`camel-spring-ai-chat` appear when the integration runs with GenAI observability (`--observe`, or
+`--dep=camel:ai-observability`), tagged with the route id; Camel records tokens and duration for those,
+not the phase split.
+
+The live figures, the context panel and the host panel need the model runner on the same machine:
+Ollama starts a `llama-server` process per loaded model and the tab reads its slot state a few times a
+second. Against a remote or containerised Ollama the tab keeps the model, per-request and session data
+and says which panels are unavailable.
+
+Press *r* to reset the request log and the session totals, *F5* to refresh immediately. The same data
+is available to AI agents through the `tui_get_ollama` MCP tool.
+
== Process Information
The Process tab (under More, shortcut *p*) shows JVM process details for the selected
@@ -1123,7 +1160,8 @@ through the TUI's confirm dialog or live replay. The `tui_` tools are the ones o
by purpose:
* *Observe* -- read the screen, get structured state, query tables/logs/errors/traces/topology/diagram/files,
- list the running infra services (brokers, databases) and read their logs
+ list the running infra services (brokers, databases) and read their logs, read the performance of the
+ local Ollama server (`tui_get_ollama`: loaded model, tokens per second, context fill, host load, request log)
* *Navigate* -- switch tabs, select integrations, select routes, send keystrokes, apply filters
* *Act* -- send test messages to endpoints, start/stop/restart routes, change log levels,
list and launch the bundled examples, start/stop/restart infra services, run any entry of
diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java
index de73bca11aed..4a572eed7d7b 100644
--- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java
+++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java
@@ -125,18 +125,25 @@ public class LlmClient {
}
/**
- * Token usage of one request, plus whatever the provider reveals about its prompt cache: hosted APIs report how
- * many input tokens were served from cache ({@code cachedTokens}: OpenAI {@code cached_tokens}, Anthropic
- * {@code cache_read_input_tokens}, Gemini {@code cachedContentTokenCount}), while Ollama reports no cache figure
- * but does report how long prompt processing and generation took ({@code prefillMillis}, {@code generationMillis}),
- * and a prompt served from its KV cache shows as a near-zero prefill. Zero means not reported.
+ * Token usage of one request, plus whatever the provider reveals about its prompt cache and timing: hosted APIs
+ * report how many input tokens were served from cache ({@code cachedTokens}: OpenAI {@code cached_tokens},
+ * Anthropic {@code cache_read_input_tokens}, Gemini {@code cachedContentTokenCount}). Ollama reports the prompt
+ * tokens served from its KV cache ({@code prompt_eval_cached_count}) and how long each phase took: prompt
+ * processing ({@code prefillMillis}), generation ({@code generationMillis}), loading the model into memory
+ * ({@code loadMillis}, near zero for a warm model) and the whole request ({@code totalMillis}). Zero means not
+ * reported.
*/
public record TokenUsage(int inputTokens, int outputTokens, int totalTokens,
- int cachedTokens, long prefillMillis, long generationMillis) {
+ int cachedTokens, long prefillMillis, long generationMillis, long loadMillis, long totalMillis) {
public static final TokenUsage EMPTY = new TokenUsage(0, 0, 0);
public TokenUsage(int inputTokens, int outputTokens, int totalTokens) {
- this(inputTokens, outputTokens, totalTokens, 0, 0, 0);
+ this(inputTokens, outputTokens, totalTokens, 0, 0, 0, 0, 0);
+ }
+
+ public TokenUsage(int inputTokens, int outputTokens, int totalTokens,
+ int cachedTokens, long prefillMillis, long generationMillis) {
+ this(inputTokens, outputTokens, totalTokens, cachedTokens, prefillMillis, generationMillis, 0, 0);
}
public TokenUsage add(TokenUsage other) {
@@ -146,7 +153,9 @@ public class LlmClient {
totalTokens + other.totalTokens,
cachedTokens + other.cachedTokens,
prefillMillis + other.prefillMillis,
- generationMillis + other.generationMillis);
+ generationMillis + other.generationMillis,
+ loadMillis + other.loadMillis,
+ totalMillis + other.totalMillis);
}
/** Whether the provider reported a prompt-cache figure or a timing split. */
@@ -1059,8 +1068,8 @@ public class LlmClient {
StringBuilder fullText = new StringBuilder();
List<ToolCall> toolCalls = new ArrayList<>();
String[] doneReasonHolder = { null };
- int[] tokenHolder = { 0, 0 };
- long[] durationHolder = { 0, 0 };
+ int[] tokenHolder = { 0, 0, 0 };
+ long[] durationHolder = { 0, 0, 0, 0 };
response.body().forEach(line -> {
if (line.isBlank()) {
@@ -1112,8 +1121,11 @@ public class LlmClient {
doneReasonHolder[0] = chunk.getString("done_reason");
tokenHolder[0] = getIntValue(chunk, "prompt_eval_count");
tokenHolder[1] = getIntValue(chunk, "eval_count");
+ tokenHolder[2] = getIntValue(chunk, "prompt_eval_cached_count");
durationHolder[0] = getLongValue(chunk, "prompt_eval_duration") / 1_000_000;
durationHolder[1] = getLongValue(chunk, "eval_duration") / 1_000_000;
+ durationHolder[2] = getLongValue(chunk, "load_duration") / 1_000_000;
+ durationHolder[3] = getLongValue(chunk, "total_duration") / 1_000_000;
}
} catch (Exception e) {
// skip malformed chunks
@@ -1129,8 +1141,8 @@ public class LlmClient {
= !toolCalls.isEmpty() ? "tool_calls" : (doneReasonHolder[0] != null ? doneReasonHolder[0] : "stop");
TokenUsage usage = new TokenUsage(
- tokenHolder[0], tokenHolder[1], tokenHolder[0] + tokenHolder[1], 0,
- durationHolder[0], durationHolder[1]);
+ tokenHolder[0], tokenHolder[1], tokenHolder[0] + tokenHolder[1], tokenHolder[2],
+ durationHolder[0], durationHolder[1], durationHolder[2], durationHolder[3]);
if (verbose) {
printer.println("[verbose] Streamed Ollama: text=" + (text != null ? truncateVerbose(text) : "null")
+ ", toolCalls=" + toolCalls.size() + ", doneReason=" + doneReasonHolder[0]
@@ -1479,9 +1491,12 @@ public class LlmClient {
int inputTokens = getIntValue(response, "prompt_eval_count");
int outputTokens = getIntValue(response, "eval_count");
TokenUsage usage = new TokenUsage(
- inputTokens, outputTokens, inputTokens + outputTokens, 0,
+ inputTokens, outputTokens, inputTokens + outputTokens,
+ getIntValue(response, "prompt_eval_cached_count"),
getLongValue(response, "prompt_eval_duration") / 1_000_000,
- getLongValue(response, "eval_duration") / 1_000_000);
+ getLongValue(response, "eval_duration") / 1_000_000,
+ getLongValue(response, "load_duration") / 1_000_000,
+ getLongValue(response, "total_duration") / 1_000_000);
if (verbose) {
printer.println("[verbose] Parsed Ollama: text=" + (content != null ? truncateVerbose(content) : "null")
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java
index 84696942d1dc..1e2608825dfa 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java
@@ -1992,6 +1992,11 @@ class AiPanel {
response.usage().totalTokens(), latencyMs,
response.stopReason(), Instant.now(),
AiUsageSource.TUI, null, questionCounter));
+ if (ctx != null && ctx.ollamaMonitor != null && client.apiType() == LlmClient.ApiType.ollama) {
+ // the Ollama tab shows this request with the timings Ollama returned
+ ctx.ollamaMonitor.adoptEndpoint(client.endpointUrl());
+ ctx.ollamaMonitor.recordRequest(model, response.usage(), latencyMs, response.stopReason());
+ }
}
void render(Frame frame, Rect area) {
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java
index dcaffdadce42..55f3feef7a82 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java
@@ -165,6 +165,7 @@ public class CamelMonitor extends CamelCommand {
// selectedPid is stored on ctx (MonitorContext) so tabs can access it
private DataRefreshService dataService;
+ private OllamaMonitor ollamaMonitor;
private String monitorNotification;
private boolean monitorNotificationError;
private long monitorNotificationExpiry;
@@ -311,6 +312,8 @@ public class CamelMonitor extends CamelCommand {
createDataService();
ctx = new MonitorContext(dataService.data(), dataService.infraData());
dataService.setContext(ctx);
+ ollamaMonitor = new OllamaMonitor();
+ ctx.ollamaMonitor = ollamaMonitor;
// popups, tabs and their cross-wiring
createActionsPopup();
@@ -2938,6 +2941,15 @@ public class CamelMonitor extends CamelCommand {
aiPanel.spanRefreshRequested = false;
dataService.refreshSpanData();
}
+ if (tabRegistry.selectedTabIndex() == TAB_MORE
+ && tabRegistry.getActiveMoreTab() == tabRegistry.ollamaTab() && ollamaMonitor != null) {
+ ollamaMonitor.poll();
+ // route calls to Ollama come from the GenAI spans of the selected integration, refreshed sparingly
+ if (ctx.selectedPid != null && ollamaMonitor.wantsSpans()) {
+ dataService.refreshSpanData();
+ ollamaMonitor.ingestSpans(dataService.otelSpans().get());
+ }
+ }
}
// ---- Helpers ----
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/McpFacade.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/McpFacade.java
index d0120e7379d3..095b3dfc467d 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/McpFacade.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/McpFacade.java
@@ -642,6 +642,18 @@ class McpFacade {
return tabRegistry.diagramTab().getTopologyDataAsJson();
}
+ /** Snapshot of the Ollama monitor for the tui_get_ollama tool; polls first so the caller sees current data. */
+ JsonObject getOllamaData(int limit) {
+ OllamaMonitor monitor = ctx.ollamaMonitor;
+ if (monitor == null) {
+ JsonObject none = new JsonObject();
+ none.put("connected", false);
+ return none;
+ }
+ monitor.poll();
+ return monitor.toJson(limit);
+ }
+
JsonObject getSpanData(String traceId, int limit) {
String pid = ctx.selectedPid;
if (pid == null) {
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/MonitorContext.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/MonitorContext.java
index 093889fcedb1..638ee812a9df 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/MonitorContext.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/MonitorContext.java
@@ -64,6 +64,8 @@ class MonitorContext {
boolean panelTop;
/** Shell/AI panel is drawn over the tab instead of taking space away from it. */
boolean panelOverlay;
+ /** Collector behind the Ollama tab and the tui_get_ollama tool; set by CamelMonitor at startup. */
+ OllamaMonitor ollamaMonitor;
BiConsumer<String, Boolean> notificationCallback;
BiConsumer<String, String> openMarkdownCallback;
OpenOptionsCallback openOptionsCallback;
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
new file mode 100644
index 000000000000..0b6cca44a239
--- /dev/null
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
@@ -0,0 +1,927 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.tui;
+
+import java.io.InputStream;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.charset.StandardCharsets;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Deque;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.camel.component.ai.observability.GenAiAttributes;
+import org.apache.camel.dsl.jbang.core.commands.LlmClient;
+import org.apache.camel.util.json.JsonArray;
+import org.apache.camel.util.json.JsonObject;
+import org.apache.camel.util.json.Jsoner;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Collects what the Ollama tab shows: the Ollama server and its loaded models over the REST API, the live state of the
+ * llama-server runner Ollama spawns on this machine, the load on the host, and a log of requests with the timings
+ * Ollama reports per request.
+ * <p/>
+ * {@link #poll()} is called from the TUI's background refresh while the tab is showing and throttles each source on its
+ * own interval. Requests arrive from two sides: the TUI's own AI panel calls {@link #recordRequest} with the usage of
+ * each Ollama reply, and calls made by Camel routes come in through {@link #ingestSpans} from the GenAI observability
+ * spans of the selected integration. The runner and host probes only run when the Ollama host is this machine; a remote
+ * or containerised Ollama still gets the API and per-request data.
+ * <p/>
+ * All I/O happens outside the lock; the {@code update*} methods apply results under it and are also what the tests
+ * feed.
+ */
+final class OllamaMonitor {
+
+ private static final Logger LOG = LoggerFactory.getLogger(OllamaMonitor.class);
+
+ static final int MAX_REQUESTS = 200;
+ static final int HISTORY_POINTS = 120;
+
+ private static final long DETECT_INTERVAL_MS = 10_000;
+ private static final long VERSION_INTERVAL_MS = 30_000;
+ private static final long RECONNECT_INTERVAL_MS = 5_000;
+ private static final long PS_INTERVAL_MS = 1_000;
+ private static final long TAGS_INTERVAL_MS = 15_000;
+ private static final long RUNNER_SCAN_INTERVAL_MS = 5_000;
+ private static final long HOST_INTERVAL_MS = 1_000;
+ private static final long RATE_WINDOW_MS = 1_500;
+ private static final int MAX_SEEN_SPANS = 4_000;
+ private static final long SPAN_INTERVAL_MS = 5_000;
+
+ // ---- data ----
+
+ record ServerInfo(String baseUrl, String version, boolean local) {
+ }
+
+ record ModelShape(String architecture, int layers, int experts, int expertsUsed, long maxContext,
+ int embeddingLength, long parameters, List<String> capabilities) {
+ }
+
+ record LoadedModel(String name, String family, String parameterSize, String quantization, long sizeBytes,
+ long sizeVram, long contextLength, Instant expiresAt, ModelShape shape) {
+
+ LoadedModel withShape(ModelShape newShape) {
+ return new LoadedModel(
+ name, family, parameterSize, quantization, sizeBytes, sizeVram, contextLength,
+ expiresAt, newShape);
+ }
+
+ /** Share of the model held in GPU memory; 100 means fully offloaded. */
+ int gpuPercent() {
+ if (sizeBytes <= 0) {
+ return 0;
+ }
+ return (int) Math.min(100, sizeVram * 100 / sizeBytes);
+ }
+ }
+
+ /** Folded state of the runner's slots; {@code decoded} counts tokens of the current or last request. */
+ record SlotState(boolean processing, long promptTokens, long promptProcessed, long cacheTokens, long decoded,
+ long contextSize, String speculative, int slots, Instant sampledAt) {
+
+ long contextUsed() {
+ return Math.max(promptTokens, cacheTokens) + decoded;
+ }
+
+ int cacheHitPercent() {
+ if (promptTokens <= 0) {
+ return 0;
+ }
+ return (int) Math.min(100, cacheTokens * 100 / promptTokens);
+ }
+ }
+
+ record RunnerInfo(long pid, int port, long contextSize, int parallel, String modelPath, String executable) {
+ }
+
+ record GpuStats(String name, int utilizationPercent, long memoryUsedBytes, long memoryTotalBytes, int count) {
+ }
+
+ record ProcessStats(long pid, String label, double cpuPercent, long rssBytes) {
+ }
+
+ record HostStats(GpuStats gpu, ProcessStats server, ProcessStats runner, Instant sampledAt) {
+ }
+
+ enum RequestSource {
+ TUI,
+ ROUTE
+ }
+
+ /** One request as Ollama reported it (TUI) or as the GenAI span of a route recorded it (no phase timings). */
+ record RequestEntry(Instant timestamp, RequestSource source, String routeId, String model, int inputTokens,
+ int outputTokens, int cachedTokens, long prefillMs, long decodeMs, long loadMs, long totalMs,
+ String doneReason) {
+
+ double prefillTokensPerSecond() {
+ return prefillMs > 0 ? inputTokens * 1000.0 / prefillMs : 0;
+ }
+
+ double decodeTokensPerSecond() {
+ return decodeMs > 0 ? outputTokens * 1000.0 / decodeMs : 0;
+ }
+
+ /** Time to first token: loading the model plus processing the prompt. Zero when timings are unknown. */
+ long ttftMs() {
+ return hasTimings() ? loadMs + prefillMs : 0;
+ }
+
+ boolean hasTimings() {
+ return prefillMs > 0 || decodeMs > 0;
+ }
+
+ /** A load of a second or more means the model was not in memory when the request arrived. */
+ boolean coldStart() {
+ return loadMs >= 1000;
+ }
+ }
+
+ record SessionTotals(int requests, long inputTokens, long outputTokens, long cachedTokens, long prefillMs,
+ long decodeMs, long loadMs, int coldStarts) {
+
+ static final SessionTotals EMPTY = new SessionTotals(0, 0, 0, 0, 0, 0, 0, 0);
+
+ SessionTotals plus(RequestEntry e) {
+ return new SessionTotals(
+ requests + 1, inputTokens + e.inputTokens(), outputTokens + e.outputTokens(),
+ cachedTokens + e.cachedTokens(), prefillMs + e.prefillMs(), decodeMs + e.decodeMs(),
+ loadMs + e.loadMs(), coldStarts + (e.coldStart() ? 1 : 0));
+ }
+
+ double avgDecodeTokensPerSecond() {
+ return decodeMs > 0 ? outputTokens * 1000.0 / decodeMs : 0;
+ }
+
+ double avgPrefillTokensPerSecond() {
+ return prefillMs > 0 ? inputTokens * 1000.0 / prefillMs : 0;
+ }
+ }
+
+ /** Immutable view for rendering and for the MCP tool. */
+ record Snapshot(ServerInfo server, List<LoadedModel> models, List<String> installed, SlotState slot,
+ RunnerInfo runner, HostStats host, List<RequestEntry> requests, double liveDecodeRate,
+ double livePrefillRate, long[] decodeHistory, SessionTotals totals, String lastError, Instant lastPoll,
+ String probedUrl) {
+
+ boolean connected() {
+ return server != null;
+ }
+
+ boolean local() {
+ return server != null && server.local();
+ }
+
+ RequestEntry lastRequest() {
+ return requests.isEmpty() ? null : requests.get(0);
+ }
+ }
+
+ // ---- state ----
+
+ private final Object lock = new Object();
+ private final AtomicBoolean polling = new AtomicBoolean();
+ private final HttpClient http = HttpClient.newBuilder().connectTimeout(Duration.ofMillis(1500)).build();
+
+ private volatile String baseUrl;
+ private ServerInfo server;
+ private List<LoadedModel> models = List.of();
+ private List<String> installed = List.of();
+ private final Map<String, ModelShape> shapes = new HashMap<>();
+ private final Set<String> shapeAttempted = new HashSet<>();
+ private SlotState slot;
+ private RunnerInfo runner;
+ private long serverPid;
+ private HostStats host;
+ private final Deque<RequestEntry> requests = new ArrayDeque<>();
+ private final LinkedHashSet<String> seenSpanIds = new LinkedHashSet<>();
+ private SessionTotals totals = SessionTotals.EMPTY;
+ private final TokenRateWindow decodeWindow = new TokenRateWindow(RATE_WINDOW_MS);
+ private final TokenRateWindow prefillWindow = new TokenRateWindow(RATE_WINDOW_MS);
+ private final long[] decodeHistory = new long[HISTORY_POINTS];
+ private String lastError;
+ private Instant lastPoll;
+
+ private long lastDetect;
+ private long lastVersion;
+ private long lastPs;
+ private long lastTags;
+ private long lastRunnerScan;
+ private long lastHost;
+ private int psFailures;
+ private long lastSpanIngest;
+ private Boolean nvidiaSmiAvailable;
+ private final Map<Long, long[]> cpuSamples = new HashMap<>();
+
+ // ---- input from the rest of the TUI ----
+
+ /** Uses the Ollama endpoint another part of the TUI already resolved (the AI panel's client, an explicit URL). */
+ void adoptEndpoint(String url) {
+ if (url == null || url.isBlank()) {
+ return;
+ }
+ String normalized = url.endsWith("/") ? url.substring(0, url.length() - 1) : url;
+ if (!normalized.equals(baseUrl)) {
+ baseUrl = normalized;
+ synchronized (lock) {
+ server = null;
+ lastVersion = 0;
+ runner = null;
+ slot = null;
+ }
+ }
+ }
+
+ /** Records a request the TUI itself made to Ollama, with the usage and timings Ollama returned. */
+ void recordRequest(String model, LlmClient.TokenUsage usage, long latencyMs, String doneReason) {
+ if (usage == null) {
+ return;
+ }
+ long total = usage.totalMillis() > 0 ? usage.totalMillis() : Math.max(0, latencyMs);
+ RequestEntry entry = new RequestEntry(
+ Instant.now(), RequestSource.TUI, null, model != null ? model : "unknown",
+ usage.inputTokens(), usage.outputTokens(), usage.cachedTokens(),
+ usage.prefillMillis(), usage.generationMillis(), usage.loadMillis(), total, doneReason);
+ synchronized (lock) {
+ addRequest(entry);
+ }
+ }
+
+ /**
+ * Adds the Ollama calls Camel routes made, from the GenAI observability spans of the selected integration. Spans
+ * are deduplicated by id, so the same list can be handed over on every refresh.
+ */
+ void ingestSpans(List<SpanEntry> spans) {
+ if (spans == null || spans.isEmpty()) {
+ return;
+ }
+ List<RequestEntry> fresh = new ArrayList<>();
+ synchronized (lock) {
+ for (SpanEntry span : spans) {
+ if (!GenAiSpanUsageExtractor.isGenAiSpan(span) || span.spanId() == null) {
+ continue;
+ }
+ Object system = span.attributes().get(GenAiAttributes.SYSTEM);
+ if (system == null || !system.toString().toLowerCase(Locale.ROOT).contains("ollama")) {
+ continue;
+ }
+ if (!seenSpanIds.add(span.spanId())) {
+ continue;
+ }
+ Map<String, Object> attrs = span.attributes();
+ String model = OllamaParsers.str(attrs, GenAiAttributes.RESPONSE_MODEL);
+ if (model == null) {
+ model = OllamaParsers.str(attrs, GenAiAttributes.REQUEST_MODEL);
+ }
+ if (model == null) {
+ model = span.name() != null ? span.name() : "unknown";
+ }
+ Instant ts = span.startEpochNanos() > 0 ? Instant.ofEpochSecond(0, span.startEpochNanos()) : Instant.now();
+ fresh.add(new RequestEntry(
+ ts, RequestSource.ROUTE, span.routeId(), model,
+ (int) OllamaParsers.num(attrs, GenAiAttributes.INPUT_TOKENS),
+ (int) OllamaParsers.num(attrs, GenAiAttributes.OUTPUT_TOKENS),
+ 0, 0, 0, 0, Math.max(0, span.durationMs()),
+ OllamaParsers.str(attrs, GenAiAttributes.FINISH_REASONS)));
+ }
+ // spans arrive oldest first; keep the log newest first
+ fresh.sort((a, b) -> a.timestamp().compareTo(b.timestamp()));
+ for (RequestEntry e : fresh) {
+ addRequest(e);
+ }
+ while (seenSpanIds.size() > MAX_SEEN_SPANS) {
+ seenSpanIds.remove(seenSpanIds.iterator().next());
+ }
+ }
+ }
+
+ /** True once every few seconds: the GenAI spans of a route are worth re-reading. */
+ boolean wantsSpans() {
+ long now = System.currentTimeMillis();
+ if (now - lastSpanIngest >= SPAN_INTERVAL_MS) {
+ lastSpanIngest = now;
+ return true;
+ }
+ return false;
+ }
+
+ /** Clears the request log, the session totals and the rate history. */
+ void reset() {
+ synchronized (lock) {
+ requests.clear();
+ totals = SessionTotals.EMPTY;
+ Arrays.fill(decodeHistory, 0);
+ decodeWindow.clear();
+ prefillWindow.clear();
+ }
+ }
+
+ private void addRequest(RequestEntry entry) {
+ requests.addFirst(entry);
+ while (requests.size() > MAX_REQUESTS) {
+ requests.removeLast();
+ }
+ totals = totals.plus(entry);
+ }
+
+ // ---- state updates (also the test seam) ----
+
+ void updateServer(ServerInfo info) {
+ synchronized (lock) {
+ server = info;
+ if (info != null) {
+ baseUrl = info.baseUrl();
+ lastError = null;
+ }
+ }
+ }
+
+ void updateModels(List<LoadedModel> loaded) {
+ synchronized (lock) {
+ models = loaded != null ? List.copyOf(loaded) : List.of();
+ }
+ }
+
+ void updateInstalled(List<String> names) {
+ synchronized (lock) {
+ installed = names != null ? List.copyOf(names) : List.of();
+ }
+ }
+
+ void updateRunner(RunnerInfo info) {
+ synchronized (lock) {
+ runner = info;
+ if (info == null) {
+ slot = null;
+ }
+ }
+ }
+
+ /** Applies a runner slot sample; the decode and prefill rate windows advance from its counters. */
+ void updateSlot(SlotState state) {
+ synchronized (lock) {
+ slot = state;
+ if (state != null) {
+ long now = state.sampledAt() != null ? state.sampledAt().toEpochMilli() : System.currentTimeMillis();
+ decodeWindow.sample(now, state.decoded());
+ prefillWindow.sample(now, state.promptProcessed());
+ }
+ }
+ }
+
+ void updateHost(HostStats stats) {
+ synchronized (lock) {
+ host = stats;
+ }
+ }
+
+ /** Ends one poll: appends the current live decode rate to the history and stamps the poll time. */
+ void tick(long nowMillis) {
+ synchronized (lock) {
+ long rate = Math.round(decodeWindow.ratePerSecond(nowMillis));
+ System.arraycopy(decodeHistory, 1, decodeHistory, 0, decodeHistory.length - 1);
+ decodeHistory[decodeHistory.length - 1] = rate;
+ lastPoll = Instant.ofEpochMilli(nowMillis);
+ }
+ }
+
+ Snapshot snapshot() {
+ synchronized (lock) {
+ long now = System.currentTimeMillis();
+ return new Snapshot(
+ server, models, installed, slot, runner, host, List.copyOf(requests),
+ decodeWindow.ratePerSecond(now), prefillWindow.ratePerSecond(now),
+ decodeHistory.clone(), totals, lastError, lastPoll, baseUrl);
+ }
+ }
+
+ // ---- polling ----
+
+ /** One refresh cycle; safe to call every few hundred milliseconds, each source keeps its own interval. */
+ void poll() {
+ if (!polling.compareAndSet(false, true)) {
+ return;
+ }
+ try {
+ doPoll(System.currentTimeMillis());
+ } catch (Exception e) {
+ LOG.debug("Ollama poll failed", e);
+ } finally {
+ polling.set(false);
+ }
+ }
+
+ private void doPoll(long now) {
+ String base = baseUrl;
+ if (base == null) {
+ if (now - lastDetect >= DETECT_INTERVAL_MS) {
+ lastDetect = now;
+ base = detectEndpoint();
+ if (base != null) {
+ adoptEndpoint(base);
+ base = baseUrl;
+ }
+ }
+ if (base == null) {
+ tick(now);
+ return;
+ }
+ }
+
+ boolean connected;
+ synchronized (lock) {
+ connected = server != null;
+ }
+ long versionInterval = connected ? VERSION_INTERVAL_MS : RECONNECT_INTERVAL_MS;
+ if (now - lastVersion >= versionInterval) {
+ lastVersion = now;
+ JsonObject version = getJsonObject(base + "/api/version");
+ if (version == null) {
+ synchronized (lock) {
+ server = null;
+ models = List.of();
+ slot = null;
+ lastError = "Ollama not reachable at " + OllamaParsers.displayHost(base);
+ }
+ tick(now);
+ return;
+ }
+ updateServer(new ServerInfo(base, OllamaParsers.str(version, "version"), OllamaParsers.isLoopbackUrl(base)));
+ connected = true;
+ }
+ if (!connected) {
+ tick(now);
+ return;
+ }
+
+ if (now - lastPs >= PS_INTERVAL_MS) {
+ lastPs = now;
+ JsonObject ps = getJsonObject(base + "/api/ps");
+ if (ps != null) {
+ psFailures = 0;
+ List<LoadedModel> loaded = new ArrayList<>();
+ for (LoadedModel m : OllamaParsers.parsePs(ps)) {
+ loaded.add(m.withShape(shapeFor(base, m.name())));
+ }
+ updateModels(loaded);
+ } else if (++psFailures >= 3) {
+ synchronized (lock) {
+ server = null;
+ lastError = "Ollama stopped answering at " + OllamaParsers.displayHost(base);
+ }
+ lastVersion = 0;
+ tick(now);
+ return;
+ }
+ }
+
+ if (now - lastTags >= TAGS_INTERVAL_MS) {
+ lastTags = now;
+ JsonObject tags = getJsonObject(base + "/api/tags");
+ if (tags != null) {
+ List<String> names = new ArrayList<>();
+ Collection<?> list = tags.getCollection("models");
+ if (list != null) {
+ for (Object o : list) {
+ if (o instanceof Map<?, ?> m) {
+ String n = OllamaParsers.str(m, "name");
+ if (n != null) {
+ names.add(n);
+ }
+ }
+ }
+ }
+ updateInstalled(names);
+ }
+ }
+
+ if (OllamaParsers.isLoopbackUrl(base)) {
+ pollLocal(now);
+ }
+ tick(now);
+ }
+
+ private void pollLocal(long now) {
+ RunnerInfo current;
+ boolean haveModels;
+ synchronized (lock) {
+ current = runner;
+ haveModels = !models.isEmpty();
+ }
+ // scan often while a model is loaded but no runner is known yet, rarely otherwise
+ long scanInterval = current == null && haveModels ? PS_INTERVAL_MS : RUNNER_SCAN_INTERVAL_MS;
+ if (now - lastRunnerScan >= scanInterval) {
+ lastRunnerScan = now;
+ RunnerInfo found = findRunner();
+ if (found == null || current == null || found.pid() != current.pid() || found.port() != current.port()) {
+ updateRunner(found);
+ current = found;
+ }
+ }
+ if (current != null) {
+ JsonArray slots = getJsonArray("http://127.0.0.1:" + current.port() + "/slots");
+ if (slots != null) {
+ updateSlot(OllamaParsers.parseSlots(slots, Instant.ofEpochMilli(now)));
+ } else {
+ updateRunner(null);
+ current = null;
+ }
+ }
+ if (now - lastHost >= HOST_INTERVAL_MS) {
+ lastHost = now;
+ updateHost(probeHost(now, current));
+ }
+ }
+
+ private ModelShape shapeFor(String base, String name) {
+ synchronized (lock) {
+ ModelShape cached = shapes.get(name);
+ if (cached != null || shapeAttempted.contains(name)) {
+ return cached;
+ }
+ shapeAttempted.add(name);
+ }
+ JsonObject body = new JsonObject();
+ body.put("model", name);
+ JsonObject show = postJsonObject(base + "/api/show", body.toJson());
+ ModelShape shape = OllamaParsers.parseShow(show);
+ if (shape != null) {
+ synchronized (lock) {
+ shapes.put(name, shape);
+ }
+ }
+ return shape;
+ }
+
+ private String detectEndpoint() {
+ try {
+ LlmClient client = LlmClient.create().withApiType(LlmClient.ApiType.ollama);
+ if (client.detectEndpoint()) {
+ return client.endpointUrl();
+ }
+ } catch (Exception e) {
+ LOG.debug("Ollama endpoint detection failed", e);
+ }
+ return null;
+ }
+
+ // ---- runner and host probes ----
+
+ private RunnerInfo findRunner() {
+ try {
+ RunnerInfo[] found = new RunnerInfo[1];
+ long[] parent = new long[1];
+ ProcessHandle.allProcesses().forEach(ph -> {
+ if (found[0] != null) {
+ return;
+ }
+ Optional<String> cmd = ph.info().commandLine();
+ if (cmd.isPresent() && OllamaParsers.isRunnerCommandLine(cmd.get())) {
+ RunnerInfo ri = OllamaParsers.parseRunnerCommandLine(ph.pid(), cmd.get());
+ if (ri != null) {
+ found[0] = ri;
+ parent[0] = ph.parent().map(ProcessHandle::pid).orElse(0L);
+ }
+ }
+ });
+ if (found[0] != null) {
+ serverPid = parent[0];
+ } else {
+ serverPid = findServerPid();
+ }
+ return found[0];
+ } catch (Exception e) {
+ LOG.debug("Ollama runner scan failed", e);
+ return null;
+ }
+ }
+
+ private long findServerPid() {
+ try {
+ return ProcessHandle.allProcesses()
+ .filter(ph -> ph.info().commandLine().map(c -> c.trim().endsWith("ollama serve")).orElse(false))
+ .mapToLong(ProcessHandle::pid)
+ .findFirst().orElse(0L);
+ } catch (Exception e) {
+ return 0L;
+ }
+ }
+
+ private HostStats probeHost(long now, RunnerInfo current) {
+ GpuStats gpu = probeGpu();
+ ProcessStats serverStats = serverPid > 0 ? processStats(serverPid, "ollama serve", now) : null;
+ ProcessStats runnerStats = current != null ? processStats(current.pid(), current.executable(), now) : null;
+ if (gpu == null && serverStats == null && runnerStats == null) {
+ return null;
+ }
+ return new HostStats(gpu, serverStats, runnerStats, Instant.ofEpochMilli(now));
+ }
+
+ private GpuStats probeGpu() {
+ String os = System.getProperty("os.name", "").toLowerCase(Locale.ROOT);
+ if (os.contains("mac")) {
+ return OllamaParsers.parseIoreg(runCommand(List.of("ioreg", "-r", "-d", "1", "-c", "IOAccelerator")));
+ }
+ if (Boolean.FALSE.equals(nvidiaSmiAvailable)) {
+ return null;
+ }
+ String out = runCommand(List.of("nvidia-smi", "--query-gpu=name,utilization.gpu,memory.used,memory.total",
+ "--format=csv,noheader,nounits"));
+ if (out == null) {
+ nvidiaSmiAvailable = Boolean.FALSE;
+ return null;
+ }
+ nvidiaSmiAvailable = Boolean.TRUE;
+ return OllamaParsers.parseNvidiaSmi(out);
+ }
+
+ /**
+ * CPU percent from the growth of the process's CPU time between two polls, plus resident memory. Read through
+ * {@code ps} on macOS and Linux (the JDK exposes no CPU time for other processes on macOS), from
+ * {@link ProcessHandle} elsewhere.
+ */
+ private ProcessStats processStats(long pid, String label, long now) {
+ Optional<ProcessHandle> handle = ProcessHandle.of(pid);
+ if (handle.isEmpty() || !handle.get().isAlive()) {
+ cpuSamples.remove(pid);
+ return null;
+ }
+ long cpuMs = -1;
+ long rss = 0;
+ String os = System.getProperty("os.name", "").toLowerCase(Locale.ROOT);
+ if (!os.contains("win")) {
+ long[] ps = OllamaParsers.parsePsCpuAndRss(
+ runCommand(List.of("ps", "-o", "cputime=,rss=", "-p", Long.toString(pid))));
+ if (ps != null) {
+ cpuMs = ps[0];
+ rss = ps[1];
+ }
+ }
+ if (cpuMs < 0) {
+ cpuMs = handle.get().info().totalCpuDuration().map(Duration::toMillis).orElse(-1L);
+ }
+ double cpu = 0;
+ if (cpuMs >= 0) {
+ long[] prev = cpuSamples.put(pid, new long[] { now, cpuMs });
+ if (prev != null && now > prev[0]) {
+ cpu = (cpuMs - prev[1]) * 100.0 / (now - prev[0]);
+ }
+ }
+ return new ProcessStats(pid, label, Math.max(0, cpu), rss);
+ }
+
+ // ---- I/O helpers ----
+
+ private JsonObject getJsonObject(String url) {
+ Object parsed = getJson(url, null);
+ return parsed instanceof JsonObject jo ? jo : null;
+ }
+
+ private JsonArray getJsonArray(String url) {
+ Object parsed = getJson(url, null);
+ return parsed instanceof JsonArray ja ? ja : null;
+ }
+
+ private JsonObject postJsonObject(String url, String body) {
+ Object parsed = getJson(url, body);
+ return parsed instanceof JsonObject jo ? jo : null;
+ }
+
+ private Object getJson(String url, String postBody) {
+ try {
+ HttpRequest.Builder b = HttpRequest.newBuilder(URI.create(url)).timeout(Duration.ofMillis(2500));
+ if (postBody != null) {
+ b.header("Content-Type", "application/json").POST(HttpRequest.BodyPublishers.ofString(postBody));
+ } else {
+ b.GET();
+ }
+ HttpResponse<String> response = http.send(b.build(), HttpResponse.BodyHandlers.ofString());
+ if (response.statusCode() / 100 != 2 || response.body() == null || response.body().isBlank()) {
+ return null;
+ }
+ return Jsoner.deserialize(response.body());
+ } catch (Exception e) {
+ return null;
+ }
+ }
+
+ private static String runCommand(List<String> command) {
+ Process process = null;
+ try {
+ process = new ProcessBuilder(command).redirectErrorStream(true).start();
+ byte[] out;
+ try (InputStream in = process.getInputStream()) {
+ out = in.readAllBytes();
+ }
+ if (!process.waitFor(2, TimeUnit.SECONDS)) {
+ process.destroyForcibly();
+ return null;
+ }
+ if (process.exitValue() != 0) {
+ return null;
+ }
+ return new String(out, StandardCharsets.UTF_8);
+ } catch (Exception e) {
+ if (process != null) {
+ process.destroyForcibly();
+ }
+ return null;
+ }
+ }
+
+ // ---- JSON for the MCP tool ----
+
+ JsonObject toJson(int requestLimit) {
+ Snapshot s = snapshot();
+ JsonObject root = new JsonObject();
+ root.put("connected", s.connected());
+ if (s.server() != null) {
+ JsonObject server = new JsonObject();
+ server.put("url", s.server().baseUrl());
+ server.put("host", OllamaParsers.displayHost(s.server().baseUrl()));
+ server.put("version", s.server().version());
+ server.put("local", s.server().local());
+ root.put("server", server);
+ } else if (s.probedUrl() != null) {
+ root.put("url", s.probedUrl());
+ }
+ if (s.lastError() != null) {
+ root.put("error", s.lastError());
+ }
+ JsonArray models = new JsonArray();
+ for (LoadedModel m : s.models()) {
+ JsonObject jm = new JsonObject();
+ jm.put("name", m.name());
+ jm.put("family", m.family());
+ jm.put("parameterSize", m.parameterSize());
+ jm.put("quantization", m.quantization());
+ jm.put("sizeBytes", m.sizeBytes());
+ jm.put("sizeVramBytes", m.sizeVram());
+ jm.put("gpuPercent", m.gpuPercent());
+ jm.put("contextLength", m.contextLength());
+ if (m.expiresAt() != null) {
+ jm.put("expiresAt", m.expiresAt().toString());
+ }
+ if (m.shape() != null) {
+ JsonObject shape = new JsonObject();
+ shape.put("architecture", m.shape().architecture());
+ shape.put("layers", m.shape().layers());
+ shape.put("experts", m.shape().experts());
+ shape.put("expertsUsed", m.shape().expertsUsed());
+ shape.put("maxContext", m.shape().maxContext());
+ shape.put("parameters", m.shape().parameters());
+ shape.put("capabilities", new JsonArray(m.shape().capabilities()));
+ jm.put("shape", shape);
+ }
+ models.add(jm);
+ }
+ root.put("loadedModels", models);
+ root.put("installedModels", new JsonArray(s.installed()));
+
+ JsonObject live = new JsonObject();
+ live.put("decodeTokensPerSecond", round1(s.liveDecodeRate()));
+ live.put("prefillTokensPerSecond", round1(s.livePrefillRate()));
+ JsonArray hist = new JsonArray();
+ for (long v : s.decodeHistory()) {
+ hist.add(v);
+ }
+ live.put("decodeHistory", hist);
+ if (s.slot() != null) {
+ JsonObject slot = new JsonObject();
+ slot.put("processing", s.slot().processing());
+ slot.put("promptTokens", s.slot().promptTokens());
+ slot.put("promptTokensProcessed", s.slot().promptProcessed());
+ slot.put("cacheTokens", s.slot().cacheTokens());
+ slot.put("decodedTokens", s.slot().decoded());
+ slot.put("contextUsed", s.slot().contextUsed());
+ slot.put("contextSize", s.slot().contextSize());
+ slot.put("cacheHitPercent", s.slot().cacheHitPercent());
+ slot.put("speculative", s.slot().speculative());
+ slot.put("slots", s.slot().slots());
+ live.put("slot", slot);
+ }
+ root.put("live", live);
+
+ if (s.runner() != null) {
+ JsonObject runner = new JsonObject();
+ runner.put("pid", s.runner().pid());
+ runner.put("port", s.runner().port());
+ runner.put("executable", s.runner().executable());
+ runner.put("contextSize", s.runner().contextSize());
+ runner.put("parallel", s.runner().parallel());
+ root.put("runner", runner);
+ }
+ if (s.host() != null) {
+ JsonObject hostJson = new JsonObject();
+ if (s.host().gpu() != null) {
+ JsonObject gpu = new JsonObject();
+ gpu.put("name", s.host().gpu().name());
+ gpu.put("utilizationPercent", s.host().gpu().utilizationPercent());
+ gpu.put("memoryUsedBytes", s.host().gpu().memoryUsedBytes());
+ gpu.put("memoryTotalBytes", s.host().gpu().memoryTotalBytes());
+ gpu.put("count", s.host().gpu().count());
+ hostJson.put("gpu", gpu);
+ }
+ if (s.host().server() != null) {
+ hostJson.put("server", processJson(s.host().server()));
+ }
+ if (s.host().runner() != null) {
+ hostJson.put("runner", processJson(s.host().runner()));
+ }
+ root.put("host", hostJson);
+ }
+
+ JsonObject session = new JsonObject();
+ session.put("requests", s.totals().requests());
+ session.put("inputTokens", s.totals().inputTokens());
+ session.put("outputTokens", s.totals().outputTokens());
+ session.put("cachedTokens", s.totals().cachedTokens());
+ session.put("avgDecodeTokensPerSecond", round1(s.totals().avgDecodeTokensPerSecond()));
+ session.put("avgPrefillTokensPerSecond", round1(s.totals().avgPrefillTokensPerSecond()));
+ session.put("coldStarts", s.totals().coldStarts());
+ root.put("session", session);
+
+ JsonArray reqs = new JsonArray();
+ int n = 0;
+ for (RequestEntry e : s.requests()) {
+ if (n++ >= requestLimit) {
+ break;
+ }
+ reqs.add(requestJson(e));
+ }
+ root.put("requests", reqs);
+ if (s.lastPoll() != null) {
+ root.put("lastPoll", s.lastPoll().toString());
+ }
+ return root;
+ }
+
+ static JsonObject requestJson(RequestEntry e) {
+ JsonObject r = new JsonObject();
+ r.put("time", e.timestamp().toString());
+ r.put("source", e.source().name().toLowerCase(Locale.ROOT));
+ if (e.routeId() != null) {
+ r.put("routeId", e.routeId());
+ }
+ r.put("model", e.model());
+ r.put("inputTokens", e.inputTokens());
+ r.put("outputTokens", e.outputTokens());
+ r.put("cachedTokens", e.cachedTokens());
+ r.put("prefillMs", e.prefillMs());
+ r.put("decodeMs", e.decodeMs());
+ r.put("loadMs", e.loadMs());
+ r.put("totalMs", e.totalMs());
+ r.put("prefillTokensPerSecond", round1(e.prefillTokensPerSecond()));
+ r.put("decodeTokensPerSecond", round1(e.decodeTokensPerSecond()));
+ r.put("ttftMs", e.ttftMs());
+ r.put("coldStart", e.coldStart());
+ if (e.doneReason() != null) {
+ r.put("doneReason", e.doneReason());
+ }
+ return r;
+ }
+
+ private static JsonObject processJson(ProcessStats p) {
+ JsonObject j = new JsonObject();
+ j.put("pid", p.pid());
+ j.put("label", p.label());
+ j.put("cpuPercent", round1(p.cpuPercent()));
+ j.put("rssBytes", p.rssBytes());
+ return j;
+ }
+
+ private static double round1(double v) {
+ return Math.round(v * 10.0) / 10.0;
+ }
+}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaParsers.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaParsers.java
new file mode 100644
index 000000000000..878a1e09820b
--- /dev/null
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaParsers.java
@@ -0,0 +1,439 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.tui;
+
+import java.net.URI;
+import java.time.Instant;
+import java.time.OffsetDateTime;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.GpuStats;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.LoadedModel;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.ModelShape;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RunnerInfo;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.SlotState;
+import org.apache.camel.util.json.JsonArray;
+import org.apache.camel.util.json.JsonObject;
+
+/**
+ * Pure parsers for what the Ollama tab reads: the Ollama REST API ({@code /api/ps}, {@code /api/show}), the
+ * llama-server runner ({@code /slots}, its command line) and the host probes ({@code ioreg} on macOS,
+ * {@code nvidia-smi} on Linux, {@code ps}). Kept free of I/O so each format has a unit test against a captured sample.
+ */
+final class OllamaParsers {
+
+ private static final Pattern IOREG_UTIL = Pattern.compile("\"Device Utilization %\"\\s*=\\s*(\\d+)");
+ private static final Pattern IOREG_MEM_IN_USE = Pattern.compile("\"In use system memory\"\\s*=\\s*(\\d+)");
+ private static final Pattern IOREG_MEM_ALLOC = Pattern.compile("\"Alloc system memory\"\\s*=\\s*(\\d+)");
+
+ private OllamaParsers() {
+ }
+
+ // ---- Ollama REST API ----
+
+ /** Models currently loaded, from {@code GET /api/ps}. */
+ static List<LoadedModel> parsePs(JsonObject json) {
+ List<LoadedModel> result = new ArrayList<>();
+ if (json == null) {
+ return result;
+ }
+ Collection<?> models = json.getCollection("models");
+ if (models == null) {
+ return result;
+ }
+ for (Object o : models) {
+ if (!(o instanceof Map<?, ?> m)) {
+ continue;
+ }
+ String name = str(m, "name");
+ if (name == null) {
+ name = str(m, "model");
+ }
+ if (name == null) {
+ continue;
+ }
+ String family = null;
+ String parameterSize = null;
+ String quantization = null;
+ if (m.get("details") instanceof Map<?, ?> details) {
+ family = str(details, "family");
+ parameterSize = str(details, "parameter_size");
+ quantization = str(details, "quantization_level");
+ }
+ result.add(new LoadedModel(
+ name, family, parameterSize, quantization,
+ num(m, "size"), num(m, "size_vram"), num(m, "context_length"),
+ parseInstant(str(m, "expires_at")), null));
+ }
+ return result;
+ }
+
+ /** Architecture facts of one model, from {@code POST /api/show}. */
+ static ModelShape parseShow(JsonObject json) {
+ if (json == null) {
+ return null;
+ }
+ String architecture = null;
+ long layers = 0;
+ long experts = 0;
+ long expertsUsed = 0;
+ long maxContext = 0;
+ long embedding = 0;
+ long parameters = 0;
+ if (json.get("model_info") instanceof Map<?, ?> info) {
+ architecture = str(info, "general.architecture");
+ parameters = num(info, "general.parameter_count");
+ for (Map.Entry<?, ?> e : info.entrySet()) {
+ String key = String.valueOf(e.getKey());
+ if (key.endsWith(".block_count")) {
+ layers = toLong(e.getValue());
+ } else if (key.endsWith(".expert_count")) {
+ experts = toLong(e.getValue());
+ } else if (key.endsWith(".expert_used_count")) {
+ expertsUsed = toLong(e.getValue());
+ } else if (key.endsWith(".context_length")) {
+ maxContext = toLong(e.getValue());
+ } else if (key.endsWith(".embedding_length")) {
+ embedding = toLong(e.getValue());
+ }
+ }
+ }
+ if (architecture == null && json.get("details") instanceof Map<?, ?> details) {
+ architecture = str(details, "family");
+ }
+ List<String> capabilities = new ArrayList<>();
+ Collection<?> caps = json.getCollection("capabilities");
+ if (caps != null) {
+ for (Object c : caps) {
+ capabilities.add(String.valueOf(c));
+ }
+ }
+ return new ModelShape(
+ architecture, (int) layers, (int) experts, (int) expertsUsed, maxContext,
+ (int) embedding, parameters, List.copyOf(capabilities));
+ }
+
+ // ---- llama-server runner ----
+
+ /**
+ * Live slot state from the runner's {@code GET /slots}. Several slots (parallel requests) are folded into one:
+ * processing when any slot is, token counts summed, the context size of the largest slot.
+ */
+ static SlotState parseSlots(JsonArray slots, Instant now) {
+ if (slots == null || slots.isEmpty()) {
+ return null;
+ }
+ boolean processing = false;
+ long promptTokens = 0;
+ long promptProcessed = 0;
+ long cacheTokens = 0;
+ long decoded = 0;
+ long contextSize = 0;
+ String speculative = null;
+ int slotCount = 0;
+ for (Object o : slots) {
+ if (!(o instanceof Map<?, ?> slot)) {
+ continue;
+ }
+ slotCount++;
+ processing |= Boolean.TRUE.equals(slot.get("is_processing"));
+ promptTokens += num(slot, "n_prompt_tokens");
+ promptProcessed += num(slot, "n_prompt_tokens_processed");
+ cacheTokens += num(slot, "n_prompt_tokens_cache");
+ contextSize = Math.max(contextSize, num(slot, "n_ctx"));
+ Object next = slot.get("next_token");
+ if (next instanceof Collection<?> c && !c.isEmpty() && c.iterator().next() instanceof Map<?, ?> nt) {
+ decoded += num(nt, "n_decoded");
+ } else if (next instanceof Map<?, ?> nt) {
+ decoded += num(nt, "n_decoded");
+ }
+ if (speculative == null && slot.get("params") instanceof Map<?, ?> params) {
+ String types = str(params, "speculative.types");
+ if (types != null && !types.isBlank() && !"none".equals(types)) {
+ // "none,draft-mtp" lists the fallback first; the active method is the last entry
+ String[] parts = types.split(",");
+ speculative = parts[parts.length - 1].trim();
+ }
+ }
+ }
+ return new SlotState(
+ processing, promptTokens, promptProcessed, cacheTokens, decoded, contextSize, speculative,
+ slotCount, now);
+ }
+
+ /** True when the command line is an Ollama model runner (upstream llama-server or the older ollama runner). */
+ static boolean isRunnerCommandLine(String commandLine) {
+ if (commandLine == null) {
+ return false;
+ }
+ String lower = commandLine.toLowerCase(Locale.ROOT);
+ if (!lower.contains("--port")) {
+ return false;
+ }
+ return lower.contains("llama-server") || (lower.contains("ollama") && lower.contains(" runner"));
+ }
+
+ /**
+ * Port, context size, parallel slots and model path from a runner command line such as
+ * {@code llama-server --model /blobs/sha256-... --port 58237 --host 127.0.0.1 -c 262144 -np 1}.
+ */
+ static RunnerInfo parseRunnerCommandLine(long pid, String commandLine) {
+ if (!isRunnerCommandLine(commandLine)) {
+ return null;
+ }
+ String[] tokens = commandLine.trim().split("\\s+");
+ int port = 0;
+ long ctx = 0;
+ int parallel = 0;
+ String modelPath = null;
+ for (int i = 0; i < tokens.length - 1; i++) {
+ String t = tokens[i];
+ String v = tokens[i + 1];
+ switch (t) {
+ case "--port" -> port = parseIntOrZero(v);
+ case "-c", "--ctx-size" -> ctx = parseLongOrZero(v);
+ case "-np", "--parallel" -> parallel = parseIntOrZero(v);
+ case "--model", "-m" -> modelPath = v;
+ default -> {
+ }
+ }
+ }
+ if (port <= 0) {
+ return null;
+ }
+ String executable = tokens[0];
+ int slash = Math.max(executable.lastIndexOf('/'), executable.lastIndexOf('\\'));
+ if (slash >= 0) {
+ executable = executable.substring(slash + 1);
+ }
+ return new RunnerInfo(pid, port, ctx, parallel, modelPath, executable);
+ }
+
+ // ---- host probes ----
+
+ /**
+ * GPU load on macOS from {@code ioreg -r -d 1 -c IOAccelerator}. Apple silicon reports a device utilization
+ * percentage and the unified memory the GPU has wired; several accelerator entries are folded into the busiest one
+ * for utilization and the sum for memory.
+ */
+ static GpuStats parseIoreg(String output) {
+ if (output == null || output.isBlank()) {
+ return null;
+ }
+ int util = -1;
+ Matcher m = IOREG_UTIL.matcher(output);
+ while (m.find()) {
+ util = Math.max(util, parseIntOrZero(m.group(1)));
+ }
+ long inUse = 0;
+ Matcher mem = IOREG_MEM_IN_USE.matcher(output);
+ while (mem.find()) {
+ inUse += parseLongOrZero(mem.group(1));
+ }
+ long alloc = 0;
+ Matcher am = IOREG_MEM_ALLOC.matcher(output);
+ while (am.find()) {
+ alloc += parseLongOrZero(am.group(1));
+ }
+ if (util < 0 && inUse == 0) {
+ return null;
+ }
+ return new GpuStats("Apple GPU", Math.max(0, util), inUse, Math.max(alloc, inUse), 1);
+ }
+
+ /**
+ * GPU load from {@code nvidia-smi --query-gpu=name,utilization.gpu,memory.used,memory.total
+ * --format=csv,noheader,nounits}; memory columns are MiB. Several GPUs are folded into the mean utilization and the
+ * summed memory.
+ */
+ static GpuStats parseNvidiaSmi(String output) {
+ if (output == null || output.isBlank()) {
+ return null;
+ }
+ String name = null;
+ int count = 0;
+ long utilSum = 0;
+ long used = 0;
+ long total = 0;
+ for (String line : output.split("\\R")) {
+ String[] cols = line.split(",");
+ if (cols.length < 4) {
+ continue;
+ }
+ if (name == null) {
+ name = cols[0].trim();
+ }
+ count++;
+ utilSum += parseLongOrZero(cols[1].trim());
+ used += parseLongOrZero(cols[2].trim()) * 1024L * 1024L;
+ total += parseLongOrZero(cols[3].trim()) * 1024L * 1024L;
+ }
+ if (count == 0) {
+ return null;
+ }
+ return new GpuStats(name, (int) (utilSum / count), used, total, count);
+ }
+
+ /**
+ * Accumulated CPU time and resident memory from {@code ps -o cputime=,rss= -p <pid>}, as {@code [cpuMillis,
+ * rssBytes]}. The JDK reports no CPU time for other users' processes on macOS, so the tab diffs this between polls.
+ */
+ static long[] parsePsCpuAndRss(String psOutput) {
+ if (psOutput == null) {
+ return null;
+ }
+ String[] cols = psOutput.trim().split("\\s+");
+ if (cols.length < 2) {
+ return null;
+ }
+ long cpu = parseCpuTimeMillis(cols[0]);
+ long rss = parseLongOrZero(cols[1]) * 1024L;
+ if (cpu < 0) {
+ return null;
+ }
+ return new long[] { cpu, rss };
+ }
+
+ /** {@code ps} CPU time: {@code mm:ss.cc}, {@code hh:mm:ss} or {@code d-hh:mm:ss}; -1 when unreadable. */
+ static long parseCpuTimeMillis(String text) {
+ if (text == null || text.isBlank()) {
+ return -1;
+ }
+ String t = text.trim();
+ long days = 0;
+ int dash = t.indexOf('-');
+ if (dash > 0) {
+ days = parseLongOrZero(t.substring(0, dash));
+ t = t.substring(dash + 1);
+ }
+ String[] parts = t.split(":");
+ double seconds;
+ try {
+ if (parts.length == 3) {
+ seconds = Long.parseLong(parts[0]) * 3600.0 + Long.parseLong(parts[1]) * 60.0
+ + Double.parseDouble(parts[2]);
+ } else if (parts.length == 2) {
+ seconds = Long.parseLong(parts[0]) * 60.0 + Double.parseDouble(parts[1]);
+ } else if (parts.length == 1) {
+ seconds = Double.parseDouble(parts[0]);
+ } else {
+ return -1;
+ }
+ } catch (NumberFormatException e) {
+ return -1;
+ }
+ return Math.round((days * 86400.0 + seconds) * 1000.0);
+ }
+
+ // ---- misc ----
+
+ /** Ollama timestamps carry a zone offset ({@code 2026-09-17T11:22:44.593036+02:00}) or a trailing Z. */
+ static Instant parseInstant(String text) {
+ if (text == null || text.isBlank()) {
+ return null;
+ }
+ try {
+ return OffsetDateTime.parse(text).toInstant();
+ } catch (Exception e) {
+ try {
+ return Instant.parse(text);
+ } catch (Exception ignored) {
+ return null;
+ }
+ }
+ }
+
+ /** True when the URL points at this machine, which is when the runner and host probes make sense. */
+ static boolean isLoopbackUrl(String url) {
+ if (url == null) {
+ return false;
+ }
+ try {
+ String host = URI.create(url).getHost();
+ return host != null && (host.equalsIgnoreCase("localhost") || host.equals("127.0.0.1")
+ || host.equals("::1") || host.equals("[::1]") || host.equals("0.0.0.0"));
+ } catch (IllegalArgumentException e) {
+ return false;
+ }
+ }
+
+ /** {@code http://localhost:11434} becomes {@code localhost:11434}. */
+ static String displayHost(String url) {
+ if (url == null) {
+ return "";
+ }
+ String host = url;
+ if (host.startsWith("http://")) {
+ host = host.substring("http://".length());
+ } else if (host.startsWith("https://")) {
+ host = host.substring("https://".length());
+ }
+ while (host.endsWith("/")) {
+ host = host.substring(0, host.length() - 1);
+ }
+ return host;
+ }
+
+ static String str(Map<?, ?> m, String key) {
+ Object v = m.get(key);
+ if (v == null) {
+ return null;
+ }
+ String s = v.toString();
+ return s.isBlank() ? null : s;
+ }
+
+ static long num(Map<?, ?> m, String key) {
+ return toLong(m.get(key));
+ }
+
+ static long toLong(Object v) {
+ if (v instanceof Number n) {
+ return n.longValue();
+ }
+ if (v instanceof String s) {
+ return parseLongOrZero(s.trim());
+ }
+ return 0;
+ }
+
+ private static int parseIntOrZero(String s) {
+ try {
+ return Integer.parseInt(s);
+ } catch (NumberFormatException e) {
+ return 0;
+ }
+ }
+
+ private static long parseLongOrZero(String s) {
+ try {
+ return Long.parseLong(s);
+ } catch (NumberFormatException e) {
+ try {
+ return (long) Double.parseDouble(s);
+ } catch (NumberFormatException e2) {
+ return 0;
+ }
+ }
+ }
+}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
new file mode 100644
index 000000000000..967233487f11
--- /dev/null
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
@@ -0,0 +1,692 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.tui;
+
+import java.time.Duration;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Locale;
+
+import dev.tamboui.layout.Constraint;
+import dev.tamboui.layout.Layout;
+import dev.tamboui.layout.Rect;
+import dev.tamboui.style.Style;
+import dev.tamboui.terminal.Frame;
+import dev.tamboui.text.Line;
+import dev.tamboui.text.Span;
+import dev.tamboui.text.Text;
+import dev.tamboui.tui.event.KeyCode;
+import dev.tamboui.tui.event.KeyEvent;
+import dev.tamboui.tui.event.MouseEvent;
+import dev.tamboui.widgets.block.Block;
+import dev.tamboui.widgets.block.BorderType;
+import dev.tamboui.widgets.block.Borders;
+import dev.tamboui.widgets.paragraph.Paragraph;
+import dev.tamboui.widgets.scrollbar.ScrollbarState;
+import dev.tamboui.widgets.table.Cell;
+import dev.tamboui.widgets.table.Row;
+import dev.tamboui.widgets.table.Table;
+import dev.tamboui.widgets.table.TableState;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.HostStats;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.LoadedModel;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.ModelShape;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RequestEntry;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RequestSource;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.SessionTotals;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.SlotState;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.Snapshot;
+import org.apache.camel.util.json.JsonArray;
+import org.apache.camel.util.json.JsonObject;
+
+import static org.apache.camel.dsl.jbang.core.commands.tui.TuiHelper.hint;
+import static org.apache.camel.dsl.jbang.core.commands.tui.TuiHelper.hintLast;
+
+/**
+ * Shows how the model served by Ollama is performing: the loaded model and its shape, decode and prefill tokens per
+ * second, context and cache usage, the load on this host, and a log of the requests made by the TUI's AI panel and by
+ * Camel routes. The data comes from {@link OllamaMonitor}.
+ */
+class OllamaTab extends AbstractTab {
+
+ private static final DateTimeFormatter TIME = DateTimeFormatter.ofPattern("HH:mm:ss").withZone(ZoneId.systemDefault());
+ private static final String[] SPARK = { " ", "▁", "▂", "▃", "▄", "▅", "▆", "▇", "█" };
+ private static final String GAUGE_FILLED = "▓";
+ private static final String GAUGE_EMPTY = "░";
+ private static final int PANELS_HEIGHT = 8;
+ private static final int MAX_HEADER_MODELS = 3;
+
+ private final OllamaMonitor monitor;
+ private final TableState tableState = new TableState();
+ private final ScrollbarState scrollState = new ScrollbarState();
+ private Rect lastTableArea;
+
+ OllamaTab(MonitorContext ctx, OllamaMonitor monitor) {
+ super(ctx);
+ this.monitor = monitor;
+ }
+
+ @Override
+ public String description() {
+ return "Ollama server and model performance: loaded models, tokens per second, context and cache usage, "
+ + "host GPU and process load, and the request log";
+ }
+
+ @Override
+ public void onTabSelected() {
+ if (monitor != null) {
+ ctx.backgroundExecutor.execute(monitor::poll);
+ }
+ }
+
+ // ---- input ----
+
+ @Override
+ public boolean handleKeyEvent(KeyEvent ke) {
+ if (ke.isUp()) {
+ navigateUp();
+ return true;
+ }
+ if (ke.isDown()) {
+ navigateDown();
+ return true;
+ }
+ if (ke.isChar('r') && monitor != null) {
+ monitor.reset();
+ tableState.select(0);
+ return true;
+ }
+ if (ke.isKey(KeyCode.F5) && monitor != null) {
+ ctx.backgroundExecutor.execute(monitor::poll);
+ return true;
+ }
+ return false;
+ }
+
+ @Override
+ public void navigateUp() {
+ tableState.selectPrevious();
+ }
+
+ @Override
+ public void navigateDown() {
+ int rows = monitor != null ? monitor.snapshot().requests().size() : 0;
+ if (rows > 0) {
+ tableState.selectNext(rows);
+ }
+ }
+
+ @Override
+ public boolean handleMouseEvent(MouseEvent me, Rect area) {
+ int rows = monitor != null ? monitor.snapshot().requests().size() : 0;
+ return handleTableClick(me, lastTableArea, tableState, rows);
+ }
+
+ @Override
+ public void renderFooter(List<Span> spans) {
+ hint(spans, TuiIcons.ARROW_UP + TuiIcons.ARROW_DOWN, "select");
+ hint(spans, "r", "reset");
+ hintLast(spans, "F5", "refresh");
+ }
+
+ // ---- rendering ----
+
+ @Override
+ public void render(Frame frame, Rect area) {
+ Snapshot s = monitor != null ? monitor.snapshot() : null;
+ if (s == null || !s.connected()) {
+ renderNotConnected(frame, area, s);
+ return;
+ }
+ int modelLines = Math.max(1, Math.min(MAX_HEADER_MODELS, s.models().size())) * 2;
+ List<Rect> rows = Layout.vertical()
+ .constraints(Constraint.length(modelLines + 2), Constraint.length(PANELS_HEIGHT), Constraint.fill())
+ .split(area);
+ renderHeader(frame, rows.get(0), s);
+ renderPanels(frame, rows.get(1), s);
+ renderRequests(frame, rows.get(2), s);
+ }
+
+ private void renderNotConnected(Frame frame, Rect area, Snapshot s) {
+ List<Line> lines = new ArrayList<>();
+ lines.add(Line.from(Span.raw("")));
+ String where = s != null && s.probedUrl() != null ? OllamaParsers.displayHost(s.probedUrl()) : "localhost:11434";
+ lines.add(Line.from(Span.styled(" Ollama not detected at " + where, Theme.warning().bold())));
+ if (s != null && s.lastError() != null) {
+ lines.add(Line.from(Span.styled(" " + s.lastError(), Theme.muted())));
+ }
+ lines.add(Line.from(Span.raw("")));
+ lines.add(Line.from(Span.styled(" Start it with ", Theme.muted()), Span.styled("ollama serve", Theme.label()),
+ Span.styled(" or ", Theme.muted()), Span.styled("camel infra run ollama", Theme.label()),
+ Span.styled(", or point the AI panel at a remote Ollama.", Theme.muted())));
+ lines.add(Line.from(Span.styled(" The tab checks again every few seconds.", Theme.muted())));
+ frame.renderWidget(Paragraph.builder()
+ .text(Text.from(lines))
+ .block(Block.builder().borderType(BorderType.ROUNDED).borders(Borders.ALL).title(" Ollama ").build())
+ .build(), area);
+ }
+
+ private void renderHeader(Frame frame, Rect area, Snapshot s) {
+ StringBuilder title = new StringBuilder(" Ollama");
+ if (s.server().version() != null) {
+ title.append(' ').append(s.server().version());
+ }
+ title.append(" · ").append(OllamaParsers.displayHost(s.server().baseUrl()));
+ if (s.runner() != null) {
+ title.append(" · runner ").append(s.runner().executable()).append(" :").append(s.runner().port());
+ } else if (!s.local()) {
+ title.append(" · remote");
+ }
+ title.append(' ');
+
+ List<Line> lines = new ArrayList<>();
+ if (s.models().isEmpty()) {
+ lines.add(Line.from(Span.styled(" No model loaded", Theme.warning().bold())));
+ String installed = s.installed().isEmpty()
+ ? "no models pulled"
+ : s.installed().size() + " installed: " + String.join(", ", s.installed().subList(0,
+ Math.min(3, s.installed().size())))
+ + (s.installed().size() > 3 ? ", ..." : "");
+ lines.add(Line.from(Span.styled(" " + installed
+ + " · a model loads on the first request and unloads after its keep-alive",
+ Theme.muted())));
+ } else {
+ for (int i = 0; i < Math.min(MAX_HEADER_MODELS, s.models().size()); i++) {
+ LoadedModel m = s.models().get(i);
+ lines.add(Line.from(Span.styled(" " + m.name(), Style.EMPTY.fg(Theme.accent()).bold()),
+ Span.styled(" " + describeShape(m), Theme.muted())));
+ lines.add(Line.from(Span.styled(" " + describeResidency(m, Instant.now()), Theme.label())));
+ }
+ }
+ frame.renderWidget(Paragraph.builder()
+ .text(Text.from(lines))
+ .block(Block.builder().borderType(BorderType.ROUNDED).borders(Borders.ALL)
+ .title(title.toString()).build())
+ .build(), area);
+ }
+
+ private void renderPanels(Frame frame, Rect area, Snapshot s) {
+ List<Rect> cols;
+ if (s.local()) {
+ cols = Layout.horizontal()
+ .constraints(Constraint.percentage(40), Constraint.percentage(30), Constraint.fill())
+ .split(area);
+ } else {
+ cols = Layout.horizontal()
+ .constraints(Constraint.percentage(55), Constraint.fill())
+ .split(area);
+ }
+ renderThroughput(frame, cols.get(0), s);
+ renderContext(frame, cols.get(1), s);
+ if (s.local()) {
+ renderHost(frame, cols.get(2), s);
+ }
+ }
+
+ private void renderThroughput(Frame frame, Rect area, Snapshot s) {
+ RequestEntry last = s.lastRequest();
+ boolean live = s.slot() != null && (s.slot().processing() || s.liveDecodeRate() > 0);
+ double decode = live ? s.liveDecodeRate() : last != null ? last.decodeTokensPerSecond() : 0;
+ double prefill = live ? s.livePrefillRate() : last != null ? last.prefillTokensPerSecond() : 0;
+ String tag = live ? "live" : last != null && last.hasTimings() ? "last request" : "";
+ Style big = Theme.success().bold();
+
+ List<Line> lines = new ArrayList<>();
+ lines.add(Line.from(
+ Span.styled(" decode ", Theme.muted()),
+ Span.styled(padLeft(decode > 0 ? formatRate(decode) : "-", 6), big),
+ Span.styled(" tok/s", Theme.muted()),
+ Span.styled(tag.isEmpty() ? "" : " " + tag, Theme.muted().dim())));
+ lines.add(Line.from(
+ Span.styled(" prefill ", Theme.muted()),
+ Span.styled(padLeft(prefill > 0 ? formatRate(prefill) : "-", 6), Theme.info().bold()),
+ Span.styled(" tok/s", Theme.muted())));
+ if (last != null && last.hasTimings()) {
+ lines.add(Line.from(
+ Span.styled(" TTFT ", Theme.muted()),
+ Span.styled(formatSeconds(last.ttftMs()), Theme.label()),
+ Span.styled(" · load ", Theme.muted()),
+ Span.styled(formatSeconds(last.loadMs()), last.coldStart() ? Theme.warning() : Theme.label()),
+ Span.styled(last.coldStart() ? " cold" : " warm", Theme.muted())));
+ } else {
+ lines.add(Line.from(Span.styled(" TTFT - · load -", Theme.muted())));
+ }
+ SessionTotals t = s.totals();
+ if (t.requests() > 0) {
+ lines.add(Line.from(
+ Span.styled(" session ", Theme.muted()),
+ Span.styled(t.avgDecodeTokensPerSecond() > 0
+ ? formatRate(t.avgDecodeTokensPerSecond()) + " tok/s"
+ : "-", Theme.label()),
+ Span.styled(" · in " + formatTokens(t.inputTokens()) + " · out " + formatTokens(t.outputTokens())
+ + " · " + t.requests() + (t.requests() == 1 ? " request" : " requests"),
+ Theme.muted())));
+ } else {
+ lines.add(Line.from(Span.styled(" session no requests yet", Theme.muted())));
+ }
+
+ Block block = Block.builder().borderType(BorderType.ROUNDED).borders(Borders.ALL).title(" Throughput ").build();
+ frame.renderWidget(Paragraph.builder().text(Text.from(lines)).block(block).build(), area);
+
+ // sparkline on the remaining rows: live decode history with the runner, otherwise one bar per request
+ Rect inner = block.inner(area);
+ int sparkY = inner.y() + lines.size();
+ if (sparkY < inner.y() + inner.height() && inner.width() > 2) {
+ long[] data = s.runner() != null ? s.decodeHistory() : requestHistory(s.requests(), inner.width() - 2);
+ String spark = sparkline(data, inner.width() - 2);
+ if (!spark.isBlank()) {
+ frame.buffer().setString(inner.x() + 1, sparkY, spark, Theme.success());
+ }
+ int labelY = sparkY + 1;
+ if (labelY < inner.y() + inner.height()) {
+ String label = s.runner() != null
+ ? "decode tok/s, last " + (OllamaMonitor.HISTORY_POINTS / 2) + "s"
+ : "decode tok/s per request";
+ frame.buffer().setString(inner.x() + 1, labelY, label, Theme.muted().dim());
+ }
+ }
+ }
+
+ private void renderContext(Frame frame, Rect area, Snapshot s) {
+ List<Line> lines = new ArrayList<>();
+ SlotState slot = s.slot();
+ LoadedModel model = s.models().isEmpty() ? null : s.models().get(0);
+ Block block = Block.builder().borderType(BorderType.ROUNDED).borders(Borders.ALL).title(" Context ").build();
+ int gaugeWidth = Math.max(8, Math.min(24, block.inner(area).width() - 22));
+ if (slot != null) {
+ long used = slot.contextUsed();
+ long size = slot.contextSize();
+ int pct = size > 0 ? (int) Math.min(100, used * 100 / size) : 0;
+ lines.add(Line.from(
+ Span.styled(" " + gaugeBar(pct, gaugeWidth) + " ", pct >= 90 ? Theme.error() : Theme.info()),
+ Span.styled(formatTokens(used) + " / " + formatTokens(size), Theme.label()),
+ Span.styled(" (" + pct + "%)", Theme.muted())));
+ lines.add(Line.from(Span.styled(" prompt " + formatTokens(slot.promptTokens()) + " · cache hit "
+ + slot.cacheHitPercent() + "%"
+ + (slot.slots() > 1 ? " · " + slot.slots() + " slots" : ""),
+ Theme.muted())));
+ List<Span> state = new ArrayList<>();
+ if (slot.processing()) {
+ state.add(Span.styled(" " + TuiIcons.SELECTED + " working", Theme.success().bold()));
+ // still in prefill: nothing decoded yet and the prompt only partly processed
+ if (slot.decoded() == 0 && slot.promptProcessed() > 0 && slot.promptProcessed() < slot.promptTokens()) {
+ state.add(Span.styled(" prefill " + formatTokens(slot.promptProcessed()) + "/"
+ + formatTokens(slot.promptTokens()),
+ Theme.muted()));
+ }
+ } else {
+ state.add(Span.styled(" " + TuiIcons.IDLE + " idle", Theme.muted()));
+ }
+ if (slot.speculative() != null) {
+ state.add(Span.styled(" · speculative " + slot.speculative(), Theme.muted()));
+ }
+ lines.add(Line.from(state));
+ } else {
+ RequestEntry last = s.lastRequest();
+ long ctx = model != null ? model.contextLength() : 0;
+ if (last != null && ctx > 0) {
+ long used = (long) last.inputTokens() + last.outputTokens() + last.cachedTokens();
+ int pct = (int) Math.min(100, used * 100 / ctx);
+ lines.add(Line.from(
+ Span.styled(" " + gaugeBar(pct, gaugeWidth) + " ", pct >= 90 ? Theme.error() : Theme.info()),
+ Span.styled(formatTokens(used) + " / " + formatTokens(ctx), Theme.label()),
+ Span.styled(" (" + pct + "%) last request", Theme.muted())));
+ lines.add(Line.from(Span.styled(" cached prompt " + formatTokens(last.cachedTokens()), Theme.muted())));
+ } else if (ctx > 0) {
+ lines.add(Line.from(Span.styled(" " + gaugeBar(0, gaugeWidth) + " ", Theme.info()),
+ Span.styled("0 / " + formatTokens(ctx), Theme.label())));
+ } else {
+ lines.add(Line.from(Span.styled(" no context data yet", Theme.muted())));
+ }
+ lines.add(Line.from(Span.styled(s.local()
+ ? " live state appears once the runner is found"
+ : " live state needs the runner on this machine", Theme.muted().dim())));
+ }
+ frame.renderWidget(Paragraph.builder().text(Text.from(lines)).block(block).build(), area);
+ }
+
+ private void renderHost(Frame frame, Rect area, Snapshot s) {
+ List<Line> lines = new ArrayList<>();
+ HostStats host = s.host();
+ Block block = Block.builder().borderType(BorderType.ROUNDED).borders(Borders.ALL).title(" Host ").build();
+ int gaugeWidth = Math.max(6, Math.min(16, block.inner(area).width() - 14));
+ if (host == null) {
+ lines.add(Line.from(Span.styled(" collecting...", Theme.muted())));
+ } else {
+ if (host.gpu() != null) {
+ int util = host.gpu().utilizationPercent();
+ Style gs = util >= 90 ? Theme.error() : util >= 60 ? Theme.warning() : Theme.success();
+ lines.add(Line.from(
+ Span.styled(" GPU ", Theme.muted()),
+ Span.styled(gaugeBar(util, gaugeWidth), gs),
+ Span.styled(" " + util + "%", gs.bold()),
+ Span.styled(host.gpu().count() > 1 ? " ×" + host.gpu().count() : "", Theme.muted())));
+ String mem = " GPU mem " + TuiHelper.formatBytes(host.gpu().memoryUsedBytes());
+ if (host.gpu().memoryTotalBytes() > host.gpu().memoryUsedBytes()) {
+ mem += " / " + TuiHelper.formatBytes(host.gpu().memoryTotalBytes());
+ }
+ lines.add(Line.from(Span.styled(mem, Theme.label())));
+ } else {
+ lines.add(Line.from(Span.styled(" GPU no reading on this host", Theme.muted())));
+ }
+ if (host.runner() != null) {
+ lines.add(processLine(host.runner().label(), host.runner().cpuPercent(), host.runner().rssBytes()));
+ }
+ if (host.server() != null) {
+ lines.add(processLine(host.server().label(), host.server().cpuPercent(), host.server().rssBytes()));
+ }
+ }
+ frame.renderWidget(Paragraph.builder().text(Text.from(lines)).block(block).build(), area);
+ }
+
+ private static Line processLine(String label, double cpu, long rss) {
+ String name = label != null ? label : "process";
+ if (name.length() > 13) {
+ name = name.substring(0, 13);
+ }
+ return Line.from(
+ Span.styled(" " + String.format(Locale.ROOT, "%-13s", name), Theme.muted()),
+ Span.styled(String.format(Locale.ROOT, "%5.1f%% cpu", cpu), Theme.label()),
+ Span.styled(rss > 0 ? " · " + TuiHelper.formatBytes(rss) : "", Theme.muted()));
+ }
+
+ private void renderRequests(Frame frame, Rect area, Snapshot s) {
+ List<RequestEntry> requests = s.requests();
+ List<Row> rows = new ArrayList<>();
+ for (RequestEntry e : requests) {
+ rows.add(Row.from(
+ Cell.from(Span.styled(" " + TIME.format(e.timestamp()), Theme.muted())),
+ Cell.from(Span.styled(sourceLabel(e), e.source() == RequestSource.ROUTE
+ ? Theme.notice()
+ : Theme.info())),
+ Cell.from(Span.raw(e.model())),
+ rightCell(formatTokens(e.inputTokens()), 6),
+ rightCell(formatTokens(e.outputTokens()), 6),
+ rightCell(e.cachedTokens() > 0 ? formatTokens(e.cachedTokens()) : "-", 6, Theme.muted()),
+ rightCell(e.prefillMs() > 0 ? formatRate(e.prefillTokensPerSecond()) : "-", 8, Theme.info()),
+ rightCell(e.decodeMs() > 0 ? formatRate(e.decodeTokensPerSecond()) : "-", 8, Theme.success()),
+ rightCell(e.hasTimings() ? formatSeconds(e.ttftMs()) : "-", 7,
+ e.coldStart() ? Theme.warning() : Style.EMPTY),
+ rightCell(e.totalMs() > 0 ? formatSeconds(e.totalMs()) : "-", 7),
+ Cell.from(Span.styled(" " + (e.doneReason() != null ? e.doneReason() : ""), Theme.muted()))));
+ }
+ if (rows.isEmpty()) {
+ lastTableArea = area;
+ frame.renderWidget(Paragraph.builder()
+ .text(Text.from(List.of(
+ Line.from(Span.styled(" No requests yet.", Theme.muted())),
+ Line.from(Span.styled(" Ask something in the AI panel (F8) with Ollama as the provider, "
+ + "or let a route call Ollama with GenAI observability on.",
+ Theme.muted())))))
+ .block(Block.builder().borderType(BorderType.ROUNDED).borders(Borders.ALL)
+ .title(" Requests (0) ").build())
+ .build(), area);
+ return;
+ }
+ Table table = Table.builder()
+ .rows(rows)
+ .header(Row.from(
+ Cell.from(Span.styled(" TIME", Style.EMPTY.bold())),
+ Cell.from(Span.styled("SOURCE", Style.EMPTY.bold())),
+ Cell.from(Span.styled("MODEL", Style.EMPTY.bold())),
+ rightCell("IN", 6, Style.EMPTY.bold()),
+ rightCell("OUT", 6, Style.EMPTY.bold()),
+ rightCell("CACHED", 6, Style.EMPTY.bold()),
+ rightCell("PREFILL", 8, Style.EMPTY.bold()),
+ rightCell("DECODE", 8, Style.EMPTY.bold()),
+ rightCell("TTFT", 7, Style.EMPTY.bold()),
+ rightCell("TOTAL", 7, Style.EMPTY.bold()),
+ Cell.from(Span.styled(" REASON", Style.EMPTY.bold()))))
+ .widths(
+ Constraint.length(9),
+ Constraint.length(18),
+ Constraint.fill(),
+ Constraint.length(6),
+ Constraint.length(6),
+ Constraint.length(6),
+ Constraint.length(8),
+ Constraint.length(8),
+ Constraint.length(7),
+ Constraint.length(7),
+ Constraint.length(12))
+ .highlightStyle(Theme.selectionBg())
+ .highlightSpacing(Table.HighlightSpacing.ALWAYS)
+ .block(Block.builder().borderType(BorderType.ROUNDED).borders(Borders.ALL)
+ .title(" Requests (" + requests.size() + ") tok/s for prefill and decode ").build())
+ .build();
+ lastTableArea = area;
+ frame.renderStatefulWidget(table, area, tableState);
+ renderTableScrollbar(frame, area, tableState, scrollState, rows.size());
+ }
+
+ // ---- MCP / table data ----
+
+ @Override
+ public JsonObject getTableDataAsJson() {
+ JsonObject result = new JsonObject();
+ result.put("tab", "Ollama");
+ JsonArray rows = new JsonArray();
+ int total = 0;
+ if (monitor != null) {
+ JsonObject full = monitor.toJson(MAX_ROWS_JSON);
+ Object reqs = full.remove("requests");
+ if (reqs instanceof JsonArray ja) {
+ rows = ja;
+ }
+ total = monitor.snapshot().requests().size();
+ result.put("summary", full);
+ }
+ result.put("rows", rows);
+ result.put("totalRows", total);
+ Integer sel = tableState.selected();
+ result.put("selectedIndex", sel != null ? sel : -1);
+ return result;
+ }
+
+ private static final int MAX_ROWS_JSON = 100;
+
+ @Override
+ public String getHelpText() {
+ return """
+ # Ollama
+
+ Performance of the model Ollama is serving, in the spirit of an LLM dashboard.
+
+ - **Header**: the loaded model with its family, parameters, quantization, layers, experts, \
+ how much of it sits in GPU memory, its context length and when it unloads.
+ - **Throughput**: decode and prefill tokens per second. *live* while the runner on this machine \
+ is generating, otherwise the figures of the last request; time to first token and load time \
+ (a load of a second or more means a cold start); session averages and a decode-rate history.
+ - **Context**: context window fill, cache hit ratio and whether the runner is working, from the \
+ runner's live state on this machine; the last request's tokens otherwise.
+ - **Host** (same machine only): GPU utilization and memory, CPU and memory of the runner and \
+ the Ollama server.
+ - **Requests**: every request the AI panel (F8) made through Ollama with Ollama's own timings, \
+ plus calls made by Camel routes when GenAI observability is on (tokens and duration only).
+
+ Keys: **↑/↓** select a request, **r** reset the log and session totals, **F5** refresh now.
+ """;
+ }
+
+ // ---- formatting helpers (package-private for tests) ----
+
+ static String describeShape(LoadedModel m) {
+ List<String> parts = new ArrayList<>();
+ ModelShape shape = m.shape();
+ String arch = shape != null && shape.architecture() != null ? shape.architecture() : m.family();
+ if (arch != null) {
+ parts.add(arch);
+ }
+ if (m.parameterSize() != null) {
+ parts.add(m.parameterSize());
+ }
+ if (m.quantization() != null) {
+ parts.add(m.quantization());
+ }
+ if (shape != null) {
+ if (shape.layers() > 0) {
+ parts.add(shape.layers() + " layers");
+ }
+ if (shape.experts() > 0) {
+ parts.add(shape.experts() + " experts"
+ + (shape.expertsUsed() > 0 ? " (" + shape.expertsUsed() + " active)" : ""));
+ }
+ if (shape.capabilities() != null && !shape.capabilities().isEmpty()) {
+ List<String> caps = new ArrayList<>();
+ for (String c : shape.capabilities()) {
+ if (!"completion".equals(c)) {
+ caps.add(c);
+ }
+ }
+ if (!caps.isEmpty()) {
+ parts.add(String.join(", ", caps));
+ }
+ }
+ }
+ return String.join(" · ", parts);
+ }
+
+ static String describeResidency(LoadedModel m, Instant now) {
+ StringBuilder sb = new StringBuilder();
+ sb.append(TuiHelper.formatBytes(m.sizeBytes())).append(" loaded");
+ if (m.sizeBytes() > 0) {
+ int gpu = m.gpuPercent();
+ sb.append(", ").append(gpu >= 100 ? "100% GPU" : gpu > 0 ? gpu + "% GPU" : "CPU only");
+ }
+ if (m.contextLength() > 0) {
+ sb.append(" · ctx ").append(String.format(Locale.ROOT, "%,d", m.contextLength()));
+ if (m.shape() != null && m.shape().maxContext() > m.contextLength()) {
+ sb.append(" of ").append(String.format(Locale.ROOT, "%,d", m.shape().maxContext()));
+ }
+ }
+ if (m.expiresAt() != null) {
+ sb.append(" · ").append(formatCountdown(m.expiresAt(), now));
+ }
+ return sb.toString();
+ }
+
+ static String formatCountdown(Instant expiresAt, Instant now) {
+ Duration left = Duration.between(now, expiresAt);
+ if (left.isNegative() || left.isZero()) {
+ return "unloading";
+ }
+ if (left.toDays() > 365) {
+ return "stays loaded";
+ }
+ long secs = left.toSeconds();
+ if (secs >= 3600) {
+ return "unloads in " + (secs / 3600) + "h" + ((secs % 3600) / 60) + "m";
+ }
+ if (secs >= 60) {
+ return "unloads in " + (secs / 60) + "m" + (secs % 60) + "s";
+ }
+ return "unloads in " + secs + "s";
+ }
+
+ static String sourceLabel(RequestEntry e) {
+ if (e.source() == RequestSource.ROUTE) {
+ return e.routeId() != null ? "route:" + e.routeId() : "route";
+ }
+ return "tui";
+ }
+
+ static String formatRate(double tokensPerSecond) {
+ if (tokensPerSecond <= 0) {
+ return "0";
+ }
+ if (tokensPerSecond < 10) {
+ return String.format(Locale.ROOT, "%.1f", tokensPerSecond);
+ }
+ return String.format(Locale.ROOT, "%.0f", tokensPerSecond);
+ }
+
+ static String formatTokens(long tokens) {
+ if (tokens < 1000) {
+ return Long.toString(tokens);
+ }
+ if (tokens < 10_000) {
+ return String.format(Locale.ROOT, "%.1fk", tokens / 1000.0);
+ }
+ if (tokens < 1_000_000) {
+ return (tokens / 1000) + "k";
+ }
+ return String.format(Locale.ROOT, "%.1fM", tokens / 1_000_000.0);
+ }
+
+ static String formatSeconds(long ms) {
+ if (ms <= 0) {
+ return "0s";
+ }
+ if (ms < 1000) {
+ return String.format(Locale.ROOT, "%.2fs", ms / 1000.0);
+ }
+ if (ms < 10_000) {
+ return String.format(Locale.ROOT, "%.1fs", ms / 1000.0);
+ }
+ if (ms < 60_000) {
+ return (ms / 1000) + "s";
+ }
+ return (ms / 60_000) + "m" + ((ms % 60_000) / 1000) + "s";
+ }
+
+ static String gaugeBar(int pct, int width) {
+ int p = Math.max(0, Math.min(100, pct));
+ int filled = p * width / 100;
+ return GAUGE_FILLED.repeat(filled) + GAUGE_EMPTY.repeat(Math.max(0, width - filled));
+ }
+
+ /** Right-aligned one-row sparkline scaled to the largest value; an all-zero series renders as spaces. */
+ static String sparkline(long[] data, int width) {
+ if (data == null || width <= 0) {
+ return "";
+ }
+ long max = 0;
+ for (long v : data) {
+ max = Math.max(max, v);
+ }
+ StringBuilder sb = new StringBuilder();
+ int start = Math.max(0, data.length - width);
+ for (int i = 0; i < width - Math.min(width, data.length); i++) {
+ sb.append(' ');
+ }
+ for (int i = start; i < data.length; i++) {
+ if (max <= 0 || data[i] <= 0) {
+ sb.append(' ');
+ } else {
+ int level = (int) Math.max(1, Math.round(data[i] * 8.0 / max));
+ sb.append(SPARK[Math.min(8, level)]);
+ }
+ }
+ return sb.toString();
+ }
+
+ /** Decode rate of the most recent requests, oldest first, for hosts without a runner to sample live. */
+ static long[] requestHistory(List<RequestEntry> newestFirst, int width) {
+ int n = Math.min(Math.max(0, width), newestFirst.size());
+ long[] data = new long[n];
+ for (int i = 0; i < n; i++) {
+ data[n - 1 - i] = Math.round(newestFirst.get(i).decodeTokensPerSecond());
+ }
+ return data;
+ }
+
+ private static String padLeft(String s, int width) {
+ return s.length() >= width ? s : " ".repeat(width - s.length()) + s;
+ }
+}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistry.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistry.java
index c81494078da6..b0f6f9555c97 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistry.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistry.java
@@ -109,6 +109,7 @@ class TabRegistry {
private TypeConvertersTab typeConvertersTab;
private TransformersTab transformersTab;
private SecretsTab secretsTab;
+ private OllamaTab ollamaTab;
private MonitorTab activeMoreTab;
private List<MoreTab> moreTabs;
@@ -163,6 +164,7 @@ class TabRegistry {
threadsTab = new ThreadsTab(ctx);
spansTab = new SpansTab(ctx, dataService.otelSpans());
processTab = new ProcessTab(ctx);
+ ollamaTab = new OllamaTab(ctx, ctx.ollamaMonitor);
overviewTab = new OverviewTab(
ctx, dataService.metrics(), dataService.stoppingPids(),
resetIntegrationTabState);
@@ -173,7 +175,7 @@ class TabRegistry {
});
// Single source of truth for the More submenu: icon, programmatic name, mnemonic label, tab instance, and group.
- // Tabs are ordered by group (Routing, Observability, Data, JVM, Project), alphabetically within each group.
+ // Tabs are ordered by group (Routing, Observability, AI, Data, JVM, Project), alphabetically within each group.
// The group field drives divider rendering in the More popup.
moreTabs = List.of(
// Routing
@@ -216,6 +218,7 @@ class TabRegistry {
TuiIcons.TAB_JFR, "JFR", "J&FR", jfrTab, "Observability",
List.of("jfr")),
// Data
+ new MoreTab(TuiIcons.TAB_OLLAMA, "Ollama", "&Ollama", ollamaTab, "AI"),
new MoreTab(
TuiIcons.TAB_DATASOURCE, "JDBC DataSource", "&JDBC DataSource", dataSourceTab, "Data",
List.of(), info -> !info.dataSources.isEmpty()),
@@ -430,6 +433,10 @@ class TabRegistry {
return spansTab;
}
+ OllamaTab ollamaTab() {
+ return ollamaTab;
+ }
+
OverviewTab overviewTab() {
return overviewTab;
}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TokenRateWindow.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TokenRateWindow.java
new file mode 100644
index 000000000000..bbe26720df61
--- /dev/null
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TokenRateWindow.java
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.tui;
+
+import java.util.ArrayDeque;
+import java.util.Deque;
+
+/**
+ * Turns a monotonically increasing token counter, sampled at irregular intervals, into a tokens-per-second rate over a
+ * sliding window. The window smooths the bursts a speculative decoder produces (several tokens land at once, then
+ * nothing for a step) while still dropping to zero within one window of the counter standing still.
+ * <p/>
+ * The counter resets when a new request starts (llama-server counts decoded tokens per request), so a sample below the
+ * previous one starts a fresh baseline instead of producing a negative rate.
+ */
+final class TokenRateWindow {
+
+ private final long windowMillis;
+ private final Deque<long[]> samples = new ArrayDeque<>();
+
+ TokenRateWindow(long windowMillis) {
+ this.windowMillis = Math.max(1, windowMillis);
+ }
+
+ /** Records the counter value observed at {@code nowMillis}. */
+ synchronized void sample(long nowMillis, long counter) {
+ long[] last = samples.peekLast();
+ if (last != null && counter < last[1]) {
+ samples.clear();
+ }
+ samples.addLast(new long[] { nowMillis, counter });
+ // keep exactly one sample at or before the window start as the baseline
+ long cutoff = nowMillis - windowMillis;
+ while (samples.size() > 2) {
+ long[] first = samples.peekFirst();
+ long[] second = samples.stream().skip(1).findFirst().orElse(first);
+ if (second[0] <= cutoff) {
+ samples.pollFirst();
+ } else {
+ break;
+ }
+ }
+ }
+
+ /** Tokens per second across the window, or zero when there is no movement or the last sample is stale. */
+ synchronized double ratePerSecond(long nowMillis) {
+ if (samples.size() < 2) {
+ return 0;
+ }
+ long[] first = samples.peekFirst();
+ long[] last = samples.peekLast();
+ if (nowMillis - last[0] > windowMillis) {
+ return 0;
+ }
+ long dt = last[0] - first[0];
+ long tokens = last[1] - first[1];
+ if (dt <= 0 || tokens <= 0) {
+ return 0;
+ }
+ return tokens * 1000.0 / dt;
+ }
+
+ synchronized void clear() {
+ samples.clear();
+ }
+}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiIcons.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiIcons.java
index c222298b545c..3dbfd01a570a 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiIcons.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiIcons.java
@@ -156,6 +156,7 @@ final class TuiIcons {
static final String TAB_MEMORY = MEMORY;
static final String TAB_MEMORY_LEAK = "💧";
static final String TAB_NETWORK = "🌐";
+ static final String TAB_OLLAMA = "🦙";
static final String TAB_METRICS = "📈";
static final String TAB_SQL_QUERY = KEY;
static final String TAB_SQL_TRACE = "🔎";
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiToolDefinitions.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiToolDefinitions.java
index 8972afc0e00c..f34f289e88e1 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiToolDefinitions.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiToolDefinitions.java
@@ -428,6 +428,15 @@ final class TuiToolDefinitions {
Map.of("directory", propDef("string",
"Absolute path to the project directory to open")),
List.of("directory"))));
+ tools.add(toToolDef(toolDef(
+ "tui_get_ollama",
+ "Performance of the Ollama server the TUI watches (same data as the Ollama tab, with or without "
+ + "a running integration): version and host, loaded models (size, GPU share, "
+ + "context length, unload time, architecture, layers, experts), live decode and "
+ + "prefill tokens per second and context fill from the local runner, GPU and "
+ + "process load on this host, session totals, and the recent requests with "
+ + "Ollama's per-request timings (prefill, decode, load, total, tokens, cache).",
+ Map.of("limit", propDef("integer", "Maximum number of requests to return (default 50)")))));
tools.add(toToolDef(toolDef(
"tui_get_spans",
"OpenTelemetry spans of the selected integration (traceId, spanId, parentSpanId, name, kind, "
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiToolRegistry.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiToolRegistry.java
index ea7f8f695cbd..4da3058ec206 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiToolRegistry.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiToolRegistry.java
@@ -173,7 +173,8 @@ class TuiToolRegistry {
/** The TUI's own read-only tools; the shared tools add those flagged read-only in the registry. */
private static final Set<String> READ_ONLY_TUI_TOOLS = Set.of(
"tui_get_ai_log", "tui_get_diagram", "tui_get_events", "tui_get_history", "tui_get_mcp_log",
- "tui_get_options", "tui_get_processor_detail", "tui_get_readme", "tui_get_screen", "tui_get_spans",
+ "tui_get_ollama", "tui_get_options", "tui_get_processor_detail", "tui_get_readme", "tui_get_screen",
+ "tui_get_spans",
"tui_get_state", "tui_get_status", "tui_get_table", "tui_get_themes", "tui_get_topology",
"tui_list_examples", "tui_locate", "tui_wait_for_idle");
@@ -248,6 +249,7 @@ class TuiToolRegistry {
case "tui_infra" -> callInfra(args);
case "tui_open_project" -> callOpenProject(args);
case "tui_get_spans" -> callGetSpans(args);
+ case "tui_get_ollama" -> callGetOllama(args);
case "tui_locate" -> callLocate(args);
case "tui_draw_shape" -> callDrawShape(args);
case "tui_canvas_open" -> callCanvasOpen(args);
@@ -1169,6 +1171,14 @@ class TuiToolRegistry {
return Jsoner.serialize(data);
}
+ private String callGetOllama(Map<String, Object> args) {
+ int limit = 50;
+ if (args.get("limit") instanceof Number n) {
+ limit = Math.max(1, n.intValue());
+ }
+ return Jsoner.serialize(facade.getOllamaData(limit));
+ }
+
private String callGetSpans(Map<String, Object> args) {
String traceId = args.get("traceId") instanceof String s ? s : null;
int limit = 500;
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
new file mode 100644
index 000000000000..cb2b325e32d9
--- /dev/null
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.tui;
+
+import java.time.Instant;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.camel.component.ai.observability.GenAiAttributes;
+import org.apache.camel.dsl.jbang.core.commands.LlmClient;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RequestEntry;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RequestSource;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.ServerInfo;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.SlotState;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.Snapshot;
+import org.apache.camel.util.json.JsonArray;
+import org.apache.camel.util.json.JsonObject;
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+class OllamaMonitorTest {
+
+ /** The usage Ollama returned for a real warm request: 16 prompt tokens in 188 ms, 6 tokens in 81 ms. */
+ private static final LlmClient.TokenUsage WARM = new LlmClient.TokenUsage(16, 6, 22, 0, 188, 81, 12, 300);
+ /** And a cold one: 11.1 s spent loading the model first. */
+ private static final LlmClient.TokenUsage COLD = new LlmClient.TokenUsage(18, 7, 25, 0, 138, 130, 11108, 11378);
+
+ @Test
+ void tuiRequestsCarryOllamaTimings() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ monitor.recordRequest("qwen3.6:35b-a3b", WARM, 350, "stop");
+
+ Snapshot s = monitor.snapshot();
+ assertEquals(1, s.requests().size());
+ RequestEntry e = s.lastRequest();
+ assertEquals(RequestSource.TUI, e.source());
+ assertEquals("qwen3.6:35b-a3b", e.model());
+ assertEquals(16, e.inputTokens());
+ assertEquals(6, e.outputTokens());
+ assertEquals(188, e.prefillMs());
+ assertEquals(81, e.decodeMs());
+ assertEquals(12, e.loadMs());
+ // Ollama's own total wins over the client-side latency
+ assertEquals(300, e.totalMs());
+ assertEquals(85.1, e.prefillTokensPerSecond(), 0.1);
+ assertEquals(74.1, e.decodeTokensPerSecond(), 0.1);
+ assertEquals(200, e.ttftMs());
+ assertFalse(e.coldStart());
+ assertEquals("stop", e.doneReason());
+ }
+
+ @Test
+ void coldStartIsDetectedFromLoadTime() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ monitor.recordRequest("qwen3.6:35b-a3b", COLD, 11400, "stop");
+ RequestEntry e = monitor.snapshot().lastRequest();
+ assertTrue(e.coldStart());
+ assertEquals(11108 + 138, e.ttftMs());
+ assertEquals(1, monitor.snapshot().totals().coldStarts());
+ }
+
+ @Test
+ void clientLatencyFillsInWhenOllamaReportsNoTotal() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ monitor.recordRequest("m", new LlmClient.TokenUsage(10, 5, 15), 1234, "stop");
+ assertEquals(1234, monitor.snapshot().lastRequest().totalMs());
+ assertFalse(monitor.snapshot().lastRequest().hasTimings());
+ }
+
+ @Test
+ void sessionTotalsAggregateAcrossRequests() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ monitor.recordRequest("m", WARM, 0, "stop");
+ monitor.recordRequest("m", COLD, 0, "stop");
+ var t = monitor.snapshot().totals();
+ assertEquals(2, t.requests());
+ assertEquals(34, t.inputTokens());
+ assertEquals(13, t.outputTokens());
+ assertEquals(188 + 138, t.prefillMs());
+ assertEquals(81 + 130, t.decodeMs());
+ assertEquals(1, t.coldStarts());
+ assertEquals(13 * 1000.0 / 211, t.avgDecodeTokensPerSecond(), 0.01);
+ }
+
+ @Test
+ void requestLogIsCappedNewestFirst() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ for (int i = 0; i < OllamaMonitor.MAX_REQUESTS + 5; i++) {
+ monitor.recordRequest("m" + i, WARM, 0, "stop");
+ }
+ List<RequestEntry> requests = monitor.snapshot().requests();
+ assertEquals(OllamaMonitor.MAX_REQUESTS, requests.size());
+ assertEquals("m" + (OllamaMonitor.MAX_REQUESTS + 4), requests.get(0).model());
+ // totals still count every request
+ assertEquals(OllamaMonitor.MAX_REQUESTS + 5, monitor.snapshot().totals().requests());
+ }
+
+ @Test
+ void routeSpansBecomeRouteRequestsOnlyForOllamaAndOnlyOnce() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ SpanEntry ollama = genAiSpan("s1", "ollama", "qwen3.6:35b-a3b", "chat-route", 412, 180, 4100);
+ SpanEntry openai = genAiSpan("s2", "openai", "gpt-4o", "other-route", 10, 10, 500);
+ SpanEntry plain = new SpanEntry("t", "s3", null, "camel-route", "INTERNAL", "OK", 0, 0, 5, "r", null, null, Map.of());
+
+ monitor.ingestSpans(List.of(ollama, openai, plain));
+ monitor.ingestSpans(List.of(ollama));
+
+ List<RequestEntry> requests = monitor.snapshot().requests();
+ assertEquals(1, requests.size());
+ RequestEntry e = requests.get(0);
+ assertEquals(RequestSource.ROUTE, e.source());
+ assertEquals("chat-route", e.routeId());
+ assertEquals("qwen3.6:35b-a3b", e.model());
+ assertEquals(412, e.inputTokens());
+ assertEquals(180, e.outputTokens());
+ assertEquals(4100, e.totalMs());
+ assertFalse(e.hasTimings());
+ assertEquals(0, e.ttftMs());
+ }
+
+ @Test
+ void slotSamplesDriveTheLiveDecodeRateAndHistory() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ long t0 = System.currentTimeMillis();
+ monitor.updateSlot(slot(true, 0, t0));
+ monitor.updateSlot(slot(true, 25, t0 + 500));
+ monitor.updateSlot(slot(true, 50, t0 + 1000));
+ monitor.tick(t0 + 1000);
+
+ Snapshot s = monitor.snapshot();
+ assertNotNull(s.slot());
+ assertTrue(s.slot().processing());
+ assertTrue(s.liveDecodeRate() > 0);
+ long[] history = s.decodeHistory();
+ assertEquals(OllamaMonitor.HISTORY_POINTS, history.length);
+ assertEquals(50, history[history.length - 1]);
+ assertEquals(0, history[0]);
+ }
+
+ @Test
+ void resetClearsRequestsTotalsAndHistoryButKeepsServer() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ monitor.updateServer(new ServerInfo("http://localhost:11434", "0.33.3", true));
+ monitor.recordRequest("m", WARM, 0, "stop");
+ monitor.reset();
+ Snapshot s = monitor.snapshot();
+ assertTrue(s.requests().isEmpty());
+ assertEquals(0, s.totals().requests());
+ assertTrue(s.connected());
+ assertEquals("0.33.3", s.server().version());
+ }
+
+ @Test
+ void adoptingADifferentEndpointForgetsTheOldServer() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ monitor.updateServer(new ServerInfo("http://localhost:11434", "0.33.3", true));
+ monitor.adoptEndpoint("http://gpu-box.lan:11434/");
+ Snapshot s = monitor.snapshot();
+ assertFalse(s.connected());
+ assertEquals("http://gpu-box.lan:11434", s.probedUrl());
+ // the same endpoint again changes nothing
+ monitor.updateServer(new ServerInfo("http://gpu-box.lan:11434", "0.33.3", false));
+ monitor.adoptEndpoint("http://gpu-box.lan:11434");
+ assertTrue(monitor.snapshot().connected());
+ }
+
+ @Test
+ void jsonCarriesServerModelsLiveSessionAndRequests() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ monitor.updateServer(new ServerInfo("http://localhost:11434", "0.33.3", true));
+ monitor.updateModels(List.of(new OllamaMonitor.LoadedModel(
+ "qwen3.6:35b-a3b", "qwen35moe", "35.5B", "Q4_K_M", 23567972432L, 23567972432L, 262144,
+ Instant.parse("2026-09-17T09:22:44Z"),
+ new OllamaMonitor.ModelShape(
+ "qwen35moe", 41, 256, 8, 262144, 2048, 35505251456L,
+ List.of("completion", "tools")))));
+ monitor.updateInstalled(List.of("qwen3.6:35b-a3b", "llama3.2:latest"));
+ monitor.updateRunner(new OllamaMonitor.RunnerInfo(23629, 58237, 262144, 1, "/blob", "llama-server"));
+ monitor.updateSlot(slot(false, 6, System.currentTimeMillis()));
+ monitor.updateHost(new OllamaMonitor.HostStats(
+ new OllamaMonitor.GpuStats("Apple GPU", 25, 31463079936L, 34170552320L, 1),
+ new OllamaMonitor.ProcessStats(70566, "ollama serve", 0.1, 82176L * 1024),
+ new OllamaMonitor.ProcessStats(23629, "llama-server", 12.5, 28273856L * 1024),
+ Instant.now()));
+ monitor.recordRequest("qwen3.6:35b-a3b", WARM, 0, "stop");
+ monitor.recordRequest("qwen3.6:35b-a3b", COLD, 0, "stop");
+
+ JsonObject json = monitor.toJson(1);
+ assertEquals(true, json.get("connected"));
+ JsonObject server = (JsonObject) json.get("server");
+ assertEquals("0.33.3", server.get("version"));
+ assertEquals("localhost:11434", server.get("host"));
+ assertEquals(true, server.get("local"));
+
+ JsonArray models = (JsonArray) json.get("loadedModels");
+ assertEquals(1, models.size());
+ JsonObject model = (JsonObject) models.get(0);
+ assertEquals("qwen3.6:35b-a3b", model.get("name"));
+ assertEquals(100, model.get("gpuPercent"));
+ assertEquals(41, ((JsonObject) model.get("shape")).get("layers"));
+ assertEquals(256, ((JsonObject) model.get("shape")).get("experts"));
+ assertEquals(2, ((JsonArray) json.get("installedModels")).size());
+
+ JsonObject live = (JsonObject) json.get("live");
+ assertNotNull(live.get("slot"));
+ assertEquals(OllamaMonitor.HISTORY_POINTS, ((JsonArray) live.get("decodeHistory")).size());
+ assertEquals(58237, ((JsonObject) json.get("runner")).get("port"));
+ JsonObject host = (JsonObject) json.get("host");
+ assertEquals(25, ((JsonObject) host.get("gpu")).get("utilizationPercent"));
+ assertEquals("llama-server", ((JsonObject) host.get("runner")).get("label"));
+
+ JsonObject session = (JsonObject) json.get("session");
+ assertEquals(2, session.get("requests"));
+ assertEquals(1, session.get("coldStarts"));
+ // limit applies to the request list only
+ JsonArray requests = (JsonArray) json.get("requests");
+ assertEquals(1, requests.size());
+ JsonObject last = (JsonObject) requests.get(0);
+ assertEquals(true, last.get("coldStart"));
+ assertEquals("tui", last.get("source"));
+ assertEquals(11108L + 138, last.get("ttftMs"));
+ }
+
+ @Test
+ void disconnectedJsonSaysSoWithoutFailing() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ JsonObject json = monitor.toJson(10);
+ assertEquals(false, json.get("connected"));
+ assertNull(json.get("server"));
+ assertEquals(0, ((JsonArray) json.get("requests")).size());
+ }
+
+ private static SlotState slot(boolean processing, long decoded, long atMillis) {
+ return new SlotState(processing, 25, 25, 9, decoded, 262144, "draft-mtp", 1, Instant.ofEpochMilli(atMillis));
+ }
+
+ private static SpanEntry genAiSpan(
+ String spanId, String system, String model, String routeId, int in, int out, long durationMs) {
+ Map<String, Object> attrs = new HashMap<>();
+ attrs.put(GenAiAttributes.OPERATION_NAME, "chat");
+ attrs.put(GenAiAttributes.SYSTEM, system);
+ attrs.put(GenAiAttributes.REQUEST_MODEL, model);
+ attrs.put(GenAiAttributes.INPUT_TOKENS, String.valueOf(in));
+ attrs.put(GenAiAttributes.OUTPUT_TOKENS, out);
+ attrs.put(GenAiAttributes.FINISH_REASONS, "stop");
+ long start = Instant.parse("2026-09-17T09:15:02Z").toEpochMilli() * 1_000_000L;
+ return new SpanEntry(
+ "trace-1", spanId, "parent", "chat " + model, "CLIENT", "OK", start,
+ start + durationMs * 1_000_000L, durationMs, routeId, null, null, attrs);
+ }
+}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaParsersTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaParsersTest.java
new file mode 100644
index 000000000000..583ec2a3e138
--- /dev/null
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaParsersTest.java
@@ -0,0 +1,222 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.tui;
+
+import java.time.Instant;
+import java.util.List;
+
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.GpuStats;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.LoadedModel;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.ModelShape;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RunnerInfo;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.SlotState;
+import org.apache.camel.util.json.JsonArray;
+import org.apache.camel.util.json.JsonObject;
+import org.apache.camel.util.json.Jsoner;
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * The samples below were captured from Ollama 0.33.3 serving qwen3.6:35b-a3b on an Apple M4 Pro, and from the
+ * llama-server runner it spawned.
+ */
+class OllamaParsersTest {
+
+ private static final String PS = """
+ {"models":[{"name":"qwen3.6:35b-a3b","model":"qwen3.6:35b-a3b","size":23567972432,
+ "digest":"096fdbd02fe6","details":{"parent_model":"","format":"gguf","family":"qwen35moe",
+ "families":["qwen35moe"],"parameter_size":"35.5B","quantization_level":"Q4_K_M"},
+ "expires_at":"2026-09-17T11:22:44.593036+02:00","size_vram":23567972432,"context_length":262144}]}
+ """;
+
+ private static final String SHOW = """
+ {"details":{"family":"qwen35moe","parameter_size":"35.5B","quantization_level":"Q4_K_M"},
+ "model_info":{"general.architecture":"qwen35moe","general.parameter_count":35505251456,
+ "qwen35moe.block_count":41,"qwen35moe.context_length":262144,"qwen35moe.embedding_length":2048,
+ "qwen35moe.expert_count":256,"qwen35moe.expert_used_count":8,"qwen35moe.attention.head_count":16},
+ "capabilities":["completion","vision","tools","thinking"]}
+ """;
+
+ private static final String SLOTS = """
+ [{"id":0,"n_ctx":262144,"speculative":true,"is_processing":true,"id_task":3,"n_prompt_tokens":25,
+ "n_prompt_tokens_processed":25,"n_prompt_tokens_cache":9,
+ "params":{"n_predict":16,"speculative.types":"none,draft-mtp"},
+ "next_token":[{"has_next_token":true,"n_remain":10,"n_decoded":6}]}]
+ """;
+
+ private static final String RUNNER_CMD = "/opt/homebrew/Cellar/ollama/0.33.3/libexec/lib/ollama/llama-server "
+ + "--model /Users/me/.ollama/models/blobs/sha256-d372de8e "
+ + "--port 58237 --host 127.0.0.1 --no-webui --offline -c 262144 -np 1 "
+ + "--log-verbosity 4 --chat-template chatml";
+
+ private static final String IOREG = """
+ +-o AGXAcceleratorG16X <class AGXAcceleratorG16X, id 0x100000a0b, registered, matched, active>
+ {
+ "PerformanceStatistics" = {"Alloc system memory"=34170552320,"Tiler Utilization %"=3,
+ "Renderer Utilization %"=12,"Device Utilization %"=25,"In use system memory"=31463079936}
+ }
+ """;
+
+ @Test
+ void parsesLoadedModelsFromPs() throws Exception {
+ List<LoadedModel> models = OllamaParsers.parsePs((JsonObject) Jsoner.deserialize(PS));
+ assertEquals(1, models.size());
+ LoadedModel m = models.get(0);
+ assertEquals("qwen3.6:35b-a3b", m.name());
+ assertEquals("qwen35moe", m.family());
+ assertEquals("35.5B", m.parameterSize());
+ assertEquals("Q4_K_M", m.quantization());
+ assertEquals(23567972432L, m.sizeBytes());
+ assertEquals(23567972432L, m.sizeVram());
+ assertEquals(100, m.gpuPercent());
+ assertEquals(262144L, m.contextLength());
+ assertEquals(Instant.parse("2026-09-17T09:22:44.593036Z"), m.expiresAt());
+ assertNull(m.shape());
+ }
+
+ @Test
+ void parsesModelShapeFromShow() throws Exception {
+ ModelShape shape = OllamaParsers.parseShow((JsonObject) Jsoner.deserialize(SHOW));
+ assertNotNull(shape);
+ assertEquals("qwen35moe", shape.architecture());
+ assertEquals(41, shape.layers());
+ assertEquals(256, shape.experts());
+ assertEquals(8, shape.expertsUsed());
+ assertEquals(262144L, shape.maxContext());
+ assertEquals(2048, shape.embeddingLength());
+ assertEquals(35505251456L, shape.parameters());
+ assertEquals(List.of("completion", "vision", "tools", "thinking"), shape.capabilities());
+ }
+
+ @Test
+ void parsesRunnerSlots() throws Exception {
+ Instant now = Instant.parse("2026-09-17T09:00:00Z");
+ SlotState slot = OllamaParsers.parseSlots((JsonArray) Jsoner.deserialize(SLOTS), now);
+ assertNotNull(slot);
+ assertTrue(slot.processing());
+ assertEquals(25, slot.promptTokens());
+ assertEquals(25, slot.promptProcessed());
+ assertEquals(9, slot.cacheTokens());
+ assertEquals(6, slot.decoded());
+ assertEquals(262144, slot.contextSize());
+ assertEquals("draft-mtp", slot.speculative());
+ assertEquals(1, slot.slots());
+ assertEquals(31, slot.contextUsed());
+ assertEquals(36, slot.cacheHitPercent());
+ assertEquals(now, slot.sampledAt());
+ }
+
+ @Test
+ void emptySlotsMeanNoRunnerState() {
+ assertNull(OllamaParsers.parseSlots(new JsonArray(), Instant.now()));
+ assertNull(OllamaParsers.parseSlots(null, Instant.now()));
+ }
+
+ @Test
+ void recognisesRunnerCommandLines() {
+ assertTrue(OllamaParsers.isRunnerCommandLine(RUNNER_CMD));
+ assertTrue(OllamaParsers.isRunnerCommandLine("/usr/bin/ollama runner --model /x --port 41231 --threads 8"));
+ assertFalse(OllamaParsers.isRunnerCommandLine("ollama serve"));
+ assertFalse(OllamaParsers.isRunnerCommandLine("/usr/bin/java -jar camel-tui.jar --port 8123"));
+ assertFalse(OllamaParsers.isRunnerCommandLine(null));
+ }
+
+ @Test
+ void parsesRunnerCommandLine() {
+ RunnerInfo runner = OllamaParsers.parseRunnerCommandLine(23629, RUNNER_CMD);
+ assertNotNull(runner);
+ assertEquals(23629, runner.pid());
+ assertEquals(58237, runner.port());
+ assertEquals(262144, runner.contextSize());
+ assertEquals(1, runner.parallel());
+ assertEquals("/Users/me/.ollama/models/blobs/sha256-d372de8e", runner.modelPath());
+ assertEquals("llama-server", runner.executable());
+ }
+
+ @Test
+ void runnerWithoutPortIsIgnored() {
+ assertNull(OllamaParsers.parseRunnerCommandLine(1, "llama-server --model /x --port"));
+ assertNull(OllamaParsers.parseRunnerCommandLine(1, "ollama serve"));
+ }
+
+ @Test
+ void parsesAppleGpuFromIoreg() {
+ GpuStats gpu = OllamaParsers.parseIoreg(IOREG);
+ assertNotNull(gpu);
+ assertEquals(25, gpu.utilizationPercent());
+ assertEquals(31463079936L, gpu.memoryUsedBytes());
+ assertEquals(34170552320L, gpu.memoryTotalBytes());
+ assertEquals("Apple GPU", gpu.name());
+ assertNull(OllamaParsers.parseIoreg(""));
+ assertNull(OllamaParsers.parseIoreg("+-o Something {\"Unrelated\"=1}"));
+ }
+
+ @Test
+ void parsesNvidiaSmiCsv() {
+ GpuStats gpu = OllamaParsers.parseNvidiaSmi("NVIDIA GeForce RTX 4090, 35, 12000, 24564\n"
+ + "NVIDIA GeForce RTX 4090, 65, 2000, 24564\n");
+ assertNotNull(gpu);
+ assertEquals("NVIDIA GeForce RTX 4090", gpu.name());
+ assertEquals(50, gpu.utilizationPercent());
+ assertEquals(14000L * 1024 * 1024, gpu.memoryUsedBytes());
+ assertEquals(2L * 24564 * 1024 * 1024, gpu.memoryTotalBytes());
+ assertEquals(2, gpu.count());
+ assertNull(OllamaParsers.parseNvidiaSmi("No devices were found"));
+ }
+
+ @Test
+ void parsesCpuTimeAndResidentSizeFromPs() {
+ long[] stats = OllamaParsers.parsePsCpuAndRss(" 0:11.04 28790208\n");
+ assertNotNull(stats);
+ assertEquals(11040, stats[0]);
+ assertEquals(28790208L * 1024, stats[1]);
+ assertNull(OllamaParsers.parsePsCpuAndRss(""));
+ assertNull(OllamaParsers.parsePsCpuAndRss(null));
+
+ assertEquals(11040, OllamaParsers.parseCpuTimeMillis("0:11.04"));
+ assertEquals(3723000, OllamaParsers.parseCpuTimeMillis("01:02:03"));
+ assertEquals(86400000L + 3600000 + 120000 + 3000, OllamaParsers.parseCpuTimeMillis("1-01:02:03"));
+ assertEquals(1500, OllamaParsers.parseCpuTimeMillis("1.5"));
+ assertEquals(-1, OllamaParsers.parseCpuTimeMillis("n/a"));
+ assertEquals(-1, OllamaParsers.parseCpuTimeMillis(null));
+ }
+
+ @Test
+ void parsesOllamaTimestamps() {
+ assertEquals(Instant.parse("2026-09-17T09:22:44.593036Z"),
+ OllamaParsers.parseInstant("2026-09-17T11:22:44.593036+02:00"));
+ assertEquals(Instant.parse("2026-09-17T09:17:44.592552Z"),
+ OllamaParsers.parseInstant("2026-09-17T09:17:44.592552Z"));
+ assertNull(OllamaParsers.parseInstant("0001-01-01"));
+ assertNull(OllamaParsers.parseInstant(null));
+ }
+
+ @Test
+ void loopbackAndDisplayHost() {
+ assertTrue(OllamaParsers.isLoopbackUrl("http://localhost:11434"));
+ assertTrue(OllamaParsers.isLoopbackUrl("http://127.0.0.1:32771"));
+ assertFalse(OllamaParsers.isLoopbackUrl("http://gpu-box.lan:11434"));
+ assertFalse(OllamaParsers.isLoopbackUrl(null));
+ assertEquals("localhost:11434", OllamaParsers.displayHost("http://localhost:11434/"));
+ assertEquals("gpu-box.lan:11434", OllamaParsers.displayHost("https://gpu-box.lan:11434"));
+ }
+}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
new file mode 100644
index 000000000000..9035849aa042
--- /dev/null
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
@@ -0,0 +1,240 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.tui;
+
+import java.time.Instant;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.camel.dsl.jbang.core.commands.LlmClient;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.GpuStats;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.HostStats;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.LoadedModel;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.ModelShape;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.ProcessStats;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RequestEntry;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RequestSource;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RunnerInfo;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.ServerInfo;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.SlotState;
+import org.apache.camel.util.json.JsonArray;
+import org.apache.camel.util.json.JsonObject;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Renders {@link OllamaTab} into a virtual terminal from a monitor fed by hand, so the layout is checked without an
+ * Ollama server.
+ */
+class OllamaTabRenderTest {
+
+ private MonitorContext ctx;
+ private OllamaMonitor monitor;
+
+ @BeforeEach
+ void setUp() {
+ Theme.resetForTesting();
+ AtomicReference<List<IntegrationInfo>> data = new AtomicReference<>(List.of());
+ AtomicReference<List<InfraInfo>> infraData = new AtomicReference<>(List.of());
+ ctx = new MonitorContext(data, infraData);
+ monitor = new OllamaMonitor();
+ ctx.ollamaMonitor = monitor;
+ }
+
+ @Test
+ void showsHowToStartOllamaWhenNotDetected() {
+ String rendered = TuiTestHelper.renderToString(new OllamaTab(ctx, monitor), 160, 30);
+ assertTrue(rendered.contains("Ollama not detected at localhost:11434"), rendered);
+ assertTrue(rendered.contains("ollama serve"), rendered);
+ assertTrue(rendered.contains("camel infra run ollama"), rendered);
+ }
+
+ @Test
+ void rendersLocalServerWithModelPanelsAndRequests() {
+ localServerWithModel();
+ monitor.updateRunner(new RunnerInfo(23629, 58237, 262144, 1, "/blob", "llama-server"));
+ monitor.updateSlot(new SlotState(false, 25, 25, 9, 6, 262144, "draft-mtp", 1, Instant.now()));
+ monitor.updateHost(new HostStats(
+ new GpuStats("Apple GPU", 25, 31463079936L, 34170552320L, 1),
+ new ProcessStats(70566, "ollama serve", 0.1, 82176L * 1024),
+ new ProcessStats(23629, "llama-server", 12.5, 28273856L * 1024),
+ Instant.now()));
+ monitor.recordRequest("qwen3.6:35b-a3b", new LlmClient.TokenUsage(16, 6, 22, 0, 188, 81, 12, 300), 0, "stop");
+
+ String rendered = TuiTestHelper.renderToString(new OllamaTab(ctx, monitor), 180, 40);
+
+ // header
+ assertTrue(rendered.contains("Ollama 0.33.3"), rendered);
+ assertTrue(rendered.contains("localhost:11434"), rendered);
+ assertTrue(rendered.contains("runner llama-server :58237"), rendered);
+ assertTrue(rendered.contains("qwen3.6:35b-a3b"), rendered);
+ assertTrue(rendered.contains("qwen35moe"), rendered);
+ assertTrue(rendered.contains("41 layers"), rendered);
+ assertTrue(rendered.contains("256 experts (8 active)"), rendered);
+ assertTrue(rendered.contains("100% GPU"), rendered);
+ assertTrue(rendered.contains("ctx 262,144"), rendered);
+ // panels
+ assertTrue(rendered.contains("Throughput"), rendered);
+ assertTrue(rendered.contains("decode"), rendered);
+ assertTrue(rendered.contains("74"), rendered); // 6 tokens in 81 ms
+ assertTrue(rendered.contains("prefill"), rendered);
+ assertTrue(rendered.contains("TTFT 0.20s"), rendered);
+ assertTrue(rendered.contains("Context"), rendered);
+ assertTrue(rendered.contains("cache hit 36%"), rendered);
+ assertTrue(rendered.contains("idle"), rendered);
+ assertTrue(rendered.contains("speculative draft-mtp"), rendered);
+ assertTrue(rendered.contains("Host"), rendered);
+ assertTrue(rendered.contains("GPU"), rendered);
+ assertTrue(rendered.contains("25%"), rendered);
+ assertTrue(rendered.contains("llama-server"), rendered);
+ assertTrue(rendered.contains("ollama serve"), rendered);
+ // request log
+ assertTrue(rendered.contains("Requests (1)"), rendered);
+ assertTrue(rendered.contains("PREFILL"), rendered);
+ assertTrue(rendered.contains("DECODE"), rendered);
+ assertTrue(rendered.contains("tui"), rendered);
+ assertTrue(rendered.contains("stop"), rendered);
+ }
+
+ @Test
+ void remoteServerHidesHostPanelAndExplainsMissingLiveState() {
+ monitor.updateServer(new ServerInfo("http://gpu-box.lan:11434", "0.33.3", false));
+ monitor.updateModels(List.of(model()));
+
+ String rendered = TuiTestHelper.renderToString(new OllamaTab(ctx, monitor), 160, 30);
+
+ assertTrue(rendered.contains("gpu-box.lan:11434"), rendered);
+ assertTrue(rendered.contains("remote"), rendered);
+ assertFalse(rendered.contains(" Host "), rendered);
+ assertTrue(rendered.contains("live state needs the runner on this machine"), rendered);
+ assertTrue(rendered.contains("No requests yet"), rendered);
+ }
+
+ @Test
+ void noModelLoadedListsInstalledModels() {
+ monitor.updateServer(new ServerInfo("http://localhost:11434", "0.33.3", true));
+ monitor.updateInstalled(List.of("qwen3.6:35b-a3b", "llama3.2:latest"));
+
+ String rendered = TuiTestHelper.renderToString(new OllamaTab(ctx, monitor), 160, 30);
+
+ assertTrue(rendered.contains("No model loaded"), rendered);
+ assertTrue(rendered.contains("2 installed: qwen3.6:35b-a3b, llama3.2:latest"), rendered);
+ }
+
+ @Test
+ void routeRequestsShowTheirRouteId() {
+ localServerWithModel();
+ monitor.recordRequest("qwen3.6:35b-a3b", new LlmClient.TokenUsage(16, 6, 22, 0, 188, 81, 12, 300), 0, "stop");
+ // a route call comes without phase timings: prefill/decode show as dashes, total from the span
+ monitor.ingestSpans(List.of(routeSpan()));
+
+ String rendered = TuiTestHelper.renderToString(new OllamaTab(ctx, monitor), 180, 40);
+ assertTrue(rendered.contains("route:chat-route"), rendered);
+ assertTrue(rendered.contains("Requests (2)"), rendered);
+ }
+
+ @Test
+ void tableDataJsonCarriesRequestsAndSummary() {
+ localServerWithModel();
+ monitor.recordRequest("qwen3.6:35b-a3b", new LlmClient.TokenUsage(16, 6, 22, 0, 188, 81, 12, 300), 0, "stop");
+
+ JsonObject json = new OllamaTab(ctx, monitor).getTableDataAsJson();
+ assertEquals("Ollama", json.get("tab"));
+ assertEquals(1, json.get("totalRows"));
+ JsonArray rows = (JsonArray) json.get("rows");
+ assertEquals(1, rows.size());
+ assertEquals("qwen3.6:35b-a3b", ((JsonObject) rows.get(0)).get("model"));
+ JsonObject summary = (JsonObject) json.get("summary");
+ assertEquals(true, summary.get("connected"));
+ assertEquals(1, ((JsonArray) summary.get("loadedModels")).size());
+ }
+
+ @Test
+ void formattingHelpers() {
+ assertEquals("54", OllamaTab.formatRate(54.04));
+ assertEquals("7.5", OllamaTab.formatRate(7.49));
+ assertEquals("0", OllamaTab.formatRate(0));
+ assertEquals("999", OllamaTab.formatTokens(999));
+ assertEquals("1.2k", OllamaTab.formatTokens(1203));
+ assertEquals("262k", OllamaTab.formatTokens(262144));
+ assertEquals("1.5M", OllamaTab.formatTokens(1_500_000));
+ assertEquals("0.14s", OllamaTab.formatSeconds(138));
+ assertEquals("2.8s", OllamaTab.formatSeconds(2829));
+ assertEquals("11s", OllamaTab.formatSeconds(11378));
+ assertEquals("1m5s", OllamaTab.formatSeconds(65_000));
+ assertEquals("▓▓▓▓▓░░░░░", OllamaTab.gaugeBar(50, 10));
+ assertEquals("░░░░", OllamaTab.gaugeBar(0, 4));
+ assertEquals("▓▓▓▓", OllamaTab.gaugeBar(140, 4));
+ assertEquals(" ▁▄█", OllamaTab.sparkline(new long[] { 10, 40, 80 }, 5));
+ assertEquals(" ", OllamaTab.sparkline(new long[] { 0, 0 }, 4));
+ assertEquals("▁█", OllamaTab.sparkline(new long[] { 1, 2, 3, 20 }, 2));
+ Instant now = Instant.parse("2026-09-17T09:18:12Z");
+ assertEquals("unloads in 4m32s", OllamaTab.formatCountdown(Instant.parse("2026-09-17T09:22:44Z"), now));
+ assertEquals("unloads in 2h5m", OllamaTab.formatCountdown(now.plusSeconds(2 * 3600 + 300), now));
+ assertEquals("unloading", OllamaTab.formatCountdown(now.minusSeconds(1), now));
+ assertEquals("stays loaded", OllamaTab.formatCountdown(now.plusSeconds(400L * 24 * 3600), now));
+ }
+
+ @Test
+ void requestHistoryIsOldestFirstAndCappedToWidth() {
+ List<RequestEntry> newestFirst = List.of(
+ request(30), request(20), request(10));
+ long[] history = OllamaTab.requestHistory(newestFirst, 2);
+ assertEquals(2, history.length);
+ assertEquals(20, history[0]);
+ assertEquals(30, history[1]);
+ }
+
+ private void localServerWithModel() {
+ monitor.updateServer(new ServerInfo("http://localhost:11434", "0.33.3", true));
+ monitor.updateModels(List.of(model()));
+ }
+
+ private static LoadedModel model() {
+ return new LoadedModel(
+ "qwen3.6:35b-a3b", "qwen35moe", "35.5B", "Q4_K_M", 23567972432L, 23567972432L, 262144,
+ Instant.now().plusSeconds(272),
+ new ModelShape(
+ "qwen35moe", 41, 256, 8, 262144, 2048, 35505251456L,
+ List.of("completion", "vision", "tools", "thinking")));
+ }
+
+ private static RequestEntry request(int decodeTokensPerSecond) {
+ return new RequestEntry(
+ Instant.now(), RequestSource.TUI, null, "m", 10, decodeTokensPerSecond, 0, 100, 1000,
+ 0, 1100, "stop");
+ }
+
+ private static SpanEntry routeSpan() {
+ Map<String, Object> attrs = new HashMap<>();
+ attrs.put("gen_ai.operation.name", "chat");
+ attrs.put("gen_ai.system", "ollama");
+ attrs.put("gen_ai.request.model", "qwen3.6:35b-a3b");
+ attrs.put("gen_ai.usage.input_tokens", "412");
+ attrs.put("gen_ai.usage.output_tokens", "180");
+ long start = Instant.now().toEpochMilli() * 1_000_000L;
+ return new SpanEntry(
+ "trace-1", "span-1", null, "chat qwen3.6:35b-a3b", "CLIENT", "OK", start,
+ start + 4100 * 1_000_000L, 4100, "chat-route", null, null, attrs);
+ }
+}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/TokenRateWindowTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/TokenRateWindowTest.java
new file mode 100644
index 000000000000..4016593b7ea0
--- /dev/null
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/TokenRateWindowTest.java
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.tui;
+
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+class TokenRateWindowTest {
+
+ @Test
+ void steadyCounterGivesTokensPerSecond() {
+ TokenRateWindow w = new TokenRateWindow(1000);
+ w.sample(0, 0);
+ w.sample(500, 25);
+ w.sample(1000, 50);
+ assertEquals(50.0, w.ratePerSecond(1000), 0.001);
+ }
+
+ @Test
+ void windowDropsOldSamplesButKeepsABaseline() {
+ TokenRateWindow w = new TokenRateWindow(1000);
+ w.sample(0, 0);
+ w.sample(500, 10);
+ w.sample(1000, 20);
+ w.sample(1500, 80); // 60 tokens in the last half second
+ w.sample(2000, 140);
+ // baseline is the sample at t=1000 (20 tokens): 120 tokens over 1 second
+ assertEquals(120.0, w.ratePerSecond(2000), 0.001);
+ }
+
+ @Test
+ void singleSampleOrStaleCounterIsZero() {
+ TokenRateWindow w = new TokenRateWindow(1000);
+ assertEquals(0.0, w.ratePerSecond(0), 0.001);
+ w.sample(0, 100);
+ assertEquals(0.0, w.ratePerSecond(0), 0.001);
+ w.sample(500, 150);
+ assertEquals(100.0, w.ratePerSecond(500), 0.001);
+ // nothing sampled for longer than the window: the model stopped
+ assertEquals(0.0, w.ratePerSecond(2000), 0.001);
+ }
+
+ @Test
+ void counterResetStartsANewBaseline() {
+ TokenRateWindow w = new TokenRateWindow(1000);
+ w.sample(0, 90);
+ w.sample(500, 100);
+ // new request: llama-server counts decoded tokens per request, so the counter drops
+ w.sample(1000, 3);
+ w.sample(1500, 33);
+ assertEquals(60.0, w.ratePerSecond(1500), 0.001);
+ }
+
+ @Test
+ void idleCounterIsZeroNotNegative() {
+ TokenRateWindow w = new TokenRateWindow(1000);
+ w.sample(0, 40);
+ w.sample(500, 40);
+ w.sample(1000, 40);
+ assertEquals(0.0, w.ratePerSecond(1000), 0.001);
+ }
+}
(camel) 08/09: CAMEL-24794: camel-jbang - TUI Ollama tab groups the request log per question
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch tui-ollama-tab
in repository https://gitbox.apache.org/repos/asf/camel.git
commit e5a0c17cb59979f5cd189b0fdf54baa9b0664de4
Author: Claus Ibsen <cl...@gmail.com>
AuthorDate: Thu Sep 17 12:57:16 2026 +0200
CAMEL-24794: camel-jbang - TUI Ollama tab groups the request log per question
A question asked in the AI panel costs one request per tool-call step, so a
single question filled ten lines of the log while the model column repeated
the one local model on every line. The log now shows one line per question:
the question text (first line, cut to fit) where the model was, the number
of requests it took, and the figures for the question as a whole: the
largest prompt, generated tokens, cache hit across the steps, the highest
context fill, prefill and decode rates across the steps, the time to the
first token of the first step and the wall time from the first request to
the last. Enter or Right unfolds the steps with their own figures and the
model, Left folds them. Route calls stay one line each.
The AI panel passes the question number and text to the monitor; the JSON
of tui_get_ollama carries them per request and adds a questions array with
the aggregates. F1 help and the user manual describe the grouped view.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Signed-off-by: Claus Ibsen <cl...@gmail.com>
---
.../modules/ROOT/pages/camel-jbang-tui.adoc | 8 +-
.../camel/dsl/jbang/core/commands/tui/AiPanel.java | 7 +-
.../dsl/jbang/core/commands/tui/OllamaMonitor.java | 197 ++++++++++++++++++++-
.../dsl/jbang/core/commands/tui/OllamaTab.java | 183 +++++++++++++++----
.../src/main/resources/tui/help/ollama.md | 42 +++--
.../jbang/core/commands/tui/OllamaMonitorTest.java | 54 ++++++
.../core/commands/tui/OllamaTabRenderTest.java | 42 ++++-
7 files changed, 474 insertions(+), 59 deletions(-)
diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
index 369e9e2de5ad..2d3f8a0373a6 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
@@ -432,9 +432,11 @@ the TUI checks every ten seconds, so it appears shortly after `ollama serve` sta
of how much of the window each AI panel prompt filled, with the session peak and the compactions seen.
* *Host* -- GPU utilization and memory (Apple silicon through `ioreg`, NVIDIA through `nvidia-smi`),
and CPU and memory of the Ollama server and its model runner.
-* *Requests* -- one line per request with input, output and cached tokens, the share of the context
- window the prompt filled, prefill and decode tokens per second, time to first token, total time and
- the stop reason.
+* *Requests* -- one line per question asked in the AI panel (a question with tool calls costs one
+ request per step; *Enter* unfolds the steps) with the question text, prompt and generated tokens,
+ cache hit, the share of the context window reached, prefill and decode tokens per second, time to
+ first token, the time you waited and the stop reason. Calls made by Camel routes are listed as
+ their own lines.
Two kinds of requests appear in the log. Questions asked in the AI panel with Ollama as the provider
come with the timings Ollama reports for each request (prompt evaluation, generation, model load,
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java
index 1e2608825dfa..bb046a7e18b0 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java
@@ -298,6 +298,8 @@ class AiPanel {
private final List<AiUsageEntry> usageHistory = new CopyOnWriteArrayList<>();
/** Sequence number of the current question; tags the usage entries recorded while answering it. */
private volatile int questionCounter;
+ /** The question the current turn answers, for grouping its requests on the Ollama tab. */
+ private volatile String currentQuestion;
/** Route usage (GenAI spans) recorded before this instant is left out after a {@code /usage reset}. */
private volatile Instant usageResetAt = Instant.EPOCH;
private AtomicReference<List<SpanEntry>> otelSpans = new AtomicReference<>(List.of());
@@ -1170,6 +1172,7 @@ class AiPanel {
// the waiting camel_write_file call returns with the question and the model answers in this turn
conversation.add(new ConversationEntry(AiRole.USER, input));
questionCounter++;
+ currentQuestion = input;
log(LogLevel.QUESTION, "Question about the edit", input);
return;
}
@@ -1354,6 +1357,7 @@ class AiPanel {
}
conversation.add(new ConversationEntry(AiRole.USER, question));
questionCounter++;
+ currentQuestion = question;
log(LogLevel.QUESTION, "Question", question);
thinkingVerb = THINKING_VERBS.get(ThreadLocalRandom.current().nextInt(THINKING_VERBS.size()));
thinkingStartTime = System.currentTimeMillis();
@@ -1995,7 +1999,8 @@ class AiPanel {
if (ctx != null && ctx.ollamaMonitor != null && client.apiType() == LlmClient.ApiType.ollama) {
// the Ollama tab shows this request with the timings Ollama returned
ctx.ollamaMonitor.adoptEndpoint(client.endpointUrl());
- ctx.ollamaMonitor.recordRequest(model, response.usage(), latencyMs, response.stopReason());
+ ctx.ollamaMonitor.recordRequest(model, response.usage(), latencyMs, response.stopReason(),
+ questionCounter, currentQuestion);
}
}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
index 553e11d8ce71..bc56758e4c57 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
@@ -146,7 +146,22 @@ final class OllamaMonitor {
*/
record RequestEntry(Instant timestamp, RequestSource source, String routeId, String model, int inputTokens,
int outputTokens, int cachedTokens, long prefillMs, long decodeMs, long loadMs, long totalMs,
- String doneReason, long contextSize) {
+ String doneReason, long contextSize, int question, String questionText) {
+
+ RequestEntry(Instant timestamp, RequestSource source, String routeId, String model, int inputTokens,
+ int outputTokens, int cachedTokens, long prefillMs, long decodeMs, long loadMs, long totalMs,
+ String doneReason, long contextSize) {
+ this(timestamp, source, routeId, model, inputTokens, outputTokens, cachedTokens, prefillMs, decodeMs,
+ loadMs, totalMs, doneReason, contextSize, 0, null);
+ }
+
+ /** Requests of the same AI panel question (its tool-call steps) share this key; a route call stands alone. */
+ String groupKey() {
+ if (source == RequestSource.ROUTE) {
+ return "route:" + timestamp.toEpochMilli() + ":" + routeId;
+ }
+ return "q" + question + ":" + (questionText != null ? questionText : "");
+ }
/**
* Everything the model had in front of it. Ollama's {@code prompt_eval_count} is the whole prompt; the cached
@@ -201,6 +216,131 @@ final class OllamaMonitor {
}
}
+ /**
+ * One AI panel question with all the requests it took (the model's tool calls each cost a request), oldest step
+ * first, or a single route call. Aggregates are what the user experienced for the whole question.
+ */
+ record QuestionGroup(String key, RequestSource source, int question, String questionText, String routeId,
+ List<RequestEntry> steps) {
+
+ RequestEntry first() {
+ return steps.get(0);
+ }
+
+ RequestEntry last() {
+ return steps.get(steps.size() - 1);
+ }
+
+ /** The largest prompt of the question: how far the context was pushed. */
+ long promptTokens() {
+ long max = 0;
+ for (RequestEntry e : steps) {
+ max = Math.max(max, e.promptTokens());
+ }
+ return max;
+ }
+
+ long outputTokens() {
+ long sum = 0;
+ for (RequestEntry e : steps) {
+ sum += e.outputTokens();
+ }
+ return sum;
+ }
+
+ int cacheHitPercent() {
+ long prompt = 0;
+ long cached = 0;
+ for (RequestEntry e : steps) {
+ prompt += e.inputTokens();
+ cached += e.cachedTokens();
+ }
+ return prompt > 0 ? (int) Math.min(100, cached * 100 / prompt) : 0;
+ }
+
+ int contextPercent() {
+ int max = -1;
+ for (RequestEntry e : steps) {
+ max = Math.max(max, e.contextPercent());
+ }
+ return max;
+ }
+
+ double prefillTokensPerSecond() {
+ long evaluated = 0;
+ long ms = 0;
+ for (RequestEntry e : steps) {
+ evaluated += e.evaluatedTokens();
+ ms += e.prefillMs();
+ }
+ return ms > 0 && evaluated > 0 ? evaluated * 1000.0 / ms : 0;
+ }
+
+ double decodeTokensPerSecond() {
+ long out = 0;
+ long ms = 0;
+ for (RequestEntry e : steps) {
+ out += e.outputTokens();
+ ms += e.decodeMs();
+ }
+ return ms > 0 && out > 0 ? out * 1000.0 / ms : 0;
+ }
+
+ /** Time to the first token of the first step: the wait before anything happened. */
+ long ttftMs() {
+ return first().ttftMs();
+ }
+
+ boolean hasTimings() {
+ return first().hasTimings();
+ }
+
+ boolean coldStart() {
+ for (RequestEntry e : steps) {
+ if (e.coldStart()) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ /** From the first request starting to the last one finishing. */
+ long wallMs() {
+ long start = first().timestamp().toEpochMilli();
+ long end = last().timestamp().toEpochMilli() + last().totalMs();
+ return Math.max(end - start, last().totalMs());
+ }
+
+ String doneReason() {
+ return last().doneReason();
+ }
+ }
+
+ /**
+ * Groups requests (newest first) into questions: consecutive AI panel requests with the same question form one
+ * group with their steps oldest first; every route call is its own group. Groups come back newest first.
+ */
+ static List<QuestionGroup> groupByQuestion(List<RequestEntry> newestFirst) {
+ List<QuestionGroup> groups = new ArrayList<>();
+ int i = 0;
+ while (i < newestFirst.size()) {
+ RequestEntry head = newestFirst.get(i);
+ String key = head.groupKey();
+ List<RequestEntry> steps = new ArrayList<>();
+ int j = i;
+ while (j < newestFirst.size() && newestFirst.get(j).groupKey().equals(key)
+ && (head.source() == RequestSource.TUI || j == i)) {
+ steps.add(0, newestFirst.get(j));
+ j++;
+ }
+ groups.add(new QuestionGroup(
+ key, head.source(), head.question(), head.questionText(), head.routeId(),
+ List.copyOf(steps)));
+ i = j;
+ }
+ return groups;
+ }
+
record SessionTotals(int requests, long inputTokens, long outputTokens, long cachedTokens, long prefillMs,
long decodeMs, long loadMs, int coldStarts, int peakContextPercent, int compactions) {
@@ -329,8 +469,18 @@ final class OllamaMonitor {
}
}
- /** Records a request the TUI itself made to Ollama, with the usage and timings Ollama returned. */
+ /** As {@link #recordRequest(String, LlmClient.TokenUsage, long, String, int, String)} without a question. */
void recordRequest(String model, LlmClient.TokenUsage usage, long latencyMs, String doneReason) {
+ recordRequest(model, usage, latencyMs, doneReason, 0, null);
+ }
+
+ /**
+ * Records a request the TUI itself made to Ollama, with the usage and timings Ollama returned. {@code question} and
+ * {@code questionText} identify the AI panel turn so the tool-call steps of one question group together.
+ */
+ void recordRequest(
+ String model, LlmClient.TokenUsage usage, long latencyMs, String doneReason, int question,
+ String questionText) {
if (usage == null) {
return;
}
@@ -339,7 +489,7 @@ final class OllamaMonitor {
Instant.now(), RequestSource.TUI, null, model != null ? model : "unknown",
usage.inputTokens(), usage.outputTokens(), usage.cachedTokens(),
usage.prefillMillis(), usage.generationMillis(), usage.loadMillis(), total, doneReason,
- contextSizeForNewRequest());
+ contextSizeForNewRequest(), question, questionText != null ? questionText.strip() : null);
synchronized (lock) {
addRequest(entry);
}
@@ -1007,6 +1157,41 @@ final class OllamaMonitor {
reqs.add(requestJson(e));
}
root.put("requests", reqs);
+ JsonArray questions = new JsonArray();
+ int q = 0;
+ for (QuestionGroup g : groupByQuestion(s.requests())) {
+ if (q++ >= requestLimit) {
+ break;
+ }
+ JsonObject jg = new JsonObject();
+ jg.put("time", g.first().timestamp().toString());
+ jg.put("source", g.source().name().toLowerCase(Locale.ROOT));
+ if (g.routeId() != null) {
+ jg.put("routeId", g.routeId());
+ }
+ if (g.question() > 0) {
+ jg.put("question", g.question());
+ }
+ if (g.questionText() != null) {
+ jg.put("questionText", g.questionText());
+ }
+ jg.put("model", g.last().model());
+ jg.put("steps", g.steps().size());
+ jg.put("promptTokens", g.promptTokens());
+ jg.put("outputTokens", g.outputTokens());
+ jg.put("cacheHitPercent", g.cacheHitPercent());
+ jg.put("contextPercent", g.contextPercent());
+ jg.put("prefillTokensPerSecond", round1(g.prefillTokensPerSecond()));
+ jg.put("decodeTokensPerSecond", round1(g.decodeTokensPerSecond()));
+ jg.put("ttftMs", g.ttftMs());
+ jg.put("wallMs", g.wallMs());
+ jg.put("coldStart", g.coldStart());
+ if (g.doneReason() != null) {
+ jg.put("doneReason", g.doneReason());
+ }
+ questions.add(jg);
+ }
+ root.put("questions", questions);
if (s.lastPoll() != null) {
root.put("lastPoll", s.lastPoll().toString());
}
@@ -1034,6 +1219,12 @@ final class OllamaMonitor {
r.put("coldStart", e.coldStart());
r.put("promptTokens", e.promptTokens());
r.put("evaluatedTokens", e.evaluatedTokens());
+ if (e.question() > 0) {
+ r.put("question", e.question());
+ }
+ if (e.questionText() != null) {
+ r.put("questionText", e.questionText());
+ }
r.put("contextSize", e.contextSize());
r.put("contextPercent", e.contextPercent());
if (e.doneReason() != null) {
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
index 7769b54d46f9..c7d9320d87f2 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
@@ -21,8 +21,10 @@ import java.time.Instant;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
+import java.util.HashSet;
import java.util.List;
import java.util.Locale;
+import java.util.Set;
import dev.tamboui.layout.Constraint;
import dev.tamboui.layout.Layout;
@@ -47,6 +49,7 @@ import dev.tamboui.widgets.table.TableState;
import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.HostStats;
import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.LoadedModel;
import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.ModelShape;
+import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.QuestionGroup;
import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RequestEntry;
import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.RequestSource;
import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.SessionTotals;
@@ -76,6 +79,10 @@ class OllamaTab extends AbstractTab {
private final TableState tableState = new TableState();
private final ScrollbarState scrollState = new ScrollbarState();
private Rect lastTableArea;
+ /** Questions whose steps are unfolded in the request log. */
+ private final Set<String> expandedGroups = new HashSet<>();
+ /** What each table row is: a {@link QuestionGroup} header or a {@link RequestEntry} step, as last rendered. */
+ private List<Object> rowRefs = List.of();
OllamaTab(MonitorContext ctx, OllamaMonitor monitor) {
super(ctx);
@@ -107,8 +114,25 @@ class OllamaTab extends AbstractTab {
navigateDown();
return true;
}
+ if (ke.isConfirm() || ke.isRight() || ke.isLeft()) {
+ QuestionGroup group = selectedGroup();
+ if (group != null && group.steps().size() > 1) {
+ if (ke.isLeft()) {
+ expandedGroups.remove(group.key());
+ selectGroupHeader(group);
+ } else if (ke.isRight()) {
+ expandedGroups.add(group.key());
+ } else if (!expandedGroups.remove(group.key())) {
+ expandedGroups.add(group.key());
+ } else {
+ selectGroupHeader(group);
+ }
+ }
+ return true;
+ }
if (ke.isChar('r') && monitor != null) {
monitor.reset();
+ expandedGroups.clear();
tableState.select(0);
return true;
}
@@ -119,6 +143,35 @@ class OllamaTab extends AbstractTab {
return false;
}
+ /** The question the selected row belongs to (a header row or one of its steps). */
+ private QuestionGroup selectedGroup() {
+ Integer sel = tableState.selected();
+ if (sel == null || sel < 0 || sel >= rowRefs.size()) {
+ return null;
+ }
+ Object ref = rowRefs.get(sel);
+ if (ref instanceof QuestionGroup g) {
+ return g;
+ }
+ if (ref instanceof RequestEntry e) {
+ for (int i = sel; i >= 0; i--) {
+ if (rowRefs.get(i) instanceof QuestionGroup g && g.steps().contains(e)) {
+ return g;
+ }
+ }
+ }
+ return null;
+ }
+
+ private void selectGroupHeader(QuestionGroup group) {
+ for (int i = 0; i < rowRefs.size(); i++) {
+ if (rowRefs.get(i) == group) {
+ tableState.select(i);
+ return;
+ }
+ }
+ }
+
@Override
public void navigateUp() {
tableState.selectPrevious();
@@ -126,7 +179,7 @@ class OllamaTab extends AbstractTab {
@Override
public void navigateDown() {
- int rows = monitor != null ? monitor.snapshot().requests().size() : 0;
+ int rows = rowRefs.size();
if (rows > 0) {
tableState.selectNext(rows);
}
@@ -134,13 +187,13 @@ class OllamaTab extends AbstractTab {
@Override
public boolean handleMouseEvent(MouseEvent me, Rect area) {
- int rows = monitor != null ? monitor.snapshot().requests().size() : 0;
- return handleTableClick(me, lastTableArea, tableState, rows);
+ return handleTableClick(me, lastTableArea, tableState, rowRefs.size());
}
@Override
public void renderFooter(List<Span> spans) {
hint(spans, TuiIcons.ARROW_UP + TuiIcons.ARROW_DOWN, "select");
+ hint(spans, "Enter", "steps");
hint(spans, "r", "reset");
hintLast(spans, "F5", "refresh");
}
@@ -457,28 +510,8 @@ class OllamaTab extends AbstractTab {
private void renderRequests(Frame frame, Rect area, Snapshot s) {
List<RequestEntry> requests = s.requests();
- List<Row> rows = new ArrayList<>();
- for (RequestEntry e : requests) {
- rows.add(Row.from(
- Cell.from(Span.styled(" " + TIME.format(e.timestamp()), Theme.muted())),
- Cell.from(Span.styled(sourceLabel(e), e.source() == RequestSource.ROUTE
- ? Theme.notice()
- : Theme.info())),
- Cell.from(Span.raw(e.model())),
- rightCell(formatTokens(e.inputTokens()), 6),
- rightCell(formatTokens(e.outputTokens()), 6),
- rightCell(e.cachedTokens() > 0 ? formatTokens(e.cachedTokens()) : "-", 6, Theme.muted()),
- rightCell(e.contextPercent() >= 0 ? e.contextPercent() + "%" : "-", 5,
- e.contextPercent() >= 80 ? Theme.error() : e.contextPercent() >= 50 ? Theme.warning()
- : Style.EMPTY),
- rightCell(e.prefillMs() > 0 ? formatRate(e.prefillTokensPerSecond()) : "-", 8, Theme.info()),
- rightCell(e.decodeMs() > 0 ? formatRate(e.decodeTokensPerSecond()) : "-", 8, Theme.success()),
- rightCell(e.hasTimings() ? formatSeconds(e.ttftMs()) : "-", 7,
- e.coldStart() ? Theme.warning() : Style.EMPTY),
- rightCell(e.totalMs() > 0 ? formatSeconds(e.totalMs()) : "-", 7),
- Cell.from(Span.styled(" " + (e.doneReason() != null ? e.doneReason() : ""), Theme.muted()))));
- }
- if (rows.isEmpty()) {
+ if (requests.isEmpty()) {
+ rowRefs = List.of();
lastTableArea = area;
frame.renderWidget(Paragraph.builder()
.text(Text.from(List.of(
@@ -491,15 +524,39 @@ class OllamaTab extends AbstractTab {
.build(), area);
return;
}
+
+ List<QuestionGroup> groups = OllamaMonitor.groupByQuestion(requests);
+ // fixed columns plus borders and the highlight gutter; the rest is the question column
+ int questionWidth = Math.max(12, area.width() - FIXED_COLUMNS_WIDTH);
+ List<Row> rows = new ArrayList<>();
+ List<Object> refs = new ArrayList<>();
+ for (QuestionGroup g : groups) {
+ boolean multi = g.steps().size() > 1;
+ boolean expanded = multi && expandedGroups.contains(g.key());
+ rows.add(questionRow(g, multi, expanded, questionWidth));
+ refs.add(g);
+ if (expanded) {
+ int n = 1;
+ for (RequestEntry e : g.steps()) {
+ rows.add(stepRow(e, n++, g.steps().size()));
+ refs.add(e);
+ }
+ }
+ }
+ rowRefs = refs;
+
+ String title = " Requests (" + groups.size() + (groups.size() == 1 ? " question, " : " questions, ")
+ + requests.size() + (requests.size() == 1 ? " request)" : " requests)")
+ + " tok/s for prefill and decode · CTX = prompt share of the context window ";
Table table = Table.builder()
.rows(rows)
.header(Row.from(
Cell.from(Span.styled(" TIME", Style.EMPTY.bold())),
Cell.from(Span.styled("SOURCE", Style.EMPTY.bold())),
- Cell.from(Span.styled("MODEL", Style.EMPTY.bold())),
+ Cell.from(Span.styled("QUESTION", Style.EMPTY.bold())),
rightCell("IN", 6, Style.EMPTY.bold()),
rightCell("OUT", 6, Style.EMPTY.bold()),
- rightCell("CACHED", 6, Style.EMPTY.bold()),
+ rightCell("CACHE", 6, Style.EMPTY.bold()),
rightCell("CTX", 5, Style.EMPTY.bold()),
rightCell("PREFILL", 8, Style.EMPTY.bold()),
rightCell("DECODE", 8, Style.EMPTY.bold()),
@@ -507,7 +564,7 @@ class OllamaTab extends AbstractTab {
rightCell("TOTAL", 7, Style.EMPTY.bold()),
Cell.from(Span.styled(" REASON", Style.EMPTY.bold()))))
.widths(
- Constraint.length(9),
+ Constraint.length(10),
Constraint.length(18),
Constraint.fill(),
Constraint.length(6),
@@ -521,16 +578,78 @@ class OllamaTab extends AbstractTab {
Constraint.length(12))
.highlightStyle(Theme.selectionBg())
.highlightSpacing(Table.HighlightSpacing.ALWAYS)
- .block(Block.builder().borderType(BorderType.ROUNDED).borders(Borders.ALL)
- .title(" Requests (" + requests.size()
- + ") tok/s for prefill and decode · CTX = prompt share of the context window ")
- .build())
+ .block(Block.builder().borderType(BorderType.ROUNDED).borders(Borders.ALL).title(title).build())
.build();
lastTableArea = area;
frame.renderStatefulWidget(table, area, tableState);
renderTableScrollbar(frame, area, tableState, scrollState, rows.size());
}
+ private static final int FIXED_COLUMNS_WIDTH = 10 + 18 + 6 + 6 + 6 + 5 + 8 + 8 + 7 + 7 + 12 + 6;
+
+ /** One question (or route call) on one line: the question text cut to fit, then the whole-question figures. */
+ private static Row questionRow(QuestionGroup g, boolean multi, boolean expanded, int questionWidth) {
+ String marker = multi ? (expanded ? TuiIcons.MORE_CHEVRON : TuiIcons.ARROW_RIGHT) : " ";
+ String source;
+ Style sourceStyle;
+ if (g.source() == RequestSource.ROUTE) {
+ source = g.routeId() != null ? "route:" + g.routeId() : "route";
+ sourceStyle = Theme.notice();
+ } else {
+ source = (g.question() > 0 ? "#" + g.question() : "tui") + (multi ? " ×" + g.steps().size() : "");
+ sourceStyle = Theme.info();
+ }
+ String text = g.questionText() != null && !g.questionText().isBlank()
+ ? firstLine(g.questionText())
+ : g.last().model();
+ int ctx = g.contextPercent();
+ return Row.from(
+ Cell.from(Span.styled(marker + TIME.format(g.first().timestamp()), Theme.muted())),
+ Cell.from(Span.styled(source, sourceStyle)),
+ Cell.from(Span.styled(TuiHelper.truncate(text, questionWidth), Theme.label())),
+ rightCell(formatTokens(g.promptTokens()), 6),
+ rightCell(formatTokens(g.outputTokens()), 6),
+ rightCell(g.cacheHitPercent() + "%", 6, Theme.muted()),
+ rightCell(ctx >= 0 ? ctx + "%" : "-", 5,
+ ctx >= 80 ? Theme.error() : ctx >= 50 ? Theme.warning() : Style.EMPTY),
+ rightCell(g.prefillTokensPerSecond() > 0 ? formatRate(g.prefillTokensPerSecond()) : "-", 8,
+ Theme.info()),
+ rightCell(g.decodeTokensPerSecond() > 0 ? formatRate(g.decodeTokensPerSecond()) : "-", 8,
+ Theme.success()),
+ rightCell(g.hasTimings() ? formatSeconds(g.ttftMs()) : "-", 7,
+ g.coldStart() ? Theme.warning() : Style.EMPTY),
+ rightCell(g.wallMs() > 0 ? formatSeconds(g.wallMs()) : "-", 7),
+ Cell.from(Span.styled(" " + (g.doneReason() != null ? g.doneReason() : ""), Theme.muted())));
+ }
+
+ /** One request of an unfolded question: step number, the model, and that request's own figures. */
+ private static Row stepRow(RequestEntry e, int step, int of) {
+ int ctx = e.contextPercent();
+ return Row.from(
+ Cell.from(Span.styled(" " + TIME.format(e.timestamp()), Theme.muted().dim())),
+ Cell.from(Span.styled(" step " + step + "/" + of, Theme.muted())),
+ Cell.from(Span.styled(e.model(), Theme.muted().dim())),
+ rightCell(formatTokens(e.inputTokens()), 6, Theme.muted()),
+ rightCell(formatTokens(e.outputTokens()), 6, Theme.muted()),
+ rightCell(e.cachedTokens() > 0 ? e.cacheHitPercent() + "%" : "-", 6, Theme.muted()),
+ rightCell(ctx >= 0 ? ctx + "%" : "-", 5, Theme.muted()),
+ rightCell(e.prefillMs() > 0 ? formatRate(e.prefillTokensPerSecond()) : "-", 8, Theme.muted()),
+ rightCell(e.decodeMs() > 0 ? formatRate(e.decodeTokensPerSecond()) : "-", 8, Theme.muted()),
+ rightCell(e.hasTimings() ? formatSeconds(e.ttftMs()) : "-", 7,
+ e.coldStart() ? Theme.warning() : Theme.muted()),
+ rightCell(e.totalMs() > 0 ? formatSeconds(e.totalMs()) : "-", 7, Theme.muted()),
+ Cell.from(Span.styled(" " + (e.doneReason() != null ? e.doneReason() : ""), Theme.muted().dim())));
+ }
+
+ static String firstLine(String text) {
+ String t = text.strip();
+ int nl = t.indexOf('\n');
+ if (nl >= 0) {
+ t = t.substring(0, nl).strip() + " …";
+ }
+ return t.replace('\t', ' ');
+ }
+
// ---- MCP / table data ----
@Override
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
index e63b78c069fb..bc3bc9ae11f0 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
@@ -108,22 +108,28 @@ Shown when Ollama runs on this machine:
## Requests
-One line per request, newest first:
-
-| Column | Meaning |
-|--------|---------|
-| TIME | When the request started |
-| SOURCE | `tui` for the AI panel, `route:<id>` for a Camel route |
-| MODEL | The model that answered |
-| IN | Prompt tokens, the whole prompt the model saw (CACHED is the part of it served from cache) |
-| OUT | Tokens generated |
-| CACHED | Prompt tokens served from Ollama's cache (`-` when none) |
-| CTX | Share of the context window the prompt filled (IN against the window that served it) |
-| PREFILL | Evaluated prompt tokens per second (IN minus CACHED over the prefill time) |
-| DECODE | Generated tokens per second |
-| TTFT | Time to first token (load plus prefill), shown in yellow after a cold start |
-| TOTAL | Whole request as Ollama measured it |
-| REASON | Why generation stopped: `stop`, `length` (hit the token limit), `tool_calls` |
+One line per question, newest first. A question asked in the AI panel
+usually costs several requests, because every tool call the model makes
+is answered in a new request with the whole prompt again; the line shows
+the question text (cut to fit) and the figures for the question as a
+whole. Press **Enter** (or **→**) on a question with `×N` in the SOURCE
+column to unfold its steps, **←** to fold them again. A call made by a
+Camel route is one line of its own.
+
+| Column | Question line | Step line |
+|--------|---------------|-----------|
+| TIME | When the first request started | When the request started |
+| SOURCE | `#7 ×10`: question number and request count, or `route:<id>` | `step 3/10` |
+| QUESTION | The question, first line cut to fit (the model for a route) | The model that answered |
+| IN | The largest prompt of the question: how far the context was pushed | Prompt tokens, the whole prompt the model saw |
+| OUT | Tokens generated across all steps | Tokens generated |
+| CACHE | Share of all prompt tokens served from Ollama's cache | Share of this prompt served from cache (`-` when none) |
+| CTX | Highest share of the context window reached | Share of the context window this prompt filled |
+| PREFILL | Evaluated prompt tokens per second across the steps (prompt minus cached over the prefill time) | Same for this request |
+| DECODE | Generated tokens per second across the steps | Same for this request |
+| TTFT | Time to the first token of the first step (load plus prefill), yellow after a cold start | Time to first token of this request |
+| TOTAL | From the first request starting to the last finishing: what you waited | This request as Ollama measured it |
+| REASON | Why the last step stopped: `stop`, `length` (hit the token limit) | `tool_calls` for every step but the last |
Route requests show `-` for prefill, decode and TTFT because the GenAI
span carries tokens and duration only.
@@ -139,7 +145,9 @@ tab says so and keeps the rest.
| Key | Action |
|-----|--------|
-| ↑ / ↓ | Select a request |
+| ↑ / ↓ | Select a question or step |
+| Enter / → | Unfold the steps of a question |
+| ← | Fold them again |
| r | Reset the request log and session totals |
| F5 | Refresh now |
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
index 60589f1933e6..04aeec6de046 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
@@ -204,6 +204,60 @@ class OllamaMonitorTest {
assertEquals(0, history[0]);
}
+ @Test
+ void requestsGroupIntoQuestionsWithStepsOldestFirst() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ monitor.updateModels(List.of(new OllamaMonitor.LoadedModel(
+ "m", "f", "35.5B", "Q4_K_M", 1, 1, 32_768, null, null)));
+ // question 7: three steps (two tool calls, then the answer); the prompt grows and the cache warms
+ // the first step loaded the model: 1.1 s load plus 5.1 s prefill
+ monitor.recordRequest("m", new LlmClient.TokenUsage(4_500, 33, 4_533, 0, 5_100, 700, 1_100, 6_900), 0,
+ "tool_calls", 7, "how many messages have camel done");
+ monitor.recordRequest("m", new LlmClient.TokenUsage(4_700, 78, 4_778, 4_500, 500, 1_600, 0, 1_900), 0,
+ "tool_calls", 7, "how many messages have camel done");
+ monitor.recordRequest("m", new LlmClient.TokenUsage(7_000, 106, 7_106, 4_700, 3_600, 1_900, 0, 5_500), 0,
+ "stop", 7, "how many messages have camel done");
+ // question 8: a single request
+ monitor.recordRequest("m", new LlmClient.TokenUsage(7_200, 40, 7_240, 7_000, 300, 700, 0, 1_000), 0,
+ "stop", 8, "thanks");
+ // and a route call
+ monitor.ingestSpans(List.of(genAiSpan("s9", "ollama", "m", "chat-route", 412, 180, 4100)));
+
+ List<OllamaMonitor.QuestionGroup> groups = OllamaMonitor.groupByQuestion(monitor.snapshot().requests());
+ assertEquals(3, groups.size());
+ OllamaMonitor.QuestionGroup route = groups.get(0);
+ assertEquals(RequestSource.ROUTE, route.source());
+ assertEquals(1, route.steps().size());
+ assertEquals("chat-route", route.routeId());
+
+ OllamaMonitor.QuestionGroup q8 = groups.get(1);
+ assertEquals(8, q8.question());
+ assertEquals("thanks", q8.questionText());
+ assertEquals(1, q8.steps().size());
+
+ OllamaMonitor.QuestionGroup q7 = groups.get(2);
+ assertEquals(7, q7.question());
+ assertEquals(3, q7.steps().size());
+ assertEquals(33, q7.steps().get(0).outputTokens(), "steps are oldest first");
+ assertEquals(7_000, q7.promptTokens());
+ assertEquals(33 + 78 + 106, q7.outputTokens());
+ // 9,200 of 16,200 prompt tokens came from the cache
+ assertEquals(56, q7.cacheHitPercent());
+ assertEquals(21, q7.contextPercent());
+ assertEquals(6_200, q7.ttftMs());
+ assertTrue(q7.coldStart());
+ assertEquals("stop", q7.doneReason());
+ // evaluated 4,500 + 200 + 2,300 tokens over 9.2 s of prefill
+ assertEquals(7_000 * 1000.0 / 9_200, q7.prefillTokensPerSecond(), 0.5);
+ assertTrue(q7.wallMs() >= 5_500);
+
+ JsonArray questions = (JsonArray) monitor.toJson(10).get("questions");
+ assertEquals(3, questions.size());
+ JsonObject jq7 = (JsonObject) questions.get(2);
+ assertEquals(3, jq7.get("steps"));
+ assertEquals("how many messages have camel done", jq7.get("questionText"));
+ }
+
@Test
void availabilityFollowsTheServer() {
OllamaMonitor monitor = new OllamaMonitor();
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
index d90b43d5dabf..8a77262cbae3 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
@@ -22,6 +22,9 @@ import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicReference;
+import dev.tamboui.tui.event.KeyCode;
+import dev.tamboui.tui.event.KeyEvent;
+import dev.tamboui.tui.event.KeyModifiers;
import org.apache.camel.dsl.jbang.core.commands.LlmClient;
import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.GpuStats;
import org.apache.camel.dsl.jbang.core.commands.tui.OllamaMonitor.HostStats;
@@ -110,7 +113,7 @@ class OllamaTabRenderTest {
assertTrue(rendered.contains("llama-server"), rendered);
assertTrue(rendered.contains("ollama serve"), rendered);
// request log
- assertTrue(rendered.contains("Requests (1)"), rendered);
+ assertTrue(rendered.contains("Requests (1 question, 1 request)"), rendered);
assertTrue(rendered.contains("CTX"), rendered);
assertTrue(rendered.contains("PREFILL"), rendered);
assertTrue(rendered.contains("DECODE"), rendered);
@@ -155,6 +158,38 @@ class OllamaTabRenderTest {
assertTrue(rendered.contains("2 installed: qwen3.6:35b-a3b, llama3.2:latest"), rendered);
}
+ @Test
+ void requestsAreGroupedPerQuestionAndUnfoldOnEnter() {
+ localServerWithModel();
+ String question = "how many messages have camel done\nand are any failing?";
+ monitor.recordRequest("qwen3.6:35b-a3b", new LlmClient.TokenUsage(4_500, 33, 4_533, 0, 620, 700, 0, 1_320), 0,
+ "tool_calls", 7, question);
+ monitor.recordRequest("qwen3.6:35b-a3b", new LlmClient.TokenUsage(4_700, 78, 4_778, 4_500, 50, 1_600, 0, 1_700), 0,
+ "tool_calls", 7, question);
+ monitor.recordRequest("qwen3.6:35b-a3b", new LlmClient.TokenUsage(7_000, 106, 7_106, 4_700, 360, 1_900, 0, 2_300), 0,
+ "stop", 7, question);
+
+ OllamaTab tab = new OllamaTab(ctx, monitor);
+ String rendered = TuiTestHelper.renderToString(tab, 200, 40);
+ assertTrue(rendered.contains("Requests (1 question, 3 requests)"), rendered);
+ assertTrue(rendered.contains("#7 ×3"), rendered);
+ assertTrue(rendered.contains("how many messages have camel done …"), rendered);
+ assertTrue(rendered.contains("QUESTION"), rendered);
+ assertFalse(rendered.contains("step 1/3"), rendered);
+
+ // Enter on the question unfolds its steps
+ tab.navigateDown();
+ tab.handleKeyEvent(KeyEvent.ofKey(KeyCode.ENTER, KeyModifiers.NONE));
+ rendered = TuiTestHelper.renderToString(tab, 200, 40);
+ assertTrue(rendered.contains("step 1/3"), rendered);
+ assertTrue(rendered.contains("step 3/3"), rendered);
+
+ // and folds them again
+ tab.handleKeyEvent(KeyEvent.ofKey(KeyCode.ENTER, KeyModifiers.NONE));
+ rendered = TuiTestHelper.renderToString(tab, 200, 40);
+ assertFalse(rendered.contains("step 1/3"), rendered);
+ }
+
@Test
void routeRequestsShowTheirRouteId() {
localServerWithModel();
@@ -164,7 +199,7 @@ class OllamaTabRenderTest {
String rendered = TuiTestHelper.renderToString(new OllamaTab(ctx, monitor), 180, 40);
assertTrue(rendered.contains("route:chat-route"), rendered);
- assertTrue(rendered.contains("Requests (2)"), rendered);
+ assertTrue(rendered.contains("Requests (2 questions, 2 requests)"), rendered);
}
@Test
@@ -205,7 +240,8 @@ class OllamaTabRenderTest {
void helpTextExplainsThePhasesAndColumns() {
String help = new OllamaTab(ctx, monitor).getHelpText();
assertTrue(help != null && help.contains("TTFT"), help);
- for (String term : List.of("Prefill", "Decode", "cold", "cache hit", "speculative", "CACHED", "REASON")) {
+ for (String term : List.of("Prefill", "Decode", "cold", "cache hit", "speculative", "CACHE", "QUESTION",
+ "REASON")) {
assertTrue(help.contains(term), "help should explain " + term);
}
}
(camel) 03/09: CAMEL-24794: camel-jbang - TUI Ollama tab shows context fill per turn and compactions
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch tui-ollama-tab
in repository https://gitbox.apache.org/repos/asf/camel.git
commit a3ed5ea24f16f8d6d2d2fe0bbbd047022ff03351
Author: Claus Ibsen <cl...@gmail.com>
AuthorDate: Thu Sep 17 12:20:03 2026 +0200
CAMEL-24794: camel-jbang - TUI Ollama tab shows context fill per turn and compactions
Each request now carries the context window of the runner that served it
(the slot size, else the loaded model's allocated context, fetched on the
spot when the tab has not polled yet). The request log gains a CTX column
with the share of the window the prompt filled (evaluated plus cached
tokens), yellow from 50% and red from 80%.
The Context panel gains a "turns" line: one bar per AI panel turn scaled to
the whole window, so a growing conversation and the drop of a compaction
are visible at a glance, followed by the latest fill, the session peak and
the number of compactions seen. A prompt a fifth smaller than the previous
turn counts as a compaction. The same figures are in tui_get_ollama
(promptTokens, contextSize, contextPercent per request; peakContextPercent
and compactions in the session block), the F1 help and the user manual.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Signed-off-by: Claus Ibsen <cl...@gmail.com>
---
.../modules/ROOT/pages/camel-jbang-tui.adoc | 8 +-
.../dsl/jbang/core/commands/tui/OllamaMonitor.java | 100 +++++++++++++++++++--
.../dsl/jbang/core/commands/tui/OllamaTab.java | 60 ++++++++++++-
.../src/main/resources/tui/help/ollama.md | 8 ++
.../jbang/core/commands/tui/OllamaMonitorTest.java | 31 +++++++
.../core/commands/tui/OllamaTabRenderTest.java | 26 +++++-
6 files changed, 218 insertions(+), 15 deletions(-)
diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
index f8c97d5ced6f..5851b3104896 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
@@ -427,11 +427,13 @@ or at the endpoint the AI panel (*F8*) is using.
otherwise from the last request; time to first token and load time (a load of a second or more
is a cold start); session averages and a sparkline of the decode rate.
* *Context* -- how full the context window is, the share of the prompt served from Ollama's cache,
- whether the model is working or idle, and the speculative decoding method in use.
+ whether the model is working or idle, the speculative decoding method in use, and a per-turn trend
+ of how much of the window each AI panel prompt filled, with the session peak and the compactions seen.
* *Host* -- GPU utilization and memory (Apple silicon through `ioreg`, NVIDIA through `nvidia-smi`),
and CPU and memory of the Ollama server and its model runner.
-* *Requests* -- one line per request with input, output and cached tokens, prefill and decode
- tokens per second, time to first token, total time and the stop reason.
+* *Requests* -- one line per request with input, output and cached tokens, the share of the context
+ window the prompt filled, prefill and decode tokens per second, time to first token, total time and
+ the stop reason.
Two kinds of requests appear in the log. Questions asked in the AI panel with Ollama as the provider
come with the timings Ollama reports for each request (prompt evaluation, generation, model load,
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
index 0b6cca44a239..ceaf4c63dd42 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
@@ -140,10 +140,26 @@ final class OllamaMonitor {
ROUTE
}
- /** One request as Ollama reported it (TUI) or as the GenAI span of a route recorded it (no phase timings). */
+ /**
+ * One request as Ollama reported it (TUI) or as the GenAI span of a route recorded it (no phase timings).
+ * {@code contextSize} is the context window of the runner that served it, 0 when unknown.
+ */
record RequestEntry(Instant timestamp, RequestSource source, String routeId, String model, int inputTokens,
int outputTokens, int cachedTokens, long prefillMs, long decodeMs, long loadMs, long totalMs,
- String doneReason) {
+ String doneReason, long contextSize) {
+
+ /** Everything the model had in front of it: prompt tokens evaluated plus those served from cache. */
+ long promptTokens() {
+ return (long) inputTokens + cachedTokens;
+ }
+
+ /** Share of the context window the prompt filled, or -1 when the window is unknown. */
+ int contextPercent() {
+ if (contextSize <= 0) {
+ return -1;
+ }
+ return (int) Math.min(100, promptTokens() * 100 / contextSize);
+ }
double prefillTokensPerSecond() {
return prefillMs > 0 ? inputTokens * 1000.0 / prefillMs : 0;
@@ -169,15 +185,16 @@ final class OllamaMonitor {
}
record SessionTotals(int requests, long inputTokens, long outputTokens, long cachedTokens, long prefillMs,
- long decodeMs, long loadMs, int coldStarts) {
+ long decodeMs, long loadMs, int coldStarts, int peakContextPercent, int compactions) {
- static final SessionTotals EMPTY = new SessionTotals(0, 0, 0, 0, 0, 0, 0, 0);
+ static final SessionTotals EMPTY = new SessionTotals(0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
- SessionTotals plus(RequestEntry e) {
+ SessionTotals plus(RequestEntry e, boolean compaction) {
return new SessionTotals(
requests + 1, inputTokens + e.inputTokens(), outputTokens + e.outputTokens(),
cachedTokens + e.cachedTokens(), prefillMs + e.prefillMs(), decodeMs + e.decodeMs(),
- loadMs + e.loadMs(), coldStarts + (e.coldStart() ? 1 : 0));
+ loadMs + e.loadMs(), coldStarts + (e.coldStart() ? 1 : 0),
+ Math.max(peakContextPercent, e.contextPercent()), compactions + (compaction ? 1 : 0));
}
double avgDecodeTokensPerSecond() {
@@ -227,6 +244,7 @@ final class OllamaMonitor {
private final Deque<RequestEntry> requests = new ArrayDeque<>();
private final LinkedHashSet<String> seenSpanIds = new LinkedHashSet<>();
private SessionTotals totals = SessionTotals.EMPTY;
+ private long lastTuiPromptTokens;
private final TokenRateWindow decodeWindow = new TokenRateWindow(RATE_WINDOW_MS);
private final TokenRateWindow prefillWindow = new TokenRateWindow(RATE_WINDOW_MS);
private final long[] decodeHistory = new long[HISTORY_POINTS];
@@ -272,12 +290,53 @@ final class OllamaMonitor {
RequestEntry entry = new RequestEntry(
Instant.now(), RequestSource.TUI, null, model != null ? model : "unknown",
usage.inputTokens(), usage.outputTokens(), usage.cachedTokens(),
- usage.prefillMillis(), usage.generationMillis(), usage.loadMillis(), total, doneReason);
+ usage.prefillMillis(), usage.generationMillis(), usage.loadMillis(), total, doneReason,
+ contextSizeForNewRequest());
synchronized (lock) {
addRequest(entry);
}
}
+ /**
+ * The context window that served a request that just finished: the runner's slot size when known, else the
+ * allocated context of the loaded model, fetched on the spot when the tab has not polled yet. A request with a
+ * different {@code num_ctx} makes Ollama reload before answering, so what is loaded after the reply is what served
+ * it.
+ */
+ private long contextSizeForNewRequest() {
+ long known = knownContextSize();
+ if (known > 0) {
+ return known;
+ }
+ String base = baseUrl;
+ if (base != null) {
+ JsonObject ps = getJsonObject(base + "/api/ps");
+ if (ps != null) {
+ List<LoadedModel> loaded = OllamaParsers.parsePs(ps);
+ if (!loaded.isEmpty()) {
+ synchronized (lock) {
+ List<LoadedModel> withShapes = new ArrayList<>();
+ for (LoadedModel m : loaded) {
+ withShapes.add(m.withShape(shapes.get(m.name())));
+ }
+ models = List.copyOf(withShapes);
+ }
+ return loaded.get(0).contextLength();
+ }
+ }
+ }
+ return 0;
+ }
+
+ private long knownContextSize() {
+ synchronized (lock) {
+ if (slot != null && slot.contextSize() > 0) {
+ return slot.contextSize();
+ }
+ return models.isEmpty() ? 0 : models.get(0).contextLength();
+ }
+ }
+
/**
* Adds the Ollama calls Camel routes made, from the GenAI observability spans of the selected integration. Spans
* are deduplicated by id, so the same list can be handed over on every refresh.
@@ -313,7 +372,7 @@ final class OllamaMonitor {
(int) OllamaParsers.num(attrs, GenAiAttributes.INPUT_TOKENS),
(int) OllamaParsers.num(attrs, GenAiAttributes.OUTPUT_TOKENS),
0, 0, 0, 0, Math.max(0, span.durationMs()),
- OllamaParsers.str(attrs, GenAiAttributes.FINISH_REASONS)));
+ OllamaParsers.str(attrs, GenAiAttributes.FINISH_REASONS), knownContextSizeLocked()));
}
// spans arrive oldest first; keep the log newest first
fresh.sort((a, b) -> a.timestamp().compareTo(b.timestamp()));
@@ -341,18 +400,36 @@ final class OllamaMonitor {
synchronized (lock) {
requests.clear();
totals = SessionTotals.EMPTY;
+ lastTuiPromptTokens = 0;
Arrays.fill(decodeHistory, 0);
decodeWindow.clear();
prefillWindow.clear();
}
}
+ /** As {@link #knownContextSize()} for callers already holding the lock. */
+ private long knownContextSizeLocked() {
+ if (slot != null && slot.contextSize() > 0) {
+ return slot.contextSize();
+ }
+ return models.isEmpty() ? 0 : models.get(0).contextLength();
+ }
+
private void addRequest(RequestEntry entry) {
+ // an AI panel prompt that shrinks by a fifth or more against the previous turn means the history was
+ // compacted (or a new conversation started); either way the context was freed
+ boolean compaction = false;
+ if (entry.source() == RequestSource.TUI) {
+ if (lastTuiPromptTokens > 0 && entry.promptTokens() < lastTuiPromptTokens * 0.8) {
+ compaction = true;
+ }
+ lastTuiPromptTokens = entry.promptTokens();
+ }
requests.addFirst(entry);
while (requests.size() > MAX_REQUESTS) {
requests.removeLast();
}
- totals = totals.plus(entry);
+ totals = totals.plus(entry, compaction);
}
// ---- state updates (also the test seam) ----
@@ -870,6 +947,8 @@ final class OllamaMonitor {
session.put("avgDecodeTokensPerSecond", round1(s.totals().avgDecodeTokensPerSecond()));
session.put("avgPrefillTokensPerSecond", round1(s.totals().avgPrefillTokensPerSecond()));
session.put("coldStarts", s.totals().coldStarts());
+ session.put("peakContextPercent", s.totals().peakContextPercent());
+ session.put("compactions", s.totals().compactions());
root.put("session", session);
JsonArray reqs = new JsonArray();
@@ -906,6 +985,9 @@ final class OllamaMonitor {
r.put("decodeTokensPerSecond", round1(e.decodeTokensPerSecond()));
r.put("ttftMs", e.ttftMs());
r.put("coldStart", e.coldStart());
+ r.put("promptTokens", e.promptTokens());
+ r.put("contextSize", e.contextSize());
+ r.put("contextPercent", e.contextPercent());
if (e.doneReason() != null) {
r.put("doneReason", e.doneReason());
}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
index df6e7d259e61..e532e60b0e79 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
@@ -337,6 +337,10 @@ class OllamaTab extends AbstractTab {
state.add(Span.styled(" · speculative " + slot.speculative(), Theme.muted()));
}
lines.add(Line.from(state));
+ Line trend = contextTrendLine(s, block.inner(area).width());
+ if (trend != null) {
+ lines.add(trend);
+ }
} else {
RequestEntry last = s.lastRequest();
long ctx = model != null ? model.contextLength() : 0;
@@ -357,10 +361,47 @@ class OllamaTab extends AbstractTab {
lines.add(Line.from(Span.styled(s.local()
? " live state appears once the runner is found"
: " live state needs the runner on this machine", Theme.muted().dim())));
+ Line trend = contextTrendLine(s, block.inner(area).width());
+ if (trend != null) {
+ lines.add(trend);
+ }
}
frame.renderWidget(Paragraph.builder().text(Text.from(lines)).block(block).build(), area);
}
+ /**
+ * How full the context was on each of the last AI panel turns, oldest first, scaled to the whole window so the bars
+ * are comparable across turns; a drop between bars is a compaction. Null when no turn has a known window.
+ */
+ static Line contextTrendLine(Snapshot s, int width) {
+ List<RequestEntry> turns = new ArrayList<>();
+ for (RequestEntry e : s.requests()) {
+ if (e.source() == RequestSource.TUI && e.contextPercent() >= 0) {
+ turns.add(e);
+ }
+ }
+ if (turns.isEmpty()) {
+ return null;
+ }
+ int last = turns.get(0).contextPercent();
+ SessionTotals t = s.totals();
+ String suffix = " " + last + "%" + (t.peakContextPercent() > last ? " (peak " + t.peakContextPercent() + "%)" : "")
+ + (t.compactions() > 0
+ ? " · " + t.compactions() + (t.compactions() == 1 ? " compaction" : " compactions")
+ : "");
+ int barWidth = Math.max(4, Math.min(24, width - 8 - suffix.length()));
+ int n = Math.min(barWidth, turns.size());
+ long[] data = new long[n];
+ for (int i = 0; i < n; i++) {
+ data[n - 1 - i] = turns.get(i).contextPercent();
+ }
+ Style level = last >= 80 ? Theme.error() : last >= 50 ? Theme.warning() : Theme.info();
+ return Line.from(
+ Span.styled(" turns ", Theme.muted()),
+ Span.styled(sparkline(data, barWidth, 100), level),
+ Span.styled(suffix, level));
+ }
+
private void renderHost(Frame frame, Rect area, Snapshot s) {
List<Line> lines = new ArrayList<>();
HostStats host = s.host();
@@ -419,6 +460,9 @@ class OllamaTab extends AbstractTab {
rightCell(formatTokens(e.inputTokens()), 6),
rightCell(formatTokens(e.outputTokens()), 6),
rightCell(e.cachedTokens() > 0 ? formatTokens(e.cachedTokens()) : "-", 6, Theme.muted()),
+ rightCell(e.contextPercent() >= 0 ? e.contextPercent() + "%" : "-", 5,
+ e.contextPercent() >= 80 ? Theme.error() : e.contextPercent() >= 50 ? Theme.warning()
+ : Style.EMPTY),
rightCell(e.prefillMs() > 0 ? formatRate(e.prefillTokensPerSecond()) : "-", 8, Theme.info()),
rightCell(e.decodeMs() > 0 ? formatRate(e.decodeTokensPerSecond()) : "-", 8, Theme.success()),
rightCell(e.hasTimings() ? formatSeconds(e.ttftMs()) : "-", 7,
@@ -448,6 +492,7 @@ class OllamaTab extends AbstractTab {
rightCell("IN", 6, Style.EMPTY.bold()),
rightCell("OUT", 6, Style.EMPTY.bold()),
rightCell("CACHED", 6, Style.EMPTY.bold()),
+ rightCell("CTX", 5, Style.EMPTY.bold()),
rightCell("PREFILL", 8, Style.EMPTY.bold()),
rightCell("DECODE", 8, Style.EMPTY.bold()),
rightCell("TTFT", 7, Style.EMPTY.bold()),
@@ -460,6 +505,7 @@ class OllamaTab extends AbstractTab {
Constraint.length(6),
Constraint.length(6),
Constraint.length(6),
+ Constraint.length(5),
Constraint.length(8),
Constraint.length(8),
Constraint.length(7),
@@ -468,7 +514,9 @@ class OllamaTab extends AbstractTab {
.highlightStyle(Theme.selectionBg())
.highlightSpacing(Table.HighlightSpacing.ALWAYS)
.block(Block.builder().borderType(BorderType.ROUNDED).borders(Borders.ALL)
- .title(" Requests (" + requests.size() + ") tok/s for prefill and decode ").build())
+ .title(" Requests (" + requests.size()
+ + ") tok/s for prefill and decode · CTX = prompt share of the context window ")
+ .build())
.build();
lastTableArea = area;
frame.renderStatefulWidget(table, area, tableState);
@@ -635,13 +683,21 @@ class OllamaTab extends AbstractTab {
/** Right-aligned one-row sparkline scaled to the largest value; an all-zero series renders as spaces. */
static String sparkline(long[] data, int width) {
- if (data == null || width <= 0) {
+ if (data == null) {
return "";
}
long max = 0;
for (long v : data) {
max = Math.max(max, v);
}
+ return sparkline(data, width, max);
+ }
+
+ /** As {@link #sparkline(long[], int)} with a fixed scale, so bars stay comparable across renders. */
+ static String sparkline(long[] data, int width, long max) {
+ if (data == null || width <= 0) {
+ return "";
+ }
StringBuilder sb = new StringBuilder();
int start = Math.max(0, data.length - width);
for (int i = 0; i < width - Math.min(width, data.length); i++) {
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
index 8e75cc50ca53..63af33fc2a0f 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
@@ -79,6 +79,13 @@ loads on the first request.
- **speculative** — the speculative decoding method the runner uses
(`draft-mtp` is multi-token prediction), which is why decode can
exceed one token per step.
+- **turns** — one bar per AI panel turn, oldest first, scaled to the
+ whole context window: how much of the window each turn's prompt
+ filled (system prompt, tool definitions, history, question). The
+ bars grow as a conversation continues and drop when the history is
+ compacted; the line ends with the latest fill, the session peak and
+ the number of compactions seen. Yellow from 50%, red from 80%: that is
+ when `/compact` in the AI panel, or a smaller toolset, pays off.
Without the local runner the panel falls back to the last request's
tokens against the model's context length.
@@ -106,6 +113,7 @@ One line per request, newest first:
| IN | Prompt tokens evaluated |
| OUT | Tokens generated |
| CACHED | Prompt tokens served from Ollama's cache (`-` when none) |
+| CTX | Share of the context window the prompt filled (IN plus CACHED against the window that served it) |
| PREFILL | Prompt tokens per second |
| DECODE | Generated tokens per second |
| TTFT | Time to first token (load plus prefill), shown in yellow after a cold start |
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
index cb2b325e32d9..bd8d7c1ecf3e 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
@@ -136,6 +136,37 @@ class OllamaMonitorTest {
assertEquals(4100, e.totalMs());
assertFalse(e.hasTimings());
assertEquals(0, e.ttftMs());
+ // no runner slot and no model polled yet: the window is unknown
+ assertEquals(0, e.contextSize());
+ assertEquals(-1, e.contextPercent());
+ }
+
+ @Test
+ void contextFillComesFromTheSlotOrTheLoadedModelAndCompactionsAreCounted() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ monitor.updateServer(new ServerInfo("http://localhost:11434", "0.33.3", true));
+ monitor.updateModels(List.of(new OllamaMonitor.LoadedModel(
+ "m", "f", "35.5B", "Q4_K_M", 1, 1, 32_768, null, null)));
+ monitor.recordRequest("m", new LlmClient.TokenUsage(4_000, 60, 4_060, 500, 900, 800, 0, 1800), 0, "stop");
+ RequestEntry first = monitor.snapshot().lastRequest();
+ assertEquals(32_768, first.contextSize());
+ assertEquals(4_500, first.promptTokens());
+ assertEquals(13, first.contextPercent());
+
+ // the runner's slot wins over the model list when present
+ monitor.updateSlot(slot(false, 0, System.currentTimeMillis()));
+ monitor.recordRequest("m", new LlmClient.TokenUsage(20_000, 60, 20_060, 0, 900, 800, 0, 1800), 0, "stop");
+ assertEquals(262144, monitor.snapshot().lastRequest().contextSize());
+ assertEquals(7, monitor.snapshot().lastRequest().contextPercent());
+ assertEquals(13, monitor.snapshot().totals().peakContextPercent());
+ assertEquals(0, monitor.snapshot().totals().compactions());
+
+ // a prompt a fifth smaller than the previous turn counts as a compaction
+ monitor.recordRequest("m", new LlmClient.TokenUsage(9_000, 60, 9_060, 0, 900, 800, 0, 1800), 0, "stop");
+ assertEquals(1, monitor.snapshot().totals().compactions());
+ JsonObject last = (JsonObject) ((JsonArray) monitor.toJson(1).get("requests")).get(0);
+ assertEquals(9_000L, last.get("promptTokens"));
+ assertEquals(3, last.get("contextPercent"));
}
@Test
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
index 46d7d515b5d0..668c73bae5ed 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
@@ -110,6 +110,7 @@ class OllamaTabRenderTest {
assertTrue(rendered.contains("ollama serve"), rendered);
// request log
assertTrue(rendered.contains("Requests (1)"), rendered);
+ assertTrue(rendered.contains("CTX"), rendered);
assertTrue(rendered.contains("PREFILL"), rendered);
assertTrue(rendered.contains("DECODE"), rendered);
assertTrue(rendered.contains("tui"), rendered);
@@ -169,6 +170,24 @@ class OllamaTabRenderTest {
assertEquals(1, ((JsonArray) summary.get("loadedModels")).size());
}
+ @Test
+ void contextTrendShowsTurnsPeakAndCompactions() {
+ localServerWithModel(); // ctx 262,144
+ // four turns: the prompt grows, then a compaction frees most of it
+ monitor.recordRequest("m", usage(26_000), 0, "stop");
+ monitor.recordRequest("m", usage(52_000), 0, "stop");
+ monitor.recordRequest("m", usage(131_000), 0, "stop");
+ monitor.recordRequest("m", usage(39_000), 0, "stop");
+
+ String rendered = TuiTestHelper.renderToString(new OllamaTab(ctx, monitor), 200, 40);
+ assertTrue(rendered.contains("turns"), rendered);
+ assertTrue(rendered.contains("14% (peak 49%) · 1 compaction"), rendered);
+ // the CTX column of the biggest turn
+ assertTrue(rendered.contains("49%"), rendered);
+ assertEquals(49, monitor.snapshot().totals().peakContextPercent());
+ assertEquals(1, monitor.snapshot().totals().compactions());
+ }
+
@Test
void helpTextExplainsThePhasesAndColumns() {
String help = new OllamaTab(ctx, monitor).getHelpText();
@@ -197,6 +216,7 @@ class OllamaTabRenderTest {
assertEquals(" ▁▄█", OllamaTab.sparkline(new long[] { 10, 40, 80 }, 5));
assertEquals(" ", OllamaTab.sparkline(new long[] { 0, 0 }, 4));
assertEquals("▁█", OllamaTab.sparkline(new long[] { 1, 2, 3, 20 }, 2));
+ assertEquals("▁▄█", OllamaTab.sparkline(new long[] { 10, 50, 100 }, 3, 100));
Instant now = Instant.parse("2026-09-17T09:18:12Z");
assertEquals("unloads in 4m32s", OllamaTab.formatCountdown(Instant.parse("2026-09-17T09:22:44Z"), now));
assertEquals("unloads in 2h5m", OllamaTab.formatCountdown(now.plusSeconds(2 * 3600 + 300), now));
@@ -219,6 +239,10 @@ class OllamaTabRenderTest {
monitor.updateModels(List.of(model()));
}
+ private static LlmClient.TokenUsage usage(int promptTokens) {
+ return new LlmClient.TokenUsage(promptTokens, 50, promptTokens + 50, 0, 900, 800, 10, 1800);
+ }
+
private static LoadedModel model() {
return new LoadedModel(
"qwen3.6:35b-a3b", "qwen35moe", "35.5B", "Q4_K_M", 23567972432L, 23567972432L, 262144,
@@ -231,7 +255,7 @@ class OllamaTabRenderTest {
private static RequestEntry request(int decodeTokensPerSecond) {
return new RequestEntry(
Instant.now(), RequestSource.TUI, null, "m", 10, decodeTokensPerSecond, 0, 100, 1000,
- 0, 1100, "stop");
+ 0, 1100, "stop", 0);
}
private static SpanEntry routeSpan() {
(camel) 07/09: CAMEL-24794: camel-jbang - TUI Ollama tab Context panel reads the cache hit from the last request when idle
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch tui-ollama-tab
in repository https://gitbox.apache.org/repos/asf/camel.git
commit c58cdae15641e40bf6453b31610714263506bcfe
Author: Claus Ibsen <cl...@gmail.com>
AuthorDate: Thu Sep 17 12:48:51 2026 +0200
CAMEL-24794: camel-jbang - TUI Ollama tab Context panel reads the cache hit from the last request when idle
llama-server clears the slot's cache count once a request completes, so the
panel showed "cache hit 0%" beside a log whose last row was 97% cached. When
the runner is idle the prompt size and cache hit now come from the last
request (marked as such); while it is working the live slot figures stay.
The turns bars start right after the label instead of being right-aligned
inside a fixed width.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Signed-off-by: Claus Ibsen <cl...@gmail.com>
---
.../camel/dsl/jbang/core/commands/tui/OllamaMonitor.java | 8 ++++++++
.../camel/dsl/jbang/core/commands/tui/OllamaTab.java | 16 ++++++++++++----
.../dsl/jbang/core/commands/tui/OllamaMonitorTest.java | 1 +
.../dsl/jbang/core/commands/tui/OllamaTabRenderTest.java | 15 ++++++++++++++-
4 files changed, 35 insertions(+), 5 deletions(-)
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
index 00cf91d53c40..553e11d8ce71 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
@@ -161,6 +161,14 @@ final class OllamaMonitor {
return Math.max(0, (long) inputTokens - cachedTokens);
}
+ /** Share of the prompt Ollama served from its cache. */
+ int cacheHitPercent() {
+ if (inputTokens <= 0) {
+ return 0;
+ }
+ return (int) Math.min(100, (long) cachedTokens * 100 / inputTokens);
+ }
+
/** Share of the context window the prompt filled, or -1 when the window is unknown. */
int contextPercent() {
if (contextSize <= 0) {
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
index ec4b7521eacd..7769b54d46f9 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
@@ -317,9 +317,16 @@ class OllamaTab extends AbstractTab {
Span.styled(" " + gaugeBar(pct, gaugeWidth) + " ", pct >= 90 ? Theme.error() : Theme.info()),
Span.styled(formatTokens(used) + " / " + formatTokens(size), Theme.label()),
Span.styled(" (" + pct + "%)", Theme.muted())));
- lines.add(Line.from(Span.styled(" prompt " + formatTokens(slot.promptTokens()) + " · cache hit "
- + slot.cacheHitPercent() + "%"
- + (slot.slots() > 1 ? " · " + slot.slots() + " slots" : ""),
+ // the runner clears its cache count once a request is done, so when idle the last request tells
+ RequestEntry last = s.lastRequest();
+ String promptLine;
+ if (slot.processing() || last == null) {
+ promptLine = " prompt " + formatTokens(slot.promptTokens()) + " · cache hit " + slot.cacheHitPercent() + "%";
+ } else {
+ promptLine = " prompt " + formatTokens(last.promptTokens()) + " · cache hit " + last.cacheHitPercent()
+ + "% (last request)";
+ }
+ lines.add(Line.from(Span.styled(promptLine + (slot.slots() > 1 ? " · " + slot.slots() + " slots" : ""),
Theme.muted())));
List<Span> state = new ArrayList<>();
if (slot.processing()) {
@@ -396,9 +403,10 @@ class OllamaTab extends AbstractTab {
data[n - 1 - i] = turns.get(i).contextPercent();
}
Style level = last >= 80 ? Theme.error() : last >= 50 ? Theme.warning() : Theme.info();
+ // bars start right after the label and grow to the right as turns are added
return Line.from(
Span.styled(" turns ", Theme.muted()),
- Span.styled(sparkline(data, barWidth, 100), level),
+ Span.styled(sparkline(data, n, 100), level),
Span.styled(suffix, level));
}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
index a4783ed5fb64..60589f1933e6 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
@@ -77,6 +77,7 @@ class OllamaMonitorTest {
"stop");
RequestEntry e = monitor.snapshot().lastRequest();
assertEquals(150, e.evaluatedTokens());
+ assertEquals(98, e.cacheHitPercent());
assertEquals(150 * 1000.0 / 532, e.prefillTokensPerSecond(), 0.1);
assertEquals(12_774, e.promptTokens());
assertEquals(150 * 1000.0 / 532, monitor.snapshot().totals().avgPrefillTokensPerSecond(), 0.1);
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
index 668c73bae5ed..d90b43d5dabf 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
@@ -100,7 +100,8 @@ class OllamaTabRenderTest {
assertTrue(rendered.contains("prefill"), rendered);
assertTrue(rendered.contains("TTFT 0.20s"), rendered);
assertTrue(rendered.contains("Context"), rendered);
- assertTrue(rendered.contains("cache hit 36%"), rendered);
+ // idle runner: the cache figure comes from the last request (none cached there), not the cleared slot
+ assertTrue(rendered.contains("cache hit 0% (last request)"), rendered);
assertTrue(rendered.contains("idle"), rendered);
assertTrue(rendered.contains("speculative draft-mtp"), rendered);
assertTrue(rendered.contains("Host"), rendered);
@@ -117,6 +118,18 @@ class OllamaTabRenderTest {
assertTrue(rendered.contains("stop"), rendered);
}
+ @Test
+ void workingRunnerShowsTheSlotCacheFigure() {
+ localServerWithModel();
+ monitor.updateRunner(new RunnerInfo(23629, 58237, 262144, 1, "/blob", "llama-server"));
+ monitor.updateSlot(new SlotState(true, 25, 25, 9, 6, 262144, "draft-mtp", 1, Instant.now()));
+ monitor.recordRequest("qwen3.6:35b-a3b", new LlmClient.TokenUsage(16, 6, 22, 0, 188, 81, 12, 300), 0, "stop");
+
+ String rendered = TuiTestHelper.renderToString(new OllamaTab(ctx, monitor), 180, 40);
+ assertTrue(rendered.contains("cache hit 36%"), rendered);
+ assertTrue(rendered.contains("working"), rendered);
+ }
+
@Test
void remoteServerHidesHostPanelAndExplainsMissingLiveState() {
monitor.updateServer(new ServerInfo("http://gpu-box.lan:11434", "0.33.3", false));
(camel) 06/09: CAMEL-24794: camel-jbang - TUI lists the Ollama tab only while an Ollama server answers
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch tui-ollama-tab
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 98b4b1adff645595239ca0b19af49f77d1543a65
Author: Claus Ibsen <cl...@gmail.com>
AuthorDate: Thu Sep 17 12:45:14 2026 +0200
CAMEL-24794: camel-jbang - TUI lists the Ollama tab only while an Ollama server answers
The More menu entry now depends on a background probe: endpoint detection
and one version request every ten seconds, run from the regular refresh
cycle whether or not the tab is showing. MoreTab gains an availableWhen
supplier that, unlike activeWhen, also applies when no integration is
selected, so the tab appears shortly after ollama serve starts and
disappears when it stops. Full polling still happens only while the tab is
active.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Signed-off-by: Claus Ibsen <cl...@gmail.com>
---
.../modules/ROOT/pages/camel-jbang-tui.adoc | 5 +-
.../dsl/jbang/core/commands/tui/CamelMonitor.java | 4 +
.../dsl/jbang/core/commands/tui/OllamaMonitor.java | 89 ++++++++++++++--------
.../dsl/jbang/core/commands/tui/TabRegistry.java | 27 +++++--
.../src/main/resources/tui/help/ollama.md | 5 +-
.../jbang/core/commands/tui/OllamaMonitorTest.java | 10 +++
.../jbang/core/commands/tui/TabRegistryTest.java | 41 ++++++++--
7 files changed, 136 insertions(+), 45 deletions(-)
diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
index 5851b3104896..369e9e2de5ad 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
@@ -150,7 +150,7 @@ The *More* menu (key *0*) opens a popup with tabs organized into groups:
* *Routing* -- Browse Endpoints, Consumers, HTTP, Inflight, Producers, Route Controller
* *Observability* -- Circuit Breaker, Health, JFR, Metrics, Network Services, Exchange Events, Recovery Tasks, OpenTelemetry Spans
-* *AI* -- Ollama
+* *AI* -- Ollama (listed when an Ollama server is detected)
* *Data* -- JDBC DataSource, Kafka, SQL Query, SQL Trace
* *JVM* -- Classpath, Heap Memory Histogram, Memory Usage, Memory Leak, Process, Startup, Threads
* *Project* -- Beans, Catalog, Configuration, CVE Audit, Maven Dependencies, Type Converters, Data Type Transformers
@@ -417,7 +417,8 @@ correlating Camel processing with external service calls (HTTP, database, messag
The Ollama tab (under More, in the *AI* group) shows how the model served by a local
https://ollama.com[Ollama] is performing, in the spirit of an LLM dashboard. It works with or without a
running integration: it finds Ollama at `localhost:11434`, at the address of `camel infra run ollama`,
-or at the endpoint the AI panel (*F8*) is using.
+or at the endpoint the AI panel (*F8*) is using. The tab is listed only while an Ollama server answers;
+the TUI checks every ten seconds, so it appears shortly after `ollama serve` starts.
* *Model* -- the loaded model with its family, parameters, quantization, layers, experts (and how many
are active per token for a mixture-of-experts model), how much of it sits in GPU memory, the
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java
index 55f3feef7a82..45ab04da7d0b 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java
@@ -2941,6 +2941,10 @@ public class CamelMonitor extends CamelCommand {
aiPanel.spanRefreshRequested = false;
dataService.refreshSpanData();
}
+ if (ollamaMonitor != null) {
+ // cheap and throttled: keeps the More menu's Ollama entry in step with whether a server answers
+ ollamaMonitor.probe();
+ }
if (tabRegistry.selectedTabIndex() == TAB_MORE
&& tabRegistry.getActiveMoreTab() == tabRegistry.ollamaTab() && ollamaMonitor != null) {
ollamaMonitor.poll();
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
index b7d0f485d183..00cf91d53c40 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
@@ -261,7 +261,7 @@ final class OllamaMonitor {
private String lastError;
private Instant lastPoll;
- private long lastDetect;
+ private long lastProbe;
private long lastVersion;
private long lastPs;
private long lastTags;
@@ -274,6 +274,36 @@ final class OllamaMonitor {
// ---- input from the rest of the TUI ----
+ /** Whether an Ollama server currently answers; the More menu lists the tab only then. */
+ boolean isAvailable() {
+ synchronized (lock) {
+ return server != null;
+ }
+ }
+
+ /**
+ * Cheap background check while the tab is not showing, so the More menu can list it as soon as Ollama comes up and
+ * drop it when Ollama goes away: endpoint detection and one version request every ten seconds. Full polling happens
+ * in {@link #poll()} while the tab is active.
+ */
+ void probe() {
+ if (!polling.compareAndSet(false, true)) {
+ return;
+ }
+ try {
+ long now = System.currentTimeMillis();
+ if (now - lastProbe >= DETECT_INTERVAL_MS) {
+ lastProbe = now;
+ lastVersion = now;
+ checkServer();
+ }
+ } catch (Exception e) {
+ LOG.debug("Ollama probe failed", e);
+ } finally {
+ polling.set(false);
+ }
+ }
+
/** Uses the Ollama endpoint another part of the TUI already resolved (the AI panel's client, an explicit URL). */
void adoptEndpoint(String url) {
if (url == null || url.isBlank()) {
@@ -530,22 +560,6 @@ final class OllamaMonitor {
}
private void doPoll(long now) {
- String base = baseUrl;
- if (base == null) {
- if (now - lastDetect >= DETECT_INTERVAL_MS) {
- lastDetect = now;
- base = detectEndpoint();
- if (base != null) {
- adoptEndpoint(base);
- base = baseUrl;
- }
- }
- if (base == null) {
- tick(now);
- return;
- }
- }
-
boolean connected;
synchronized (lock) {
connected = server != null;
@@ -553,24 +567,14 @@ final class OllamaMonitor {
long versionInterval = connected ? VERSION_INTERVAL_MS : RECONNECT_INTERVAL_MS;
if (now - lastVersion >= versionInterval) {
lastVersion = now;
- JsonObject version = getJsonObject(base + "/api/version");
- if (version == null) {
- synchronized (lock) {
- server = null;
- models = List.of();
- slot = null;
- lastError = "Ollama not reachable at " + OllamaParsers.displayHost(base);
- }
- tick(now);
- return;
- }
- updateServer(new ServerInfo(base, OllamaParsers.str(version, "version"), OllamaParsers.isLoopbackUrl(base)));
- connected = true;
+ lastProbe = now;
+ connected = checkServer();
}
if (!connected) {
tick(now);
return;
}
+ String base = baseUrl;
if (now - lastPs >= PS_INTERVAL_MS) {
lastPs = now;
@@ -671,6 +675,31 @@ final class OllamaMonitor {
return shape;
}
+ /** Detects the endpoint when none is known yet and confirms the server answers; true when connected. */
+ private boolean checkServer() {
+ String base = baseUrl;
+ if (base == null) {
+ base = detectEndpoint();
+ if (base == null) {
+ return false;
+ }
+ adoptEndpoint(base);
+ base = baseUrl;
+ }
+ JsonObject version = getJsonObject(base + "/api/version");
+ if (version == null) {
+ synchronized (lock) {
+ server = null;
+ models = List.of();
+ slot = null;
+ lastError = "Ollama not reachable at " + OllamaParsers.displayHost(base);
+ }
+ return false;
+ }
+ updateServer(new ServerInfo(base, OllamaParsers.str(version, "version"), OllamaParsers.isLoopbackUrl(base)));
+ return true;
+ }
+
private String detectEndpoint() {
try {
LlmClient client = LlmClient.create().withApiType(LlmClient.ApiType.ollama);
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistry.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistry.java
index b0f6f9555c97..3cfc2b5221f5 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistry.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistry.java
@@ -18,6 +18,7 @@ package org.apache.camel.dsl.jbang.core.commands.tui;
import java.util.ArrayList;
import java.util.List;
+import java.util.function.BooleanSupplier;
import java.util.function.Predicate;
import dev.tamboui.widgets.tabs.TabsState;
@@ -164,7 +165,8 @@ class TabRegistry {
threadsTab = new ThreadsTab(ctx);
spansTab = new SpansTab(ctx, dataService.otelSpans());
processTab = new ProcessTab(ctx);
- ollamaTab = new OllamaTab(ctx, ctx.ollamaMonitor);
+ OllamaMonitor ollamaMonitor = ctx.ollamaMonitor;
+ ollamaTab = new OllamaTab(ctx, ollamaMonitor);
overviewTab = new OverviewTab(
ctx, dataService.metrics(), dataService.stoppingPids(),
resetIntegrationTabState);
@@ -218,7 +220,10 @@ class TabRegistry {
TuiIcons.TAB_JFR, "JFR", "J&FR", jfrTab, "Observability",
List.of("jfr")),
// Data
- new MoreTab(TuiIcons.TAB_OLLAMA, "Ollama", "&Ollama", ollamaTab, "AI"),
+ new MoreTab(
+ TuiIcons.TAB_OLLAMA, "Ollama", "&Ollama", ollamaTab, "AI", List.of(), null,
+ // listed only while an Ollama server answers; the monitor probes in the background
+ () -> ollamaMonitor != null && ollamaMonitor.isAvailable()),
new MoreTab(
TuiIcons.TAB_DATASOURCE, "JDBC DataSource", "&JDBC DataSource", dataSourceTab, "Data",
List.of(), info -> !info.dataSources.isEmpty()),
@@ -481,21 +486,28 @@ class TabRegistry {
* @param requiredConsoles dev console IDs that must be present (any-of) for this tab to be active; empty = always
* active
* @param activeWhen runtime predicate on IntegrationInfo; null = always active (after console check)
+ * @param availableWhen runtime check independent of any integration (e.g. a local service answers); null =
+ * always available. Unlike {@code activeWhen} it also applies when nothing is selected.
*/
record MoreTab(String icon, String name, String label, MonitorTab tab, String group,
- List<String> requiredConsoles, Predicate<IntegrationInfo> activeWhen) {
+ List<String> requiredConsoles, Predicate<IntegrationInfo> activeWhen, BooleanSupplier availableWhen) {
+
+ MoreTab(String icon, String name, String label, MonitorTab tab, String group,
+ List<String> requiredConsoles, Predicate<IntegrationInfo> activeWhen) {
+ this(icon, name, label, tab, group, requiredConsoles, activeWhen, null);
+ }
MoreTab(String icon, String name, String label, MonitorTab tab, String group,
List<String> requiredConsoles) {
- this(icon, name, label, tab, group, requiredConsoles, null);
+ this(icon, name, label, tab, group, requiredConsoles, null, null);
}
MoreTab(String icon, String name, String label, MonitorTab tab, String group) {
- this(icon, name, label, tab, group, List.of(), null);
+ this(icon, name, label, tab, group, List.of(), null, null);
}
MoreTab(String icon, String name, String label, MonitorTab tab) {
- this(icon, name, label, tab, null, List.of(), null);
+ this(icon, name, label, tab, null, List.of(), null, null);
}
MoreTab {
@@ -524,6 +536,9 @@ class TabRegistry {
}
static boolean isMoreTabActive(MoreTab mt, IntegrationInfo info) {
+ if (mt.availableWhen() != null && !mt.availableWhen().getAsBoolean()) {
+ return false;
+ }
if (mt.requiredConsoles().isEmpty() && mt.activeWhen() == null) {
return true;
}
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
index da15e63a38dd..e63b78c069fb 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
@@ -3,7 +3,10 @@
The Ollama tab shows how the model served by Ollama is performing, in
the spirit of an LLM dashboard. It finds Ollama at `localhost:11434`,
at the address of `camel infra run ollama`, or at the endpoint the AI
-panel (F8) uses, and works with or without a running integration.
+panel (F8) uses, and works with or without a running integration. The
+tab is listed in the More menu only while an Ollama server answers; the
+TUI checks every ten seconds, so it appears shortly after `ollama serve`
+starts and disappears when it stops.
Two kinds of requests feed it. Questions asked in the AI panel with
Ollama as the provider arrive with the timings Ollama reports for every
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
index 467422474974..a4783ed5fb64 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
@@ -203,6 +203,16 @@ class OllamaMonitorTest {
assertEquals(0, history[0]);
}
+ @Test
+ void availabilityFollowsTheServer() {
+ OllamaMonitor monitor = new OllamaMonitor();
+ assertFalse(monitor.isAvailable());
+ monitor.updateServer(new ServerInfo("http://localhost:11434", "0.33.3", true));
+ assertTrue(monitor.isAvailable());
+ monitor.updateServer(null);
+ assertFalse(monitor.isAvailable());
+ }
+
@Test
void resetClearsRequestsTotalsAndHistoryButKeepsServer() {
OllamaMonitor monitor = new OllamaMonitor();
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistryTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistryTest.java
index 4594d1495664..081443e3dcc5 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistryTest.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/TabRegistryTest.java
@@ -80,6 +80,30 @@ class TabRegistryTest {
});
}
+ @Test
+ void availableWhenHidesATabEvenWithoutASelectedIntegration() {
+ MonitorTab tab = registry.moreTabs().get(0).tab();
+ TabRegistry.MoreTab hidden = new TabRegistry.MoreTab(
+ "🦙", "Probe", "&Probe", tab, "AI", List.of(), null, () -> false);
+ TabRegistry.MoreTab shown = new TabRegistry.MoreTab(
+ "🦙", "Probe", "&Probe", tab, "AI", List.of(), null, () -> true);
+ IntegrationInfo info = new IntegrationInfo();
+
+ assertFalse(TabRegistry.isMoreTabActive(hidden, null));
+ assertFalse(TabRegistry.isMoreTabActive(hidden, info));
+ assertTrue(TabRegistry.isMoreTabActive(shown, null));
+ assertTrue(TabRegistry.isMoreTabActive(shown, info));
+ }
+
+ @Test
+ void ollamaTabIsListedOnlyWhileTheMonitorSeesAServer() {
+ TabRegistry.MoreTab ollama = registry.moreTabs().stream()
+ .filter(mt -> "Ollama".equals(mt.name())).findFirst().orElseThrow();
+ // the registry in this test has no monitor at all: never listed
+ assertFalse(TabRegistry.isMoreTabActive(ollama, null));
+ assertEquals("AI", ollama.group());
+ }
+
@Test
void everyMoreTabLabelHasMnemonicMarker() {
for (TabRegistry.MoreTab mt : registry.moreTabs()) {
@@ -118,7 +142,9 @@ class TabRegistryTest {
@Test
void allTabEntriesExposeDigitsIconsAndMoreShortcuts() {
List<TabRegistry.TabEntry> entries = registry.allTabEntries();
- assertEquals(9 + registry.moreTabs().size(), entries.size());
+ // the Ollama tab is listed only while a server answers, and this registry has no monitor
+ assertEquals(registry.moreTabs().size() - 1, registry.activeMoreTabs(null).size());
+ assertEquals(9 + registry.activeMoreTabs(null).size(), entries.size());
// Primary tabs: digit shortcuts 1-9, moreIndex -1, icon indexed by tabIndex.
for (int i = 0; i < 9; i++) {
@@ -128,12 +154,15 @@ class TabRegistryTest {
assertEquals(TuiIcons.PRIMARY_TAB_ICONS.get(e.tabIndex()), e.icon());
}
- // More tabs: tabIndex TAB_MORE, ascending moreIndex, shortcut/name/icon carried from the owning MoreTab.
- for (int i = 0; i < registry.moreTabs().size(); i++) {
- TabRegistry.TabEntry e = entries.get(9 + i);
- TabRegistry.MoreTab mt = registry.moreTabs().get(i);
+ // More tabs: tabIndex TAB_MORE, ascending moreIndex into the full list, shortcut/name/icon carried from
+ // the owning MoreTab; a tab that is not listed leaves a gap in the indexes.
+ int previousIndex = -1;
+ for (int i = 9; i < entries.size(); i++) {
+ TabRegistry.TabEntry e = entries.get(i);
+ TabRegistry.MoreTab mt = registry.moreTabs().get(e.moreIndex());
assertEquals(TabRegistry.TAB_MORE, e.tabIndex());
- assertEquals(i, e.moreIndex());
+ assertTrue(e.moreIndex() > previousIndex, "More indexes ascend");
+ previousIndex = e.moreIndex();
assertEquals(String.valueOf(mt.shortcut()), e.shortcut());
assertEquals(mt.name(), e.name());
assertEquals(mt.icon(), e.icon());
(camel) 02/09: CAMEL-24794: camel-jbang - TUI Ollama tab F1 help explains the phases, panels and columns
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch tui-ollama-tab
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 8dbf082cfb19fdd9314da5d76a5ee652639442ff
Author: Claus Ibsen <cl...@gmail.com>
AuthorDate: Thu Sep 17 12:15:30 2026 +0200
CAMEL-24794: camel-jbang - TUI Ollama tab F1 help explains the phases, panels and columns
Moves the help to tui/help/ollama.md like the other tabs and explains how a
local request is spent (load, prefill, decode), what TTFT, cold start, cache
hit and speculative decoding mean, each panel and every column of the
request log, and what differs against a remote or containerised Ollama.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Signed-off-by: Claus Ibsen <cl...@gmail.com>
---
.../dsl/jbang/core/commands/tui/OllamaTab.java | 20 +--
.../src/main/resources/tui/help/ollama.md | 134 +++++++++++++++++++++
.../core/commands/tui/OllamaTabRenderTest.java | 9 ++
3 files changed, 144 insertions(+), 19 deletions(-)
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
index 967233487f11..df6e7d259e61 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
@@ -503,25 +503,7 @@ class OllamaTab extends AbstractTab {
@Override
public String getHelpText() {
- return """
- # Ollama
-
- Performance of the model Ollama is serving, in the spirit of an LLM dashboard.
-
- - **Header**: the loaded model with its family, parameters, quantization, layers, experts, \
- how much of it sits in GPU memory, its context length and when it unloads.
- - **Throughput**: decode and prefill tokens per second. *live* while the runner on this machine \
- is generating, otherwise the figures of the last request; time to first token and load time \
- (a load of a second or more means a cold start); session averages and a decode-rate history.
- - **Context**: context window fill, cache hit ratio and whether the runner is working, from the \
- runner's live state on this machine; the last request's tokens otherwise.
- - **Host** (same machine only): GPU utilization and memory, CPU and memory of the runner and \
- the Ollama server.
- - **Requests**: every request the AI panel (F8) made through Ollama with Ollama's own timings, \
- plus calls made by Camel routes when GenAI observability is on (tokens and duration only).
-
- Keys: **↑/↓** select a request, **r** reset the log and session totals, **F5** refresh now.
- """;
+ return DocHelper.loadHelpText("ollama");
}
// ---- formatting helpers (package-private for tests) ----
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
new file mode 100644
index 000000000000..8e75cc50ca53
--- /dev/null
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
@@ -0,0 +1,134 @@
+# Ollama
+
+The Ollama tab shows how the model served by Ollama is performing, in
+the spirit of an LLM dashboard. It finds Ollama at `localhost:11434`,
+at the address of `camel infra run ollama`, or at the endpoint the AI
+panel (F8) uses, and works with or without a running integration.
+
+Two kinds of requests feed it. Questions asked in the AI panel with
+Ollama as the provider arrive with the timings Ollama reports for every
+request. Calls made by Camel routes (`camel-langchain4j-chat`,
+`camel-openai`, `camel-spring-ai-chat`) arrive through the GenAI
+observability spans when the integration runs with `--observe`; Camel
+records tokens and duration for those, not the phase split.
+
+## How a Request Is Spent
+
+A local model answers in phases, and every column on this tab maps to
+one of them:
+
+1. **Load** — if the model is not in memory yet (first request, or the
+ keep-alive expired, or a request asked for a different context
+ size), Ollama starts a runner and loads the weights. Seconds to tens
+ of seconds. A load of a second or more is shown as a **cold** start.
+2. **Prefill** — the prompt (system prompt, tool definitions, history,
+ your question) is processed in one batch. Reported as *prefill
+ tokens per second*. Prompt tokens already in Ollama's cache from the
+ previous turn are skipped; they show as **cached**.
+3. **Decode** — the answer is generated one token at a time. Reported
+ as *decode tokens per second*; this is the "typing speed" you see.
+
+**TTFT**, time to first token, is load plus prefill: how long you wait
+before anything appears. **Total** is the whole request as Ollama
+measured it, including its own queueing.
+
+## Header
+
+The loaded model and its shape:
+
+- Family, parameter count and quantization (`Q4_K_M` is 4-bit)
+- Layers, and for a mixture-of-experts model the expert count with the
+ number active per token (`256 experts (8 active)`)
+- Capabilities beyond plain completion: vision, tools, thinking
+- Size in memory and the share held by the GPU (`100% GPU` means fully
+ offloaded; a lower figure means part of the model runs on the CPU
+ and decode will be slow)
+- `ctx 32,768 of 262,144` — the context window Ollama allocated for
+ this load, and the maximum the model supports. The AI panel asks for
+ `32768` unless `OLLAMA_CONTEXT_LENGTH` is set. A request with a
+ different size makes Ollama reload the model.
+- When the model unloads (its keep-alive)
+
+With no model loaded the installed models are listed instead; a model
+loads on the first request.
+
+## Throughput
+
+- **decode** — tokens per second. Marked *live* while the runner on this
+ machine is generating (sampled from the runner several times a
+ second), otherwise the figure of the *last request*.
+- **prefill** — prompt tokens per second, same rule.
+- **TTFT** and **load** of the last request, with *cold* or *warm*.
+- **session** — average decode rate, total input and output tokens and
+ request count since the TUI started or since the last reset.
+- The sparkline is the live decode rate over the last 60 seconds when
+ the runner is on this machine, otherwise one bar per request.
+
+## Context
+
+- The bar is how full the context window is: prompt plus generated
+ tokens against the allocated size. Near 100% the model starts
+ forgetting the beginning of the conversation.
+- **prompt** and **cache hit** — the prompt size of the current or last
+ request and the share of it served from Ollama's cache. A high cache
+ hit on the second and later turns of a conversation means the system
+ prompt and tool block were reused; zero means every turn pays the
+ full prefill again.
+- **working** / **idle** — whether the runner is generating right now.
+ During a long prompt the prefill progress is shown.
+- **speculative** — the speculative decoding method the runner uses
+ (`draft-mtp` is multi-token prediction), which is why decode can
+ exceed one token per step.
+
+Without the local runner the panel falls back to the last request's
+tokens against the model's context length.
+
+## Host
+
+Shown when Ollama runs on this machine:
+
+- **GPU** — utilization and memory. Apple silicon through `ioreg`
+ (unified memory wired by the GPU), NVIDIA through `nvidia-smi`.
+- **llama-server** — CPU and resident memory of the model runner
+ Ollama started for the loaded model. Resident memory is roughly the
+ model size plus the context.
+- **ollama serve** — the Ollama server process itself.
+
+## Requests
+
+One line per request, newest first:
+
+| Column | Meaning |
+|--------|---------|
+| TIME | When the request started |
+| SOURCE | `tui` for the AI panel, `route:<id>` for a Camel route |
+| MODEL | The model that answered |
+| IN | Prompt tokens evaluated |
+| OUT | Tokens generated |
+| CACHED | Prompt tokens served from Ollama's cache (`-` when none) |
+| PREFILL | Prompt tokens per second |
+| DECODE | Generated tokens per second |
+| TTFT | Time to first token (load plus prefill), shown in yellow after a cold start |
+| TOTAL | Whole request as Ollama measured it |
+| REASON | Why generation stopped: `stop`, `length` (hit the token limit), `tool_calls` |
+
+Route requests show `-` for prefill, decode and TTFT because the GenAI
+span carries tokens and duration only.
+
+## Remote and Containerised Ollama
+
+The Ollama API and the per-request data work against any host. The
+live figures, the context panel's live state and the host panel need
+the runner on this machine; against a remote host or a container the
+tab says so and keeps the rest.
+
+## Keyboard Shortcuts
+
+| Key | Action |
+|-----|--------|
+| ↑ / ↓ | Select a request |
+| r | Reset the request log and session totals |
+| F5 | Refresh now |
+
+The same data is available to AI agents as the `tui_get_ollama` MCP
+tool and through `tui_get_table` on this tab.
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
index 9035849aa042..46d7d515b5d0 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTabRenderTest.java
@@ -169,6 +169,15 @@ class OllamaTabRenderTest {
assertEquals(1, ((JsonArray) summary.get("loadedModels")).size());
}
+ @Test
+ void helpTextExplainsThePhasesAndColumns() {
+ String help = new OllamaTab(ctx, monitor).getHelpText();
+ assertTrue(help != null && help.contains("TTFT"), help);
+ for (String term : List.of("Prefill", "Decode", "cold", "cache hit", "speculative", "CACHED", "REASON")) {
+ assertTrue(help.contains(term), "help should explain " + term);
+ }
+ }
+
@Test
void formattingHelpers() {
assertEquals("54", OllamaTab.formatRate(54.04));
(camel) 04/09: CAMEL-24794: camel-jbang - TUI Ollama tab counts cached prompt tokens once
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch tui-ollama-tab
in repository https://gitbox.apache.org/repos/asf/camel.git
commit 3f195298bffc23bf10fcd39c289c0b4ad5c0b290
Author: Claus Ibsen <cl...@gmail.com>
AuthorDate: Thu Sep 17 12:24:12 2026 +0200
CAMEL-24794: camel-jbang - TUI Ollama tab counts cached prompt tokens once
Ollama's prompt_eval_count is the whole prompt and prompt_eval_cached_count
the part of it served from the KV cache, not an addition. The context fill
added the two, reporting 77% for a prompt the runner's slot showed at 39%.
The prefill rate now covers only the evaluated tokens (prompt minus cached),
so a long prompt with a high cache hit no longer looks like a 24k tok/s
prefill. Help text corrected accordingly; tui_get_ollama gains
evaluatedTokens per request.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Signed-off-by: Claus Ibsen <cl...@gmail.com>
---
.../dsl/jbang/core/commands/tui/OllamaMonitor.java | 19 +++++++++++++++----
.../dsl/jbang/core/commands/tui/OllamaTab.java | 2 +-
.../src/main/resources/tui/help/ollama.md | 14 ++++++++------
.../jbang/core/commands/tui/OllamaMonitorTest.java | 21 ++++++++++++++++++---
4 files changed, 42 insertions(+), 14 deletions(-)
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
index ceaf4c63dd42..b7d0f485d183 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitor.java
@@ -148,9 +148,17 @@ final class OllamaMonitor {
int outputTokens, int cachedTokens, long prefillMs, long decodeMs, long loadMs, long totalMs,
String doneReason, long contextSize) {
- /** Everything the model had in front of it: prompt tokens evaluated plus those served from cache. */
+ /**
+ * Everything the model had in front of it. Ollama's {@code prompt_eval_count} is the whole prompt; the cached
+ * count is the part of it served from the KV cache, not an addition.
+ */
long promptTokens() {
- return (long) inputTokens + cachedTokens;
+ return inputTokens;
+ }
+
+ /** Prompt tokens the runner actually had to evaluate this time. */
+ long evaluatedTokens() {
+ return Math.max(0, (long) inputTokens - cachedTokens);
}
/** Share of the context window the prompt filled, or -1 when the window is unknown. */
@@ -161,8 +169,9 @@ final class OllamaMonitor {
return (int) Math.min(100, promptTokens() * 100 / contextSize);
}
+ /** Prefill speed over the tokens that were not served from cache. */
double prefillTokensPerSecond() {
- return prefillMs > 0 ? inputTokens * 1000.0 / prefillMs : 0;
+ return prefillMs > 0 && evaluatedTokens() > 0 ? evaluatedTokens() * 1000.0 / prefillMs : 0;
}
double decodeTokensPerSecond() {
@@ -202,7 +211,8 @@ final class OllamaMonitor {
}
double avgPrefillTokensPerSecond() {
- return prefillMs > 0 ? inputTokens * 1000.0 / prefillMs : 0;
+ long evaluated = Math.max(0, inputTokens - cachedTokens);
+ return prefillMs > 0 && evaluated > 0 ? evaluated * 1000.0 / prefillMs : 0;
}
}
@@ -986,6 +996,7 @@ final class OllamaMonitor {
r.put("ttftMs", e.ttftMs());
r.put("coldStart", e.coldStart());
r.put("promptTokens", e.promptTokens());
+ r.put("evaluatedTokens", e.evaluatedTokens());
r.put("contextSize", e.contextSize());
r.put("contextPercent", e.contextPercent());
if (e.doneReason() != null) {
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
index e532e60b0e79..ec4b7521eacd 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaTab.java
@@ -345,7 +345,7 @@ class OllamaTab extends AbstractTab {
RequestEntry last = s.lastRequest();
long ctx = model != null ? model.contextLength() : 0;
if (last != null && ctx > 0) {
- long used = (long) last.inputTokens() + last.outputTokens() + last.cachedTokens();
+ long used = (long) last.inputTokens() + last.outputTokens();
int pct = (int) Math.min(100, used * 100 / ctx);
lines.add(Line.from(
Span.styled(" " + gaugeBar(pct, gaugeWidth) + " ", pct >= 90 ? Theme.error() : Theme.info()),
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
index 63af33fc2a0f..da15e63a38dd 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/resources/tui/help/ollama.md
@@ -22,9 +22,11 @@ one of them:
size), Ollama starts a runner and loads the weights. Seconds to tens
of seconds. A load of a second or more is shown as a **cold** start.
2. **Prefill** — the prompt (system prompt, tool definitions, history,
- your question) is processed in one batch. Reported as *prefill
- tokens per second*. Prompt tokens already in Ollama's cache from the
- previous turn are skipped; they show as **cached**.
+ your question) is processed in one batch. Prompt tokens already in
+ Ollama's cache from the previous turn or tool step are skipped; they
+ show as **cached**. *Prefill tokens per second* counts only the
+ tokens that had to be evaluated, so a long prompt with a high cache
+ hit still has a short prefill.
3. **Decode** — the answer is generated one token at a time. Reported
as *decode tokens per second*; this is the "typing speed" you see.
@@ -110,11 +112,11 @@ One line per request, newest first:
| TIME | When the request started |
| SOURCE | `tui` for the AI panel, `route:<id>` for a Camel route |
| MODEL | The model that answered |
-| IN | Prompt tokens evaluated |
+| IN | Prompt tokens, the whole prompt the model saw (CACHED is the part of it served from cache) |
| OUT | Tokens generated |
| CACHED | Prompt tokens served from Ollama's cache (`-` when none) |
-| CTX | Share of the context window the prompt filled (IN plus CACHED against the window that served it) |
-| PREFILL | Prompt tokens per second |
+| CTX | Share of the context window the prompt filled (IN against the window that served it) |
+| PREFILL | Evaluated prompt tokens per second (IN minus CACHED over the prefill time) |
| DECODE | Generated tokens per second |
| TTFT | Time to first token (load plus prefill), shown in yellow after a cold start |
| TOTAL | Whole request as Ollama measured it |
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
index bd8d7c1ecf3e..467422474974 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/test/java/org/apache/camel/dsl/jbang/core/commands/tui/OllamaMonitorTest.java
@@ -69,6 +69,19 @@ class OllamaMonitorTest {
assertEquals("stop", e.doneReason());
}
+ @Test
+ void prefillRateCountsOnlyTokensNotServedFromCache() {
+ // a tool-loop step observed live: 12,774 prompt tokens of which 12,624 came from the cache, 532 ms prefill
+ OllamaMonitor monitor = new OllamaMonitor();
+ monitor.recordRequest("m", new LlmClient.TokenUsage(12_774, 161, 12_935, 12_624, 532, 2_990, 0, 3_588), 0,
+ "stop");
+ RequestEntry e = monitor.snapshot().lastRequest();
+ assertEquals(150, e.evaluatedTokens());
+ assertEquals(150 * 1000.0 / 532, e.prefillTokensPerSecond(), 0.1);
+ assertEquals(12_774, e.promptTokens());
+ assertEquals(150 * 1000.0 / 532, monitor.snapshot().totals().avgPrefillTokensPerSecond(), 0.1);
+ }
+
@Test
void coldStartIsDetectedFromLoadTime() {
OllamaMonitor monitor = new OllamaMonitor();
@@ -150,15 +163,17 @@ class OllamaMonitorTest {
monitor.recordRequest("m", new LlmClient.TokenUsage(4_000, 60, 4_060, 500, 900, 800, 0, 1800), 0, "stop");
RequestEntry first = monitor.snapshot().lastRequest();
assertEquals(32_768, first.contextSize());
- assertEquals(4_500, first.promptTokens());
- assertEquals(13, first.contextPercent());
+ // prompt_eval_count is the whole prompt; the cached 500 are part of it, not on top
+ assertEquals(4_000, first.promptTokens());
+ assertEquals(3_500, first.evaluatedTokens());
+ assertEquals(12, first.contextPercent());
// the runner's slot wins over the model list when present
monitor.updateSlot(slot(false, 0, System.currentTimeMillis()));
monitor.recordRequest("m", new LlmClient.TokenUsage(20_000, 60, 20_060, 0, 900, 800, 0, 1800), 0, "stop");
assertEquals(262144, monitor.snapshot().lastRequest().contextSize());
assertEquals(7, monitor.snapshot().lastRequest().contextPercent());
- assertEquals(13, monitor.snapshot().totals().peakContextPercent());
+ assertEquals(12, monitor.snapshot().totals().peakContextPercent());
assertEquals(0, monitor.snapshot().totals().compactions());
// a prompt a fifth smaller than the previous turn counts as a compaction