You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by da...@apache.org on 2026/09/08 14:07:18 UTC
(camel) 02/02: Camel TUI: Recommend qwen3.6:35b-a3b as the local Ollama model for the AI panel
This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
commit d202eff67b1745427de78452f777e3323bf944a1
Author: Claus Ibsen <cl...@gmail.com>
AuthorDate: Tue Sep 8 16:05:40 2026 +0200
Camel TUI: Recommend qwen3.6:35b-a3b as the local Ollama model for the AI panel
The F8 panel sends a tool-heavy prompt of ~8k tokens on every question, so
prompt processing dominates latency with local models. A dense 27B/32B model
prefills at ~110 tok/s on an M4 Pro (about a minute per question), while the
mixture-of-experts qwen3.6:35b-a3b with 3B active parameters prefills at
~750 tok/s and decodes faster too, so answers start in seconds.
Update the F2 Setup AI page, the F8 getting-started guide, the doctor hint,
the model auto-selection preference, and the TUI and AI user-manual pages to
recommend qwen3.6:35b-a3b, and note that on Apple Silicon the default GGUF
tags should be used because the Ollama MLX engine cannot yet reuse the cached
prompt for Qwen 3.x models.
Co-Authored-By: Claude Fable 5.1 <no...@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DHRH7agh6Fh1ExD9e261H6
Signed-off-by: Claus Ibsen <cl...@gmail.com>
---
.../modules/ROOT/pages/camel-jbang-ai.adoc | 15 ++++++++++-----
.../modules/ROOT/pages/camel-jbang-tui.adoc | 17 ++++++++++++-----
.../camel/dsl/jbang/core/commands/LlmClient.java | 2 +-
.../camel/dsl/jbang/core/commands/tui/AiPanel.java | 14 ++++++++------
.../dsl/jbang/core/commands/tui/DocViewerPopup.java | 19 +++++++++++++------
.../dsl/jbang/core/commands/tui/DoctorPopup.java | 2 +-
6 files changed, 45 insertions(+), 24 deletions(-)
diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-ai.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-ai.adoc
index baa4f1e33852..74f11eecd891 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-jbang-ai.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-ai.adoc
@@ -148,7 +148,7 @@ brew install ollama
curl -fsSL https://ollama.com/install.sh | sh
# Pull a model and start asking
-ollama pull qwen2.5:32b
+ollama pull qwen3.6:35b-a3b
camel ask "what routes are running?"
----
@@ -159,19 +159,24 @@ The CLI checks what models are available and auto-selects a suitable one.
`camel ask` and the TUI F8 panel rely on tool calling to inspect your running Camel process.
Models smaller than ~14B do not reliably invoke tools and answer from training knowledge instead.
-Use at least a 14B model; 32B is recommended.
+Use at least a 14B model. Prefer a mixture-of-experts model such as `qwen3.6:35b-a3b`: with only
+3B parameters active per token it processes the tool-heavy prompt many times faster than a dense
+27B/32B model, so answers start in seconds instead of a minute.
[options="header"]
|===
| Model | RAM (Q4) | Notes
-| `qwen2.5:14b` | ~9 GB | Minimum recommended
-| `qwen2.5:32b` | ~20 GB | Best balance of speed and quality
-| `deepseek-r1:32b` | ~20 GB | Strong reasoning
+| `qwen3.6:35b-a3b` | ~23 GB | Recommended: fastest prompt processing, needs 32 GB+
+| `qwen2.5:14b` | ~9 GB | Minimum for 16 GB machines
+| `qwen3.6:27b` | ~18 GB | Strong dense model, several times slower prompt processing
+| `qwen2.5:32b` | ~20 GB | Good quality, slow prompt processing
| `hermes3:70b` | ~43 GB | Excellent tool calling, needs 64 GB+
| `llama3.3:70b` | ~43 GB | Best open model, needs 64 GB+
|===
On Apple Silicon, all RAM is unified — a 64 GB M-series Mac can run `llama3.3:70b` comfortably alongside the OS and other dev tools.
+Use the default (GGUF) tags rather than the `-mlx` tags: the Ollama MLX engine cannot yet reuse the
+cached prompt for Qwen 3.x models, so every question re-processes the whole prompt.
=== Using an OpenAI-compatible local server
diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
index 43b25261142f..90c4d0b0f1ed 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-tui.adoc
@@ -782,7 +782,7 @@ brew install ollama
curl -fsSL https://ollama.com/install.sh | sh
# Pull a model — then open the TUI and press F8
-ollama pull qwen2.5:32b
+ollama pull qwen3.6:35b-a3b
camel tui
----
@@ -790,16 +790,19 @@ Ollama at `localhost:11434` is auto-detected. No configuration needed.
IMPORTANT: The F8 AI panel works by invoking built-in tools to inspect your running Camel process.
Models smaller than ~14B do not reliably call tools and answer from training knowledge instead.
-Use at least a 14B model; 32B is recommended.
+Use at least a 14B model. Prefer a mixture-of-experts model such as `qwen3.6:35b-a3b`: with only
+3B parameters active per token it processes the tool-heavy prompt many times faster than a dense
+27B/32B model, so answers start in seconds instead of a minute.
*Models that work well* (tool-calling capable, ≥14B, default Q4_K_M quantization):
[options="header"]
|===
| Model | RAM | Notes
-| `qwen2.5:14b` | ~9 GB | Minimum recommended
-| `qwen2.5:32b` | ~20 GB | Best balance of speed and quality
-| `deepseek-r1:32b` | ~20 GB | Strong reasoning
+| `qwen3.6:35b-a3b` | ~23 GB | Recommended: fastest prompt processing, needs 32 GB+
+| `qwen2.5:14b` | ~9 GB | Minimum for 16 GB machines
+| `qwen3.6:27b` | ~18 GB | Strong dense model, several times slower prompt processing
+| `qwen2.5:32b` | ~20 GB | Good quality, slow prompt processing
| `hermes3:70b` | ~43 GB | Excellent tool calling, needs 64 GB+
| `llama3.3:70b` | ~43 GB | Best open model, needs 64 GB+
|===
@@ -807,6 +810,10 @@ Use at least a 14B model; 32B is recommended.
NOTE: `camel infra run ollama` runs Ollama in Docker and bypasses GPU acceleration,
making inference significantly slower. Native install is preferred for development.
+NOTE: On Apple Silicon, use the default (GGUF) tags rather than the `-mlx` tags. The Ollama MLX
+engine cannot yet reuse the cached prompt for Qwen 3.x models, so every question re-processes the
+whole prompt, while the default engine reuses it and only processes what is new.
+
==== Using an OpenAI-compatible local server
Set `LLM_API_KEY` and `LLM_BASE_URL` to connect to any OpenAI-compatible server
diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java
index 4ada1d9fb0c6..1d8470f287e7 100644
--- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java
+++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/LlmClient.java
@@ -1965,7 +1965,7 @@ public class LlmClient {
}
List<String> preferred
- = List.of("qwen3.5", "qwen3", "nemotron-3-nano", "mistral-nemo",
+ = List.of("qwen3.6", "qwen3.5", "qwen3", "nemotron-3-nano", "mistral-nemo",
"qwen2.5", "granite4.1", "llama3.1", "llama3.3", "mistral");
for (String pref : preferred) {
for (String avail : available) {
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java
index 0a0e15112ba6..19caf4aecfd1 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/AiPanel.java
@@ -1595,8 +1595,9 @@ class AiPanel {
> **Tool calling is required.** This panel inspects your Camel process by
> invoking built-in tools. Models smaller than ~14B do not reliably call
- > tools and will answer from training knowledge instead — use at least 14B,
- > 32B recommended.
+ > tools and will answer from training knowledge instead — use at least 14B.
+ > Prefer a mixture-of-experts model such as qwen3.6:35b-a3b: it processes
+ > the tool-heavy prompt many times faster than a dense 27B/32B model.
---
@@ -1613,7 +1614,7 @@ class AiPanel {
# then on both:
ollama serve # start the daemon (skip if auto-started)
- ollama pull qwen2.5:32b # recommended
+ ollama pull qwen3.6:35b-a3b # recommended
```
Ollama is auto-detected at `localhost:11434` — no configuration needed.
@@ -1622,9 +1623,10 @@ class AiPanel {
| Model | RAM | Notes |
|---|---|---|
- | qwen2.5:14b | ~9 GB | Minimum recommended |
- | qwen2.5:32b | ~20 GB | Best balance of speed and quality |
- | deepseek-r1:32b | ~20 GB | Strong reasoning |
+ | qwen3.6:35b-a3b | ~23 GB | Recommended: only 3B active per token, fastest prompt processing |
+ | qwen2.5:14b | ~9 GB | Minimum for 16 GB machines |
+ | qwen3.6:27b | ~18 GB | Strong dense model, several times slower prompt processing |
+ | qwen2.5:32b | ~20 GB | Good quality, slow prompt processing |
| hermes3:70b | ~43 GB | Excellent tool calling, needs 64 GB+ |
| llama3.3:70b | ~43 GB | Best open model, needs 64 GB+ |
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocViewerPopup.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocViewerPopup.java
index ce1f0c11de77..2a75a6523991 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocViewerPopup.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DocViewerPopup.java
@@ -390,7 +390,7 @@ class DocViewerPopup {
status = "**Detected:** Ollama at " + OllamaDoctorSupport.formatDisplayHost(ollama.baseUrl())
+ " with " + OllamaDoctorSupport.modelCountLabel(ollama.models());
} else if (ollama.running()) {
- status = "**Detected:** Ollama is running but has no models. Run `ollama pull qwen2.5:32b`.";
+ status = "**Detected:** Ollama is running but has no models. Run `ollama pull qwen3.6:35b-a3b`.";
} else {
status = "**Status:** No AI provider detected. Set an API key or start Ollama, then press F8.";
}
@@ -422,18 +422,25 @@ class DocViewerPopup {
+ "Install Ollama natively for GPU acceleration, pull a model, then press F8:\n\n"
+ " brew install ollama # macOS\n"
+ " curl -fsSL https://ollama.com/install.sh | sh # Linux\n"
- + " ollama pull qwen2.5:32b\n\n"
+ + " ollama pull qwen3.6:35b-a3b\n\n"
+ "The AI panel relies on tool calling. Models smaller than ~14B do not call tools\n"
- + "reliably and answer from training data instead. Use at least 14B; 32B is recommended.\n\n"
+ + "reliably and answer from training data instead. Use at least 14B. Prefer a\n"
+ + "mixture-of-experts model such as `qwen3.6:35b-a3b`: with only 3B parameters active\n"
+ + "per token it processes the tool-heavy prompt many times faster than a dense 27B/32B\n"
+ + "model, so answers start in seconds instead of a minute.\n\n"
+ "| Model | RAM | Notes |\n"
+ "|-------|-----|-------|\n"
- + "| `qwen2.5:14b` | ~9 GB | Minimum recommended |\n"
- + "| `qwen2.5:32b` | ~20 GB | Best balance of speed and quality |\n"
- + "| `deepseek-r1:32b` | ~20 GB | Strong reasoning |\n"
+ + "| `qwen3.6:35b-a3b` | ~23 GB | Recommended: fastest prompt processing, needs 32 GB+ |\n"
+ + "| `qwen2.5:14b` | ~9 GB | Minimum for 16 GB machines |\n"
+ + "| `qwen3.6:27b` | ~18 GB | Strong dense model, several times slower prompt processing |\n"
+ + "| `qwen2.5:32b` | ~20 GB | Good quality, slow prompt processing |\n"
+ "| `hermes3:70b` | ~43 GB | Excellent tool calling, needs 64 GB+ |\n"
+ "| `llama3.3:70b` | ~43 GB | Best open model, needs 64 GB+ |\n\n"
+ "`camel infra run ollama` runs Ollama in Docker without GPU acceleration, which is\n"
+ "much slower. Prefer the native install for development.\n\n"
+ + "On Apple Silicon, use the default (GGUF) tags rather than the `-mlx` tags: the Ollama\n"
+ + "MLX engine cannot yet reuse the cached prompt for Qwen 3.x models, so every question\n"
+ + "re-processes the whole prompt.\n\n"
+ "## 3. OpenAI-compatible local servers\n\n"
+ "LM Studio, vLLM, llama.cpp, GPT4All and similar servers work through `LLM_BASE_URL`:\n\n"
+ " export LLM_API_KEY=any-value\n"
diff --git a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DoctorPopup.java b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DoctorPopup.java
index 048a90839c1a..bb33273a5362 100644
--- a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DoctorPopup.java
+++ b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/DoctorPopup.java
@@ -397,7 +397,7 @@ class DoctorPopup {
Style.EMPTY.dim())));
if (allSmall) {
result.add(Line.from(Span.styled(
- " F8 needs ≥14B — run: ollama pull qwen2.5:14b",
+ " F8 needs ≥14B — run: ollama pull qwen3.6:35b-a3b",
Style.EMPTY.dim())));
}
} else {