You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@maka.apache.org by as...@apache.org on 2026/09/24 18:50:38 UTC
(maka) branch feat/desktop-gpui-spike updated (f843f9b803 -> ed3e23193d)
This is an automated email from the ASF dual-hosted git repository.
Astro-Han pushed a change to branch feat/desktop-gpui-spike
in repository https://gitbox.apache.org/repos/asf/maka.git
from f843f9b803 fix(desktop): ask for older history only when the tail is measured short
new 88a6f3188d feat(desktop): name every control and expose transcript text to assistive technology
new ed3e23193d fix(desktop): size the user bubble to its text
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
crates/desktop/COMPONENTS.md | 5 ++--
crates/desktop/src/chat/composer.rs | 2 +-
crates/desktop/src/chat/permission.rs | 7 ++++-
crates/desktop/src/chat/view.rs | 38 ++++++++++++++++++------
crates/desktop/src/md/view.rs | 54 +++++++++++++++++++++++++++++++----
crates/desktop/src/sidebar.rs | 46 ++++++++++++++++++++---------
crates/desktop/src/ui/mod.rs | 14 ++++++---
7 files changed, 130 insertions(+), 36 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
(maka) 01/02: feat(desktop): name every control and expose transcript text to assistive technology
Posted by as...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
Astro-Han pushed a commit to branch feat/desktop-gpui-spike
in repository https://gitbox.apache.org/repos/asf/maka.git
commit 88a6f3188d13fa70cc86e7a2e1fee579b191bf94
Author: AstroHan <le...@outlook.com>
AuthorDate: Fri Sep 25 02:50:11 2026 +0800
feat(desktop): name every control and expose transcript text to assistive technology
The macOS accessibility tree showed nameless buttons and no transcript
at all: GPUI reports an element only with an id and a role, and never
reports StyledText. `pressable` now takes a name, so a control without
one does not compile; session rows carry their selection and a
description, folds report expanded or collapsed, and the permission card
is a labeled group. Each markdown block gets a node carrying its text
(paragraph, heading level, list item and task state, code, table), and
each message is a group naming who wrote it.
The tree is also how the client is checked without screenshots: this
change was verified by reading it from a background window.
Generated-by: Claude Code
---
crates/desktop/COMPONENTS.md | 5 ++--
crates/desktop/src/chat/composer.rs | 2 +-
crates/desktop/src/chat/permission.rs | 7 ++++-
crates/desktop/src/chat/view.rs | 38 +++++++++++++++++++-------
crates/desktop/src/md/view.rs | 50 ++++++++++++++++++++++++++++++++---
crates/desktop/src/sidebar.rs | 46 ++++++++++++++++++++++----------
crates/desktop/src/ui/mod.rs | 14 +++++++---
7 files changed, 128 insertions(+), 34 deletions(-)
diff --git a/crates/desktop/COMPONENTS.md b/crates/desktop/COMPONENTS.md
index d20ecb6c3b..5b668f0cc1 100644
--- a/crates/desktop/COMPONENTS.md
+++ b/crates/desktop/COMPONENTS.md
@@ -71,9 +71,10 @@ Problems Waku hit and solved, restated as checks on our code.
Every component is done only when the macOS accessibility tree shows it with a role, a name and its state; it is also how components are checked without screenshots. GPUI reports an element only when it has both an id and a role.
-- Done: the sidebar and main area are landmarks; every `pressable` is a button; the composer is a text input (gpui-kit's `Textarea`).
+- Done: the sidebar and main area are landmarks; `pressable` takes a name, so every button has one; session rows carry selection and a description; folds report expanded or collapsed; the permission card is a labeled group; the composer is a text input (gpui-kit's `Textarea`).
+- `StyledText` is never reported, so each markdown block has its own node carrying its text (paragraph, heading level, list item and task state, code, table), and each message is a group naming who wrote it.
- GPUI drops a cached view's nodes when it reuses the view's last frame, so views are drawn uncached while assistive technology is listening (`workspace::embed`). To report upstream.
-- To do: icon buttons need a name (reuse the tooltip text); session rows become list items with their position; transcript text is `StyledText`, which GPUI never reports, so each block needs an id and a role carrying its text (heading level, list item, code, table cells); the permission card is a labeled group; folds report expanded or collapsed.
+- To do: the header title, the composer's status and errors, and the empty and connection states are plain text; list items lack their position; table cells are one label, not cells. Focus is not reported while the window is inactive, so focus checks need the window in front.
### Frame budget
diff --git a/crates/desktop/src/chat/composer.rs b/crates/desktop/src/chat/composer.rs
index 53a6ab2c56..28669fb8aa 100644
--- a/crates/desktop/src/chat/composer.rs
+++ b/crates/desktop/src/chat/composer.rs
@@ -104,7 +104,7 @@ impl Chat {
}),
)
.tooltip(ui::tooltip(tip));
- let action = ui::pressable(action, move |_, cx| {
+ let action = ui::pressable(action, tip, move |_, cx| {
let _ = this.update(cx, |chat, cx| {
if chat.running().is_some() {
chat.stop(cx);
diff --git a/crates/desktop/src/chat/permission.rs b/crates/desktop/src/chat/permission.rs
index 2d17d00287..a8374c57b9 100644
--- a/crates/desktop/src/chat/permission.rs
+++ b/crates/desktop/src/chat/permission.rs
@@ -28,7 +28,8 @@ use crate::{
};
use gpui_kit::{
AnyElement, Context, Focusable, FontWeight, InteractiveElement, IntoElement, ParentElement,
- SharedString, StatefulInteractiveElement, Styled, Window, div, prelude::FluentBuilder, px,
+ Role, SharedString, StatefulInteractiveElement, Styled, Window, div, prelude::FluentBuilder,
+ px,
};
use maka_protocol::interaction::{self, InteractionRequest};
use serde_json::{Value, json};
@@ -166,6 +167,10 @@ impl Chat {
.pb(px(8.))
.child(
div()
+ .id("interaction")
+ .role(Role::Group)
+ .aria_label(title)
+ .aria_description(detail.clone())
.w_full()
.max_w(px(super::view::COLUMN))
.p(px(12.))
diff --git a/crates/desktop/src/chat/view.rs b/crates/desktop/src/chat/view.rs
index 3404d39cec..57dbe4fa2b 100644
--- a/crates/desktop/src/chat/view.rs
+++ b/crates/desktop/src/chat/view.rs
@@ -30,7 +30,7 @@ use crate::{
use chrono::{Local, TimeZone};
use gpui_kit::{
AnyElement, App, Context, Div, Entity, FontWeight, InteractiveElement, IntoElement,
- MouseButton, ParentElement, Render, SharedString, StatefulInteractiveElement, Styled,
+ MouseButton, ParentElement, Render, Role, SharedString, StatefulInteractiveElement, Styled,
StyledText, Window, component::input, div, list, prelude::FluentBuilder, px,
};
use std::time::{Duration, Instant};
@@ -140,6 +140,9 @@ impl Chat {
.justify_end()
.child(
div()
+ .id(key.clone())
+ .role(Role::Group)
+ .aria_label("你的消息")
.max_w(px(540.))
.min_w_0()
.rounded(px(12.))
@@ -162,6 +165,9 @@ impl Chat {
let key: SharedString = format!("text:{id}").into();
match self.bodies.get(key.as_ref()) {
Some(body) => div()
+ .id(key.clone())
+ .role(Role::Group)
+ .aria_label("Maka 的回答")
.py(px(4.))
.child(md::render(
body,
@@ -202,6 +208,7 @@ impl Chat {
div()
.id(group.clone())
.group(group.clone())
+ .aria_expanded(open)
.h(px(26.))
.flex()
.items_center()
@@ -213,7 +220,7 @@ impl Chat {
.text_color(theme.muted)
.hover(|style| style.text_color(theme.text))
.focus_visible(|style| style.text_color(theme.text))
- .child(div().min_w_0().truncate().child(title))
+ .child(div().min_w_0().truncate().child(title.clone()))
.child(
icon(
if open {
@@ -226,6 +233,7 @@ impl Chat {
.size(px(10.))
.group_hover(group, |style| style.text_color(theme.text)),
),
+ title,
move |_, cx| {
let key = toggle_key.clone();
let _ = chat.update(cx, |chat, cx| chat.toggle(key, !open, row, cx));
@@ -316,6 +324,12 @@ impl Chat {
} else {
div().into_any_element()
};
+ let name = match (target.is_empty(), failed) {
+ (true, false) => kind.verb().to_owned(),
+ (true, true) => format!("{}（失败）", kind.verb()),
+ (false, false) => format!("{} {target}", kind.verb()),
+ (false, true) => format!("{} {target}（失败）", kind.verb()),
+ };
let header = div()
.id(SharedString::from(format!("{item_key}:header")))
.h(px(28.))
@@ -340,15 +354,17 @@ impl Chat {
let key = item_key.clone();
ui::pressable(
header
+ .aria_expanded(open)
.hover(|style| style.bg(theme.hover))
.focus_visible(|style| style.bg(theme.hover)),
+ name,
move |_, cx| {
let key = key.clone();
let _ = chat.update(cx, |chat, cx| chat.toggle(key, !open, row, cx));
},
)
} else {
- header
+ header.role(Role::Label).aria_label(name)
};
div()
.flex()
@@ -483,6 +499,7 @@ impl Chat {
div()
.id(group.clone())
.group(group.clone())
+ .aria_expanded(open)
.h(px(24.))
.flex()
.items_center()
@@ -500,7 +517,7 @@ impl Chat {
.text_size(px(13.5))
.line_height(px(18.))
.font_weight(FontWeight::MEDIUM)
- .child(label)
+ .child(label.clone())
.child(
icon(
if open {
@@ -515,6 +532,7 @@ impl Chat {
),
)
.child(line()),
+ label,
move |_, cx| {
let turn = turn.clone();
let _ = chat.update(cx, |chat, cx| chat.toggle_turn(turn, cx));
@@ -541,6 +559,7 @@ impl Chat {
let text = answer.join("\n\n");
let copy_key: SharedString = format!("footer:{turn}").into();
let copied = self.copied.read(cx).is(©_key);
+ let tip = if copied { "已复制" } else { "复制回答" };
let source = self.copied.clone();
let time = self
.transcript
@@ -564,6 +583,7 @@ impl Chat {
} else {
"icons/copy.svg"
},
+ tip,
false,
move |_, cx| {
let text = text.clone();
@@ -572,11 +592,7 @@ impl Chat {
},
cx,
)
- .tooltip(ui::tooltip(if copied {
- "已复制"
- } else {
- "复制回答"
- })),
+ .tooltip(ui::tooltip(tip)),
)
.when_some(time, |this, time| {
this.child(
@@ -607,6 +623,9 @@ impl Chat {
None => "正在工作".to_owned(),
};
div()
+ .id("working")
+ .role(Role::Status)
+ .aria_label(label.clone())
.h(px(22.))
.flex()
.items_center()
@@ -661,6 +680,7 @@ impl Chat {
.hover(|style| style.bg(theme.overlay))
.focus_visible(|style| style.border_color(theme.accent))
.child(icon("icons/arrow-down.svg", theme.text).size(px(16.))),
+ "回到最新",
move |_, cx| {
let _ = chat.update(cx, |chat, cx| chat.jump_to_latest(cx));
},
diff --git a/crates/desktop/src/md/view.rs b/crates/desktop/src/md/view.rs
index bb53ce1cad..dffda21d24 100644
--- a/crates/desktop/src/md/view.rs
+++ b/crates/desktop/src/md/view.rs
@@ -33,8 +33,8 @@ use crate::{
};
use gpui_kit::{
AnyElement, App, Div, ElementId, Entity, FontStyle, FontWeight, HighlightStyle,
- InteractiveElement, InteractiveText, IntoElement, ParentElement, SharedString,
- StatefulInteractiveElement, StrikethroughStyle, Styled, StyledText, div, px, relative,
+ InteractiveElement, InteractiveText, IntoElement, ParentElement, Role, SharedString, Stateful,
+ StatefulInteractiveElement, StrikethroughStyle, Styled, StyledText, Toggled, div, px, relative,
};
use std::{ops::Range, time::Instant};
@@ -209,12 +209,56 @@ pub fn render(body: &Body, scope: &Scope, cx: &App) -> Div {
(Some(Kind::Item(_)), Kind::Item(_)) => metrics.gap * 0.5,
_ => metrics.gap,
};
- column = column.child(div().pt(px(gap)).min_w_0().child(element));
+ column = column.child(
+ accessible(block, scope, ordinal)
+ .pt(px(gap))
+ .min_w_0()
+ .child(element),
+ );
previous = Some(block);
}
column
}
+/// The block's node for assistive technology, which never sees `StyledText`.
+fn accessible(block: &Block, scope: &Scope, ordinal: u32) -> Stateful<Div> {
+ let text = || block.texts[0].text.clone();
+ let node = div().id(ElementId::NamedInteger(
+ scope.row.clone(),
+ (ordinal | 0xffff) as u64,
+ ));
+ match &block.kind {
+ Kind::Paragraph => node.role(Role::Paragraph).aria_label(text()),
+ Kind::Heading(level) => node
+ .role(Role::Heading)
+ .aria_level(*level as usize)
+ .aria_label(text()),
+ Kind::Item(Marker::Task(done)) => node
+ .role(Role::ListItem)
+ .aria_toggled(if *done { Toggled::True } else { Toggled::False })
+ .aria_label(text()),
+ Kind::Item(_) => node.role(Role::ListItem).aria_label(text()),
+ Kind::Code { .. } => node.role(Role::Code).aria_label(text()),
+ Kind::Rule => node,
+ Kind::Table { columns } => {
+ let rows: Vec<String> = block
+ .texts
+ .chunks(*columns)
+ .map(|row| {
+ row.iter()
+ .map(|cell| cell.text.as_str())
+ .collect::<Vec<_>>()
+ .join(" | ")
+ })
+ .collect();
+ node.role(Role::Table)
+ .aria_row_count(rows.len())
+ .aria_column_count(*columns)
+ .aria_label(rows.join("\n"))
+ }
+ }
+}
+
fn text(
scope: &Scope,
ordinal: u32,
diff --git a/crates/desktop/src/sidebar.rs b/crates/desktop/src/sidebar.rs
index 536a31c53d..f3b01c9c08 100644
--- a/crates/desktop/src/sidebar.rs
+++ b/crates/desktop/src/sidebar.rs
@@ -27,8 +27,8 @@ use crate::{
use chrono::{Datelike, Local, NaiveDate, TimeZone};
use gpui_kit::{
AnyElement, Context, EventEmitter, FontWeight, InteractiveElement, IntoElement, ListAlignment,
- ListState, ParentElement, Render, SharedString, Styled, Task, Transformation, Window, div,
- list, percentage, prelude::FluentBuilder, px,
+ ListState, ParentElement, Render, Role, SharedString, StatefulInteractiveElement, Styled, Task,
+ Transformation, Window, div, list, percentage, prelude::FluentBuilder, px,
};
use maka_protocol::session::{SessionCatalogProjection, SessionStatus};
use std::time::Duration;
@@ -153,6 +153,9 @@ impl Sidebar {
) -> AnyElement {
match self.items.get(ix) {
Some(Item::Group(label)) => div()
+ .id(SharedString::from(self.keys[ix].clone()))
+ .role(Role::Heading)
+ .aria_label(*label)
.h(px(30.))
.px(px(8.))
.pt(px(8.))
@@ -186,19 +189,25 @@ impl Sidebar {
) -> AnyElement {
let theme = theme(cx);
let selected = self.selected.as_deref() == Some(session.id.as_str());
- let status = match session.status {
- SessionStatus::Running => Some(
- icon("icons/loader-circle.svg", theme.accent)
- .size(px(12.))
- .with_transformation(Transformation::rotate(percentage(turn.unwrap_or(0.))))
- .into_any_element(),
+ let (state, status) = match session.status {
+ SessionStatus::Running => (
+ Some("运行中"),
+ Some(
+ icon("icons/loader-circle.svg", theme.accent)
+ .size(px(12.))
+ .with_transformation(Transformation::rotate(percentage(turn.unwrap_or(0.))))
+ .into_any_element(),
+ ),
),
- SessionStatus::WaitingForUser | SessionStatus::Blocked => Some(
- icon("icons/triangle-alert.svg", theme.warning)
- .size(px(12.))
- .into_any_element(),
+ SessionStatus::WaitingForUser | SessionStatus::Blocked => (
+ Some("等待回应"),
+ Some(
+ icon("icons/triangle-alert.svg", theme.warning)
+ .size(px(12.))
+ .into_any_element(),
+ ),
),
- _ => None,
+ _ => (None, None),
};
let project = session
.workspace
@@ -209,12 +218,19 @@ impl Sidebar {
.to_owned();
let id = session.id.clone();
let entity = cx.weak_entity();
+ let when = ago(now, recency(session));
+ let description = match state {
+ Some(state) => format!("{state} · {project} · {when}"),
+ None => format!("{project} · {when}"),
+ };
div()
.h(px(52.))
.pb(px(1.))
.child(ui::pressable(
div()
.id(SharedString::from(format!("session:{}", session.id)))
+ .aria_selected(selected)
+ .aria_description(description)
.h(px(51.))
.px(px(8.))
.py(px(7.))
@@ -254,8 +270,9 @@ impl Sidebar {
.text_color(theme.muted)
.child(icon("icons/folder.svg", theme.muted).size(px(12.)))
.child(div().flex_1().min_w_0().truncate().child(project))
- .child(div().flex_none().child(ago(now, recency(session)))),
+ .child(div().flex_none().child(when)),
),
+ title(session),
move |_, cx| {
let id = id.clone();
let _ = entity.update(cx, |sidebar, cx| {
@@ -323,6 +340,7 @@ impl Render for Sidebar {
.child(icon("icons/square-pen.svg", theme.muted)),
)
.child("新建会话"),
+ "新建会话",
move |_, cx| {
if can_create {
let _ = entity.update(cx, |_, cx| cx.emit(SidebarEvent::Create));
diff --git a/crates/desktop/src/ui/mod.rs b/crates/desktop/src/ui/mod.rs
index 81c13129c2..b105cc5f49 100644
--- a/crates/desktop/src/ui/mod.rs
+++ b/crates/desktop/src/ui/mod.rs
@@ -90,16 +90,19 @@ pub fn splice(list: &ListState, old: &[String], new: &[String]) {
type Handler = Rc<dyn Fn(&mut Window, &mut App)>;
-/// Makes `element` a control: clicked, or activated with bare Enter or Space
-/// once focused. Chords are left alone so their commands still run.
+/// Makes `element` a control named `name` for assistive technology: clicked,
+/// or activated with bare Enter or Space once focused. Chords are left alone
+/// so their commands still run.
pub fn pressable(
element: Stateful<Div>,
+ name: impl Into<SharedString>,
on_press: impl Fn(&mut Window, &mut App) + 'static,
) -> Stateful<Div> {
let on_press: Handler = Rc::new(on_press);
let on_key = on_press.clone();
element
.role(Role::Button)
+ .aria_label(name)
.tab_index(0)
.on_click(move |_: &ClickEvent, window, cx| {
cx.stop_propagation();
@@ -131,6 +134,7 @@ pub fn button(
cx: &App,
) -> Stateful<Div> {
let theme = theme(cx);
+ let label = label.into();
let base = div()
.id(id)
.h(px(28.))
@@ -156,11 +160,11 @@ pub fn button(
this.hover(|style| style.bg(theme.hover).text_color(theme.text))
}),
})
- .child(label.into());
+ .child(label.clone());
if disabled {
base.opacity(0.5)
} else {
- pressable(base, on_press).active(|style| style.opacity(0.8))
+ pressable(base, label, on_press).active(|style| style.opacity(0.8))
}
}
@@ -168,6 +172,7 @@ pub fn button(
pub fn icon_button(
id: impl Into<ElementId>,
path: &'static str,
+ name: impl Into<SharedString>,
small: bool,
on_press: impl Fn(&mut Window, &mut App) + 'static,
cx: &App,
@@ -195,6 +200,7 @@ pub fn icon_button(
.size(px(icon_size))
.group_hover(group, |style| style.text_color(theme.text)),
),
+ name,
on_press,
)
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
(maka) 02/02: fix(desktop): size the user bubble to its text
Posted by as...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
Astro-Han pushed a commit to branch feat/desktop-gpui-spike
in repository https://gitbox.apache.org/repos/asf/maka.git
commit ed3e23193d4e8f350587c501e26a3a3597524ecd
Author: AstroHan <le...@outlook.com>
AuthorDate: Fri Sep 25 02:50:31 2026 +0800
fix(desktop): size the user bubble to its text
The bubble shrinks to its content, but the markdown root was a flex
column, whose max-content width taffy reports as zero; the bubble
collapsed to its padding and wrapped one character per line. A block
root stacks the blocks the same way and reports their real width.
Found through the accessibility tree: the bubble measured 22 px wide
with its text at 0 px over two lines; it now measures 34 px with the
text on one line, and a long prompt caps at 540 px.
Generated-by: Claude Code
---
crates/desktop/src/md/view.rs | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/crates/desktop/src/md/view.rs b/crates/desktop/src/md/view.rs
index dffda21d24..9b17547ed2 100644
--- a/crates/desktop/src/md/view.rs
+++ b/crates/desktop/src/md/view.rs
@@ -125,9 +125,9 @@ pub struct Scope<'a> {
pub fn render(body: &Body, scope: &Scope, cx: &App) -> Div {
let theme = theme(cx);
let metrics = scope.metrics;
+ // A block, not a flex column: taffy sizes a flex column's content to
+ // zero, which shrinks a user bubble to one character per line.
let mut column = div()
- .flex()
- .flex_col()
.min_w_0()
.text_size(px(metrics.text))
.line_height(px(metrics.line))
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org