You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@maka.apache.org by "sunrioa (via GitHub)" <gi...@apache.org> on 2026/10/22 09:33:02 UTC
[PR] feat(ui): highlight Java, Go, Rust, C/C++, C#, and SQL Markdown blocks [maka]
sunrioa opened a new pull request, #6239:
URL: https://github.com/apache/maka/pull/6239
## Summary
Fixes #6238
Markdown code blocks for Java, Go, Rust, C/C++, C#, and SQL currently render without syntax colors. This PR adds highlighting through Astryx's existing `tokenizer` extension point.
- Lazily load the selected Refractor grammars and map their tokens to Astryx's existing syntax colors. Reuse the current `CodeBlock`, copy button, collapse behavior, and light/dark themes.
- Normalize language aliases, including `c++` / `cpp` and `c#` / `csharp`. Preserve document-global UTF-16 offsets and split multiline tokens at line boundaries.
- Keep existing built-in highlighting unchanged and update the generated third-party notices for the added dependency.
Scope is shared Markdown code blocks. Editors, diff previews, and standalone tool-output code blocks are unchanged.
## Highlighting limits
The custom tokenizer reads prepared tokens rather than doing grammar work during React rendering. Streaming updates are debounced by 120 ms, and obsolete results are discarded when the source or language changes.
Extra-language highlighting is limited to 20,000 UTF-16 code units per block. Larger blocks, unknown languages, and highlighting failures remain readable as plain code without truncation. The optional grammar bundle is approximately 20 KB gzipped and loads only when needed.
## Verification
- UI suite: **819 tests passed**, including 24 new tokenizer and Markdown integration cases.
- Chromium suite: **7 tests passed**, covering actual light/dark colors, aliases, lazy loading, clipboard contents, collapse controls, streaming/source changes, failed chunk loading, and long-code fallback.
- Build, typecheck, lint, format check, Knip for UI/Desktop, renderer architecture checks, third-party notices, and ASF header checks passed locally.
- Compared identical snippets against the unmodified baseline and the patched renderer. The contributor also manually tested the local macOS Desktop build and confirmed that highlighting works.
Not run: the full all-workspace test suite or native Windows/Linux application tests.
### Before and after
The screenshots use the same Markdown snippets, Maka theme, and code-block components before and after the change.
| Theme | Before | After |
| --- | --- | --- |
| Dark |  |  |
| Light |  |  |
## AI use
- [ ] No generative tool made a substantive contribution
- [x] Generative tooling made a substantive contribution
Tool(s) and scope: OpenAI Codex assisted with implementation, regression tests, local verification, and this PR description. The contributor manually tested the local Desktop build. This PR is published by Codex on the contributor's behalf after their review and approval.
## Checklist
- [x] Tests cover the change and fail without it
- [x] Lint, format, typecheck and the affected suites pass locally
Does this PR entail a change in behavior?
- [x] Yes — described under Summary above
- [ ] No
<details>
<summary>简体中文</summary>
## 修改概述
关联并解决 #6238。
目前 Java、Go、Rust、C/C++、C# 和 SQL 的 Markdown 代码块没有语法颜色。本 PR 通过 Astryx 现有的 `tokenizer` 扩展接口补充高亮。
- 按需加载所选 Refractor 语法定义，将分词结果映射到 Astryx 现有配色。继续复用原来的 `CodeBlock`、复制按钮、折叠行为和明暗主题。
- 统一识别 `c++` / `cpp`、`c#` / `csharp` 等语言别名。保留全文 UTF-16 偏移，并按行拆分多行 token，避免后续行高亮错位。
- 原有内置语言的高亮保持不变，同时更新新增依赖对应的第三方许可声明。
修改范围仅限共享 Markdown 代码块，不涉及编辑器、diff 预览或独立的工具输出代码块。
## 高亮限制
自定义 tokenizer 只读取预先准备好的分词结果，不在 React 渲染过程中执行语法分析。流式更新使用 120 毫秒防抖；代码内容或语言发生变化后，过期结果不会应用到新内容上。
新增语言的高亮上限为每个代码块 20,000 个 UTF-16 代码单元。超过上限、语言未知或高亮失败时，仍完整显示纯文本代码，不截断原文。新增语法包 gzip 后约 20 KB，仅在需要时加载。
## 验证结果
- UI 测试：**819 项通过**，包含新增的 24 项分词器及 Markdown 集成测试。
- Chromium 测试：**7 项通过**，覆盖明暗主题实际颜色、语言别名、懒加载、复制原文、折叠控件、流式与内容切换、资源加载失败，以及长代码回退。
- 本地构建、类型检查、lint、格式检查、UI/Desktop 的 Knip 检查、渲染器架构检查、第三方许可声明及 ASF 文件头检查均通过。
- 已使用相同代码对比未修改的基线和修改后的渲染结果；贡献者也已在本地 macOS Desktop 中手动测试，确认高亮可用。
未执行：全仓库所有工作区测试，以及 Windows/Linux 原生应用测试。
明暗主题的前后对比截图见上方英文正文。
## AI 使用说明
OpenAI Codex 协助完成实现、回归测试、本地验证及 PR 描述。贡献者已手动测试本地 Desktop 构建。本 PR 发布前已由贡献者本人审核并批准，由 Codex 代为发布。
## 检查清单
- [x] 测试覆盖本次修改，缺少相应实现时无法通过
- [x] 本地 lint、格式、类型检查及受影响测试均通过
- [x] 存在行为变化，已在修改概述中说明
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
Re: [PR] feat(ui): highlight Java, Go, Rust, C/C++, C#, and SQL Markdown blocks [maka]
Posted by "orangeCatDeveloper (via GitHub)" <gi...@apache.org>.
orangeCatDeveloper merged PR #6239:
URL: https://github.com/apache/maka/pull/6239
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org