You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@xmlgraphics.apache.org by "plutext (via GitHub)" <gi...@apache.org> on 2026/10/02 21:35:13 UTC
[PR] FOP-2918: A surrogate pair is not split by word breaking, and both its units get one bidi level [xmlgraphics-fop]
plutext opened a new pull request, #115:
URL: https://github.com/apache/xmlgraphics-fop/pull/115
Addresses [FOP-2918](https://issues.apache.org/jira/browse/FOP-2918), open since 2020 with a patch from kwilkerson attached. This carries that patch's two changes, the issue's own layout test, a second case the issue did not have, and the root cause underneath both.
### What goes wrong
A character outside the BMP is two UTF-16 units. `TextLayoutManager` ends a word where the bidi level changes or, with `font-selection-strategy="character-by-character"`, where the selected font changes. Either could fall between the two units of a pair, and `MultiByteFont.mapCharsToGlyphs` then rejected the fragment:
```
java.lang.IllegalArgumentException: ill-formed UTF-16 sequence, contains isolated high surrogate at end of sequence
```
No PDF is produced.
### The commits
1. and 2. The word-splitting guard on the bidi-level branch and the `CharUtilities.containsSurrogatePairAt` bound (`>` to `>=`). These are the same two changes as `2918.patch`; they reached this branch by way of the Metanorma fork, whose commits are kept with their author.
3. `containsSurrogatePairAt` raises the documented `IllegalArgumentException` for a high surrogate at the end of a sequence, with a test.
4. The same guard on the font-selection branch, which the issue did not cover: an emoji under character-by-character font selection (Aegean600 in the test tree) failed the same way. `PDFEncodingTestCase` renders it.
5. **The root cause.** `UnicodeBidiAlgorithm.resolveLevels(CharSequence, Direction)` leaves a placeholder in the slot of each low surrogate, and its javadoc says both members of a pair come back with the same level. They did not: no rule resolves the placeholder's class, so it stayed at the embedding level while its character took the level of its script. U+10826 (Cypriot, right-to-left) came back as levels 1 and 0, which is why the word was split there in the first place. With the guards alone the word is kept whole and the line's reordering then asserts in `InlineRun.split` ("heterogeneous inlines not yet supported"). Each placeholder now takes the level of the character before it. `SurrogatePairLevelsTestCase` holds it.
6. `wordbreak_surrogates.xml`, the layout test from `2918.patch`, unchanged, credited to kwilkerson. It fails on `main` with the exception above, fails with the guards alone on the assertion, and passes with commit 5.
### Tests
`SurrogatePairLevelsTestCase`, `CharUtilitiesTestCase`, `PDFEncodingTestCase`, and the layout engine test `wordbreak_surrogates.xml`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xmlgraphics.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: commits-help@xmlgraphics.apache.org
Re: [PR] FOP-2918: A surrogate pair is not split by word breaking, and both its units get one bidi level [xmlgraphics-fop]
Posted by "plutext (via GitHub)" <gi...@apache.org>.
plutext commented on PR #115:
URL: https://github.com/apache/xmlgraphics-fop/pull/115#issuecomment-5962990508
Added a seventh commit. Commit 5's level copy fixes a right-to-left character outside the BMP, but not a neutral one: the low surrogate's placeholder had a class of its own, which ended rule N1's run of neutrals, so an emoji (U+1F300 here) between two Hebrew words resolved to the embedding direction and cut the right-to-left run in two, where U+263A resolves with the text. The placeholder now takes its character's class. Two tests added; the neutral one fails on commit 5 alone. Full build (`mvn -B package checkstyle:check spotbugs:check`) green on `main`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xmlgraphics.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: commits-help@xmlgraphics.apache.org