[PR] Minor: Compose AlignedText stages with AlignedText.andThen (opennlp)

["rzo1 (via GitHub)" <gi...@apache.org>]

rzo1 opened a new pull request, #1358:
URL: https://github.com/apache/opennlp/pull/1358

   Adds AlignedText.andThen to compose one normalization stage onto another, and uses it in the whitespace, line-break, paragraph and quote normalizers, the aligned aggregate normalizer and AbstractDL instead of inline composition. Offsets and output are unchanged.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Minor: Compose AlignedText stages with AlignedText.andThen (opennlp)

["krickert (via GitHub)" <gi...@apache.org>]

krickert commented on PR #1358:
URL: https://github.com/apache/opennlp/pull/1358#issuecomment-5957667661

   Made a small edit to the javadocs.
   
   I pushed one commit on top, details followed by a more specific AI-generated summary:
   
   - The new paragraph on `WordpieceEncoder.MappedText` is not in the description and was inaccurate: positions are UTF-16 code units (see `add`), and a supplementary code point takes two positions that share one range. Dropped it.
   - `{@inheritDoc}` was added to `normalizeAligned` in three normalizers but not to their `normalize`, while `ParagraphPreservingWhitespaceCharSequenceNormalizer` has it on both. Added it on `normalize` too.
   - `AlignedTextTest` had two tests over the same collapse and trim chain. One of them is now a three-stage chain with an expanding stage, compared with the alignments composed by hand over the full offset grid, end offsets included.
   
   AI summary specifics, validated and edited:
   > Chained five stages (collapse, remove, expand U+2026 to "..." and U+03C2 to "ss", identity substitute, trim) over 15 inputs with supplementary characters, combining marks at stage boundaries and expansions at both ends, and compared the old hand-threaded `Alignment.andThen` chain with `AlignedText.andThen`, left-nested and right-nested. All `toOriginalSpan` and `toNormalizedSpan` results over the full offset grid were identical, 614 spans, and `original()` is the same object in all three. The four normalizers keep their serialVersionUIDs; only method bodies changed. `api`, `runtime` and `dl` pass locally.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Minor: Compose AlignedText stages with AlignedText.andThen (opennlp)

["mawiesne (via GitHub)" <gi...@apache.org>]

mawiesne merged PR #1358:
URL: https://github.com/apache/opennlp/pull/1358


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org