[PR] (fix) Spellcheck correction loses the capital of a supplementary-plane initial (opennlp)

["krickert (via GitHub)" <gi...@apache.org>]

krickert opened a new pull request, #1385:
URL: https://github.com/apache/opennlp/pull/1385

   Fixes #1380
   
   `SpellCheckingCharSequenceNormalizer` checked letters and case one `char` at a time. A token starting with a supplementary-plane capital, such as U+10400, lost its capital in the correction, and a token made only of supplementary letters was never corrected. The letter check and the casing now work on code points. BMP behavior does not change.
   
   A test covers upper, initial-capital and lower case tokens with supplementary letters, with punctuation and Unicode whitespace around them. The manual notes the behavior.
   
   ### For all changes:
   - [x] Is there a GitHub issue associated with this PR? Is it referenced in the PR description?
   - [x] Has your PR been rebased against the latest commit within the target branch (typically main)?
   - [x] Is your initial contribution a single, squashed commit?
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via mvn clean install at the root opennlp folder?
   - [x] Have you written or updated unit tests to verify your changes?
   - n/a: new dependencies, LICENSE, NOTICE
   
   ### For documentation related changes:
   - [x] Have you ensured that format looks appropriate for the output in which it is rendered?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-1380: Spellcheck correction loses the capital of a supplementary-plane initial (opennlp)

["krickert (via GitHub)" <gi...@apache.org>]

krickert commented on PR #1385:
URL: https://github.com/apache/opennlp/pull/1385#issuecomment-6002834302

   This should be g2g.. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-1380: Spellcheck correction loses the capital of a supplementary-plane initial (opennlp)

["rzo1 (via GitHub)" <gi...@apache.org>]

rzo1 merged PR #1385:
URL: https://github.com/apache/opennlp/pull/1385


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] (fix) GH-1380: Spellcheck correction loses the capital of a supplementary-plane initial (opennlp)

["rzo1 (via GitHub)" <gi...@apache.org>]

rzo1 commented on code in PR #1385:
URL: https://github.com/apache/opennlp/pull/1385#discussion_r4178510785


##########
opennlp-extensions/opennlp-spellcheck/src/main/java/opennlp/spellcheck/normalizer/SpellCheckingCharSequenceNormalizer.java:
##########
@@ -300,48 +305,61 @@ private boolean isCorrectable(String core) {
     if (skipNumbers && isNumberLike(core)) {
       return false;
     }
-    // A token with no letters at all (pure symbols) cannot be a spelling error.

Review Comment:
   I think this comment is still valid, so perhaps restore it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org