[PR] GH-1480: Read Spanish head rules without regular expressions (opennlp)

["krickert (via GitHub)" <gi...@apache.org>]

krickert opened a new pull request, #1482:
URL: https://github.com/apache/opennlp/pull/1482

   AncoraSpanishHeadRules matches head-rule tags with a small matcher instead of java.util.regex. Each tag is parsed once when the rules load into literals, * for zero or more characters and the character choices of the AnCora tagset ([MAS], [CS], [IS], [12], [AC]), and is matched without backtracking. The class has no @SuppressForbidden members left.
   
   Rule files start with the header "# opennlp-ancora-head-rules 2" and write tags without regex escapes; the bundled es-head-rules files use it. A file without the header is read as the regular expression format of earlier releases, with dots and dollar signs escaped once or twice, and converted. Other regex constructs, unknown character choices, a direction other than 0 or 1, a repeated constituent type and blank lines fail to load with the line number and the reason.
   
   Two compatibility points, also in the parser chapter:
   - An escaped dot such as `GRUP\\.VERB` in Spanish models trained before 3.0 matched no constituent type. It now matches `GRUP.VERB`, so those models choose different heads at parse time than in training and need retraining.
   - OpenNLP 2.x cannot read the version 2 format, so Spanish parser models trained with this release do not load there.
   
   Tests: the matcher, rule and constituent priority, the longest pattern and a long near miss, both legacy escape forms, the bundled rules in both earlier forms read through HeadRulesSerializer, each dotted tag of the bundled rules, and each rejected rule with its message.
   
   An eval build is needed before merge, since the bundled Spanish head rules change.
   
   Fixes #1480
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-1480: Read Spanish head rules without regular expressions (opennlp)

["krickert (via GitHub)" <gi...@apache.org>]

krickert commented on PR #1482:
URL: https://github.com/apache/opennlp/pull/1482#issuecomment-6111919014

   > Hi @krickert - thanks for the work here. Playing devil's advocate: I'm not sure we need this PR.
   > 
   > * Head-rule tags are user-supplied regular expressions, which is exactly the case #1348 allows with `@SuppressForbidden`. The guard already passes with that suppression, and the escaping bug in the bundled rules is fixed by #1461.
   > * The new rule format (`# opennlp-ancora-head-rules 2`) is stored in trained models, so Spanish parser models trained with 3.0 would no longer load in 2.x.
   > * Converting legacy rule files changes how existing models pick heads (`GRUP\\.VERB` suddenly matches), so models trained before 3.0 would need retraining. #1461 deliberately avoided that.
   > * Custom rule files that use other regex constructs would stop loading.
   > * No measured performance or correctness problem is addressed, and we would maintain a small regex engine of our own instead of `java.util.regex`.
   > 
   > I'd suggest closing this, or deferring it beyond 3.0.0. If the clearer load errors are the main benefit, we could add line-numbered validation on top of the existing regex matching, without a new format version. WDYT?
   
   I'll close it.  I was looking under covers and looking for any regex that might come out of the woodwork and found 2 PRs, this is one of them.  Let's see how the other one does... 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-1480: Read Spanish head rules without regular expressions (opennlp)

["krickert (via GitHub)" <gi...@apache.org>]

krickert closed pull request #1482: GH-1480: Read Spanish head rules without regular expressions
URL: https://github.com/apache/opennlp/pull/1482


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-1480: Read Spanish head rules without regular expressions (opennlp)

["krickert (via GitHub)" <gi...@apache.org>]

krickert commented on PR #1482:
URL: https://github.com/apache/opennlp/pull/1482#issuecomment-6111921657

   Closing, not needed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org