[GitHub] [lucene] rmuir commented on pull request #452: LUCENE-10239: upgrade jflex (1.7.0 -> 1.8.2)

[GitBox <gi...@apache.org>]

rmuir commented on pull request #452:
URL: https://github.com/apache/lucene/pull/452#issuecomment-972384034


   I thought to try to do a large refactoring here and quickly backed off, I think let's just upgrade to the latest jflex as a standalone change. 
   
   The trickiest parts (and ones needing close review):
   
   1. the skeleton files. I reviewed `diff -u` of the two current skeleton files, making sure i understood the changes  needed to work without buffer expansion. Then I updated the default skeleton file from jflex sources (there are many changes) and created the "without buffer expansion" version with appropriate logic.
   
   2. jflex change to support > 2GB files. Some variables become `long` and we have to change some code to work with it. In general Lucene won't work with such files (e.g. `OffsetAttribute` is based on `int`), so I tried to just add minimal casts. I'd rather not use `Math.toIntExact` as it may impact performance. If we want to improve safety, maybe it should be via a cheaper mechanism.
   
   3. warnings suppression. jflex starts to try to suppress its own warnings, but they do the warning in a nonstandard way, and you can't stack these annotations, so we have to add a little hack. I commented on the issues in the jflex bug tracker (linked in the gradle hack).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org