You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2022/09/19 18:36:28 UTC
[GitHub] [lucene] reta opened a new issue, #11788: Upgrade ANTLR to version 4.11.1
reta opened a new issue, #11788:
URL: https://github.com/apache/lucene/issues/11788
### Description
The Apache Lucene is using quite old version of ANTLR 4.5.1-1. By itseld, it is not a showstopper, but more profound issue is that some ANTLR 3.x bits are used [1]. Since ANTLR 4.10.x (or even earlier), the compatibility layer with `3.x` release line has been dropped in `4.x` (see please [2]), which makes Apache Lucene impossile to use with recent ANTLR 4.10.x+ releases [3]. The sample exception is below.
```
> java.lang.UnsupportedOperationException: java.io.InvalidClassException: org.antlr.v4.runtime.atn.ATN; Could not deserialize ATN with version 3 (expected 4).
> at org.antlr.antlr4.runtime@4.11.1/org.antlr.v4.runtime.atn.ATNDeserializer.deserialize(ATNDeserializer.java:56)
> at org.antlr.antlr4.runtime@4.11.1/org.antlr.v4.runtime.atn.ATNDeserializer.deserialize(ATNDeserializer.java:48)
> at org.apache.lucene.expressions@10.0.0-SNAPSHOT/org.apache.lucene.expressions.js.JavascriptLexer.<clinit>(JavascriptLexer.java:279)
```
[1] https://github.com/apache/lucene/blob/main/lucene/expressions/src/java/org/apache/lucene/expressions/js/JavascriptLexer.java#L189
[2] https://github.com/antlr/antlr4/commit/c68e127a7cf14470565d6e6ae1eff06db3e56ea7
[3] https://github.com/opensearch-project/OpenSearch/pull/4546
@uschindler @jpountz any objections in migrating to ANTLR `4.11.1`? I would be happy to offer my help here, thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir commented on issue #11788: Upgrade ANTLR to version 4.11.1
Posted by GitBox <gi...@apache.org>.
rmuir commented on issue #11788:
URL: https://github.com/apache/lucene/issues/11788#issuecomment-1254645510
looks like an antlr problem, if they broke backwards compat, they prolly should have named it `5.x`?
let's be careful about upgrading to new versions. newer antlr versions have historically been trappy, e.g. happily doing extremely slow things instead of simply failing at "compile time" if there are problems in the grammar.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] reta commented on issue #11788: Upgrade ANTLR to version 4.11.1
Posted by GitBox <gi...@apache.org>.
reta commented on issue #11788:
URL: https://github.com/apache/lucene/issues/11788#issuecomment-1254977405
@rmuir @uschindler thanks guys
> looks like an antlr problem, if they broke backwards compat, they prolly should have named it 5.x?
Sadly I don't know the story, I believe it was merged / reverted / and than brought up again.
> let's be careful about upgrading to new versions. newer antlr versions have historically been trappy, e.g. happily doing extremely slow things instead of simply failing at "compile time" if there are problems in the grammar.
I see the risks now, may be we could explore the route to convert problematic serialized blobs from v3 to v4?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir closed issue #11788: Upgrade ANTLR to version 4.11.1
Posted by GitBox <gi...@apache.org>.
rmuir closed issue #11788: Upgrade ANTLR to version 4.11.1
URL: https://github.com/apache/lucene/issues/11788
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] uschindler commented on issue #11788: Upgrade ANTLR to version 4.11.1
Posted by GitBox <gi...@apache.org>.
uschindler commented on issue #11788:
URL: https://github.com/apache/lucene/issues/11788#issuecomment-1254658670
Thanks Robert. I would have said the same. In the worst case we should (like most projects do for ASM, e.g. forbidden apis) shade the antrlr runtime to lucenes package name and include into the expressions jar.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir commented on issue #11788: Upgrade ANTLR to version 4.11.1
Posted by GitBox <gi...@apache.org>.
rmuir commented on issue #11788:
URL: https://github.com/apache/lucene/issues/11788#issuecomment-1255064053
i'd prefer not changing anything without addressing the testing. I need to reiterate just how insanely trappy antlr v4 is. for painless to work with v4 and prevent insanely slow performance we used some tricks to fail tests instead of doing slow things: https://github.com/opensearch-project/OpenSearch/blob/main/modules/lang-painless/src/main/java/org/opensearch/painless/antlr/Walker.java#L224-L245
It is still not as good as "compile-time" checking of the grammar, because you need 100% test coverage to ensure things never go slow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] reta commented on issue #11788: Upgrade ANTLR to version 4.11.1
Posted by GitBox <gi...@apache.org>.
reta commented on issue #11788:
URL: https://github.com/apache/lucene/issues/11788#issuecomment-1256210216
:+1: thanks @rmuir, I will start with tests first (with respect to the changes needed) and we could make the decision having the evidence / numbers at hand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org