You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Alessandro Solimando <al...@gmail.com> on 2023/01/27 17:35:35 UTC
several JavaCC warnings "Choice conflict involving two expansions"
Hello everyone,
while checking CI logs I have noticed that we have lots of JavaCC warnings
related to ambiguous prefixes in the productions of one of our grammars.
They also seem related to time functions, for which I have seen several
related developments for BigQuery lately.
Have we verified that our grammar is still behaving properly under this
situation? Have we considered increasing the lookahead value as suggested?
Shall we open a Jira ticket to have a closer look?
Here is an example of CI logs showing the problem (although it is
reproducible locally):
https://ci-builds.apache.org/job/Calcite/job/Calcite-sonar/job/main/18/consoleFull
In what follows the extract that is relevant to the discussion at hand:
> > Task :core:javaCCMain
> Java Compiler Compiler Version 4.0 (Parser Generator)
> (type "javacc" with no arguments for help)
> Reading from file
> /home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/fmpp/fmppMain/javacc/Parser.jj
> . . .
> Warning: Output directory
> "/home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/javacc/javaCCMain/org/apache/calcite/sql/parser/impl"
> does not exist. Creating the directory.
> Note: UNICODE_INPUT option is specified. Please make sure you create the
> parser/lexer using a Reader with the correct character encoding.
> Warning: Choice conflict involving two expansions at
> line 4930, column 5 and line 4956, column 5 respectively.
> A common prefix is: "MICROSECOND"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4931, column 5 and line 4956, column 5 respectively.
> A common prefix is: "MILLISECOND"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4936, column 5 and line 4956, column 5 respectively.
> A common prefix is: "DOW"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4937, column 5 and line 4956, column 5 respectively.
> A common prefix is: "DOY"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4938, column 5 and line 4956, column 5 respectively.
> A common prefix is: "ISODOW"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4939, column 5 and line 4956, column 5 respectively.
> A common prefix is: "ISOYEAR"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4940, column 5 and line 4956, column 5 respectively.
> A common prefix is: "WEEK"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4950, column 5 and line 4956, column 5 respectively.
> A common prefix is: "QUARTER"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4952, column 5 and line 4956, column 5 respectively.
> A common prefix is: "EPOCH"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4953, column 5 and line 4956, column 5 respectively.
> A common prefix is: "DECADE"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4954, column 5 and line 4956, column 5 respectively.
> A common prefix is: "CENTURY"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4955, column 5 and line 4956, column 5 respectively.
> A common prefix is: "MILLENNIUM"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 6549, column 9 and line 6551, column 9 respectively.
> A common prefix is: "WEEK" "("
> Consider using a lookahead of 3 or more for earlier expansion.
> File "TokenMgrError.java" does not exist. Will create one.
> File "ParseException.java" does not exist. Will create one.
> File "Token.java" does not exist. Will create one.
> File "SimpleCharStream.java" does not exist. Will create one.
> Parser generated with 0 errors and 14 warnings.
Best regards,
Alessandro
Re: several JavaCC warnings "Choice conflict involving two expansions"
Posted by Alessandro Solimando <al...@gmail.com>.
Thanks Julian, I have logged CALCITE-5505
<https://issues.apache.org/jira/browse/CALCITE-5505> to keep this under our
radar.
Best regards,
Alessandro
On Sat, 28 Jan 2023 at 00:26, Julian Hyde <jh...@apache.org> wrote:
> Good catch. Yes, we want to stay on top of these kinds of warnings.
> (See broken windows theory [1].) Probably introduced by
> https://issues.apache.org/jira/browse/CALCITE-5450. Probably missing
> one or two LOOKAHEAD directives in the parser. Can you log a JIRA case
> please.
>
> Julian
>
> [1] https://en.wikipedia.org/wiki/Broken_windows_theory
>
>
> On Fri, Jan 27, 2023 at 9:50 AM Alessandro Solimando
> <al...@gmail.com> wrote:
> >
> > Hello everyone,
> > while checking CI logs I have noticed that we have lots of JavaCC
> warnings
> > related to ambiguous prefixes in the productions of one of our grammars.
> >
> > They also seem related to time functions, for which I have seen several
> > related developments for BigQuery lately.
> >
> > Have we verified that our grammar is still behaving properly under this
> > situation? Have we considered increasing the lookahead value as
> suggested?
> > Shall we open a Jira ticket to have a closer look?
> >
> > Here is an example of CI logs showing the problem (although it is
> > reproducible locally):
> >
> https://ci-builds.apache.org/job/Calcite/job/Calcite-sonar/job/main/18/consoleFull
> >
> >
> > In what follows the extract that is relevant to the discussion at hand:
> >
> > > > Task :core:javaCCMain
> > > Java Compiler Compiler Version 4.0 (Parser Generator)
> > > (type "javacc" with no arguments for help)
> > > Reading from file
> > >
> /home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/fmpp/fmppMain/javacc/Parser.jj
> > > . . .
> > > Warning: Output directory
> > >
> "/home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/javacc/javaCCMain/org/apache/calcite/sql/parser/impl"
> > > does not exist. Creating the directory.
> > > Note: UNICODE_INPUT option is specified. Please make sure you create
> the
> > > parser/lexer using a Reader with the correct character encoding.
> > > Warning: Choice conflict involving two expansions at
> > > line 4930, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "MICROSECOND"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4931, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "MILLISECOND"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4936, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "DOW"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4937, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "DOY"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4938, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "ISODOW"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4939, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "ISOYEAR"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4940, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "WEEK"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4950, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "QUARTER"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4952, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "EPOCH"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4953, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "DECADE"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4954, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "CENTURY"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 4955, column 5 and line 4956, column 5 respectively.
> > > A common prefix is: "MILLENNIUM"
> > > Consider using a lookahead of 2 for earlier expansion.
> > > Warning: Choice conflict involving two expansions at
> > > line 6549, column 9 and line 6551, column 9 respectively.
> > > A common prefix is: "WEEK" "("
> > > Consider using a lookahead of 3 or more for earlier expansion.
> > > File "TokenMgrError.java" does not exist. Will create one.
> > > File "ParseException.java" does not exist. Will create one.
> > > File "Token.java" does not exist. Will create one.
> > > File "SimpleCharStream.java" does not exist. Will create one.
> > > Parser generated with 0 errors and 14 warnings.
> >
> >
> > Best regards,
> > Alessandro
>
Re: several JavaCC warnings "Choice conflict involving two expansions"
Posted by Julian Hyde <jh...@apache.org>.
Good catch. Yes, we want to stay on top of these kinds of warnings.
(See broken windows theory [1].) Probably introduced by
https://issues.apache.org/jira/browse/CALCITE-5450. Probably missing
one or two LOOKAHEAD directives in the parser. Can you log a JIRA case
please.
Julian
[1] https://en.wikipedia.org/wiki/Broken_windows_theory
On Fri, Jan 27, 2023 at 9:50 AM Alessandro Solimando
<al...@gmail.com> wrote:
>
> Hello everyone,
> while checking CI logs I have noticed that we have lots of JavaCC warnings
> related to ambiguous prefixes in the productions of one of our grammars.
>
> They also seem related to time functions, for which I have seen several
> related developments for BigQuery lately.
>
> Have we verified that our grammar is still behaving properly under this
> situation? Have we considered increasing the lookahead value as suggested?
> Shall we open a Jira ticket to have a closer look?
>
> Here is an example of CI logs showing the problem (although it is
> reproducible locally):
> https://ci-builds.apache.org/job/Calcite/job/Calcite-sonar/job/main/18/consoleFull
>
>
> In what follows the extract that is relevant to the discussion at hand:
>
> > > Task :core:javaCCMain
> > Java Compiler Compiler Version 4.0 (Parser Generator)
> > (type "javacc" with no arguments for help)
> > Reading from file
> > /home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/fmpp/fmppMain/javacc/Parser.jj
> > . . .
> > Warning: Output directory
> > "/home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/javacc/javaCCMain/org/apache/calcite/sql/parser/impl"
> > does not exist. Creating the directory.
> > Note: UNICODE_INPUT option is specified. Please make sure you create the
> > parser/lexer using a Reader with the correct character encoding.
> > Warning: Choice conflict involving two expansions at
> > line 4930, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "MICROSECOND"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4931, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "MILLISECOND"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4936, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "DOW"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4937, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "DOY"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4938, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "ISODOW"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4939, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "ISOYEAR"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4940, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "WEEK"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4950, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "QUARTER"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4952, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "EPOCH"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4953, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "DECADE"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4954, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "CENTURY"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 4955, column 5 and line 4956, column 5 respectively.
> > A common prefix is: "MILLENNIUM"
> > Consider using a lookahead of 2 for earlier expansion.
> > Warning: Choice conflict involving two expansions at
> > line 6549, column 9 and line 6551, column 9 respectively.
> > A common prefix is: "WEEK" "("
> > Consider using a lookahead of 3 or more for earlier expansion.
> > File "TokenMgrError.java" does not exist. Will create one.
> > File "ParseException.java" does not exist. Will create one.
> > File "Token.java" does not exist. Will create one.
> > File "SimpleCharStream.java" does not exist. Will create one.
> > Parser generated with 0 errors and 14 warnings.
>
>
> Best regards,
> Alessandro
Fwd: several JavaCC warnings "Choice conflict involving two expansions"
Posted by Alessandro Solimando <al...@gmail.com>.
Hello everyone,
while checking CI logs I have noticed that we have lots of JavaCC warnings
related to ambiguous prefixes in the productions of one of our grammars.
They also seem related to time functions, for which I have seen several
related developments for BigQuery lately.
Have we verified that our grammar is still behaving properly under this
situation? Have we considered increasing the lookahead value as suggested?
Shall we open a Jira ticket to have a closer look?
Here is an example of CI logs showing the problem (although it is
reproducible locally):
https://ci-builds.apache.org/job/Calcite/job/Calcite-sonar/job/main/18/consoleFull
In what follows the extract that is relevant to the discussion at hand:
> > Task :core:javaCCMain
> Java Compiler Compiler Version 4.0 (Parser Generator)
> (type "javacc" with no arguments for help)
> Reading from file
> /home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/fmpp/fmppMain/javacc/Parser.jj
> . . .
> Warning: Output directory
> "/home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/javacc/javaCCMain/org/apache/calcite/sql/parser/impl"
> does not exist. Creating the directory.
> Note: UNICODE_INPUT option is specified. Please make sure you create the
> parser/lexer using a Reader with the correct character encoding.
> Warning: Choice conflict involving two expansions at
> line 4930, column 5 and line 4956, column 5 respectively.
> A common prefix is: "MICROSECOND"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4931, column 5 and line 4956, column 5 respectively.
> A common prefix is: "MILLISECOND"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4936, column 5 and line 4956, column 5 respectively.
> A common prefix is: "DOW"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4937, column 5 and line 4956, column 5 respectively.
> A common prefix is: "DOY"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4938, column 5 and line 4956, column 5 respectively.
> A common prefix is: "ISODOW"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4939, column 5 and line 4956, column 5 respectively.
> A common prefix is: "ISOYEAR"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4940, column 5 and line 4956, column 5 respectively.
> A common prefix is: "WEEK"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4950, column 5 and line 4956, column 5 respectively.
> A common prefix is: "QUARTER"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4952, column 5 and line 4956, column 5 respectively.
> A common prefix is: "EPOCH"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4953, column 5 and line 4956, column 5 respectively.
> A common prefix is: "DECADE"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4954, column 5 and line 4956, column 5 respectively.
> A common prefix is: "CENTURY"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 4955, column 5 and line 4956, column 5 respectively.
> A common prefix is: "MILLENNIUM"
> Consider using a lookahead of 2 for earlier expansion.
> Warning: Choice conflict involving two expansions at
> line 6549, column 9 and line 6551, column 9 respectively.
> A common prefix is: "WEEK" "("
> Consider using a lookahead of 3 or more for earlier expansion.
> File "TokenMgrError.java" does not exist. Will create one.
> File "ParseException.java" does not exist. Will create one.
> File "Token.java" does not exist. Will create one.
> File "SimpleCharStream.java" does not exist. Will create one.
> Parser generated with 0 errors and 14 warnings.
Best regards,
Alessandro
Re: several JavaCC warnings "Choice conflict involving two expansions"
Posted by Alessandro Solimando <al...@gmail.com>.
My apologies, Hive ML wasn't obviously the right target!
Best regards,
Alessandro
On Fri, 27 Jan 2023 at 18:35, Alessandro Solimando <
alessandro.solimando@gmail.com> wrote:
> Hello everyone,
> while checking CI logs I have noticed that we have lots of JavaCC warnings
> related to ambiguous prefixes in the productions of one of our grammars.
>
> They also seem related to time functions, for which I have seen several
> related developments for BigQuery lately.
>
> Have we verified that our grammar is still behaving properly under this
> situation? Have we considered increasing the lookahead value as suggested?
> Shall we open a Jira ticket to have a closer look?
>
> Here is an example of CI logs showing the problem (although it is
> reproducible locally):
> https://ci-builds.apache.org/job/Calcite/job/Calcite-sonar/job/main/18/consoleFull
>
>
> In what follows the extract that is relevant to the discussion at hand:
>
>> > Task :core:javaCCMain
>> Java Compiler Compiler Version 4.0 (Parser Generator)
>> (type "javacc" with no arguments for help)
>> Reading from file
>> /home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/fmpp/fmppMain/javacc/Parser.jj
>> . . .
>> Warning: Output directory
>> "/home/jenkins/jenkins-agent/workspace/Calcite_Calcite-sonar_main/core/build/javacc/javaCCMain/org/apache/calcite/sql/parser/impl"
>> does not exist. Creating the directory.
>> Note: UNICODE_INPUT option is specified. Please make sure you create the
>> parser/lexer using a Reader with the correct character encoding.
>> Warning: Choice conflict involving two expansions at
>> line 4930, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "MICROSECOND"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4931, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "MILLISECOND"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4936, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "DOW"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4937, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "DOY"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4938, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "ISODOW"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4939, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "ISOYEAR"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4940, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "WEEK"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4950, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "QUARTER"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4952, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "EPOCH"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4953, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "DECADE"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4954, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "CENTURY"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 4955, column 5 and line 4956, column 5 respectively.
>> A common prefix is: "MILLENNIUM"
>> Consider using a lookahead of 2 for earlier expansion.
>> Warning: Choice conflict involving two expansions at
>> line 6549, column 9 and line 6551, column 9 respectively.
>> A common prefix is: "WEEK" "("
>> Consider using a lookahead of 3 or more for earlier expansion.
>> File "TokenMgrError.java" does not exist. Will create one.
>> File "ParseException.java" does not exist. Will create one.
>> File "Token.java" does not exist. Will create one.
>> File "SimpleCharStream.java" does not exist. Will create one.
>> Parser generated with 0 errors and 14 warnings.
>
>
> Best regards,
> Alessandro
>