You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Roy <ro...@gmail.com> on 2004/08/02 21:30:24 UTC
Is the latest StandardTokenizer.jj corrupted?
The latest StandardTokenizer.jj in cvs repository seems to be corrupted.
I used ant javacc-StandardAnalyzer to regenerate the java code, then
tried to rebuild the lucene package but got some strange errors as
follows. It seems javacc didn't generated correct code against the
latest jj file: the last two errors indicate there are some characters
missing from those symbols. I checked out the 1.4 jj file and did the
same thing again. It works! So I think there's some problems with the
latest version. The diff shows the change is minor. I didn't figure
the cause. Maybe somebody in the core team can look into this?
My purpose is to modify the StandardTokenizer.jj to build my own. But
I didn't succeed. Then I found out even the vanilla jj has some
problems.
compile-core:
[javac] Compiling 160 source files to
/home/roy/jakarta-lucene/build/classes/java
[javac] /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/StandardTokenizer.java:15:
cannot resolve symbol
[javac] symbol : class Reader
[javac] location: class
org.apache.lucene.analysis.standard.StandardTokenizer
[javac] public StandardTokenizer(Reader reader) {
[javac] ^
[javac] /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/StandardTokenizer.java:24:
cannot resolve symbol
[javac] symbol : class IOException
[javac] location: class
org.apache.lucene.analysis.standard.StandardTokenizer
[javac] final public org.apache.lucene.analysis.Token next()
throws ParseException, IOException {
[javac]
^
[javac] /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/StandardTokenizer.java:15:
recursive constructor invocation
[javac] public StandardTokenizer(Reader reader) {
[javac] ^
[javac] /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/StandardTokenizerTokenManager.java:493:
cannot resolve symbol
[javac] symbol : method jCheckNAdd (int)
[javac] location: class
org.apache.lucene.analysis.standard.StandardTokenizerTokenManager
[javac] jCheckNAdd(25);
[javac] ^
[javac] /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/StandardTokenizerTokenManager.java:942:
cannot resolve symbol
[javac] symbol : variable ind
[javac] location: class
org.apache.lucene.analysis.standard.StandardTokenizerTokenManager
[javac] ind = 4;
[javac] ^
[javac] 5 errors
BUILD FAILED
/home/roy/jakarta-lucene/build.xml:140: Compile failed; see the
compiler error output for details.
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: Is the latest StandardTokenizer.jj corrupted?
Posted by Roy <ro...@gmail.com>.
Hi, Erik:
Thank you for fixing it. Otis mentioned in the CVS comments that the
import statement was removed in order to avoid eclipse warning.
However, I think it's better to keep the import statement because
compilation integrity is critical.
I did further research about the weird javacc problem. I found out the
javacc didn't work well with my system language encoding. After I set
my system to standard en_US.UTF-8, it starts working properly.
Thanks.
Roy
On Mon, 2 Aug 2004 19:35:13 -0400, Erik Hatcher
<er...@ehatchersolutions.com> wrote:
> Roy,
>
> I've committed a fix that fixes the import statement issues (Otis -
> what happened? - I guess you didn't try regenerating from the JavaCC
> ..jj files)
>
> All seems to be well now. Let me know if you still have issues.
>
> Erik
>
>
>
> On Aug 2, 2004, at 3:30 PM, Roy wrote:
>
> > The latest StandardTokenizer.jj in cvs repository seems to be
> > corrupted.
> >
> > I used ant javacc-StandardAnalyzer to regenerate the java code, then
> > tried to rebuild the lucene package but got some strange errors as
> > follows. It seems javacc didn't generated correct code against the
> > latest jj file: the last two errors indicate there are some characters
> > missing from those symbols. I checked out the 1.4 jj file and did the
> > same thing again. It works! So I think there's some problems with the
> > latest version. The diff shows the change is minor. I didn't figure
> > the cause. Maybe somebody in the core team can look into this?
> >
> > My purpose is to modify the StandardTokenizer.jj to build my own. But
> > I didn't succeed. Then I found out even the vanilla jj has some
> > problems.
> >
> > compile-core:
> > [javac] Compiling 160 source files to
> > /home/roy/jakarta-lucene/build/classes/java
> > [javac]
> > /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
> > StandardTokenizer.java:15:
> > cannot resolve symbol
> > [javac] symbol : class Reader
> > [javac] location: class
> > org.apache.lucene.analysis.standard.StandardTokenizer
> > [javac] public StandardTokenizer(Reader reader) {
> > [javac] ^
> > [javac]
> > /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
> > StandardTokenizer.java:24:
> > cannot resolve symbol
> > [javac] symbol : class IOException
> > [javac] location: class
> > org.apache.lucene.analysis.standard.StandardTokenizer
> > [javac] final public org.apache.lucene.analysis.Token next()
> > throws ParseException, IOException {
> > [javac]
> > ^
> > [javac]
> > /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
> > StandardTokenizer.java:15:
> > recursive constructor invocation
> > [javac] public StandardTokenizer(Reader reader) {
> > [javac] ^
> > [javac]
> > /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
> > StandardTokenizerTokenManager.java:493:
> > cannot resolve symbol
> > [javac] symbol : method jCheckNAdd (int)
> > [javac] location: class
> > org.apache.lucene.analysis.standard.StandardTokenizerTokenManager
> > [javac] jCheckNAdd(25);
> > [javac] ^
> > [javac]
> > /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
> > StandardTokenizerTokenManager.java:942:
> > cannot resolve symbol
> > [javac] symbol : variable ind
> > [javac] location: class
> > org.apache.lucene.analysis.standard.StandardTokenizerTokenManager
> > [javac] ind = 4;
> > [javac] ^
> > [javac] 5 errors
> >
> > BUILD FAILED
> > /home/roy/jakarta-lucene/build.xml:140: Compile failed; see the
> > compiler error output for details.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: Is the latest StandardTokenizer.jj corrupted?
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hm, not sure. As far as I remember, I did run 'ant javacc', because I
also committed the generated .java files.
Otis
--- Erik Hatcher <er...@ehatchersolutions.com> wrote:
> Roy,
>
> I've committed a fix that fixes the import statement issues (Otis -
> what happened? - I guess you didn't try regenerating from the JavaCC
>
> .jj files)
>
> All seems to be well now. Let me know if you still have issues.
>
> Erik
>
> On Aug 2, 2004, at 3:30 PM, Roy wrote:
>
> > The latest StandardTokenizer.jj in cvs repository seems to be
> > corrupted.
> >
> > I used ant javacc-StandardAnalyzer to regenerate the java code,
> then
> > tried to rebuild the lucene package but got some strange errors as
> > follows. It seems javacc didn't generated correct code against the
> > latest jj file: the last two errors indicate there are some
> characters
> > missing from those symbols. I checked out the 1.4 jj file and did
> the
> > same thing again. It works! So I think there's some problems with
> the
> > latest version. The diff shows the change is minor. I didn't figure
> > the cause. Maybe somebody in the core team can look into this?
> >
> > My purpose is to modify the StandardTokenizer.jj to build my own.
> But
> > I didn't succeed. Then I found out even the vanilla jj has some
> > problems.
> >
> > compile-core:
> > [javac] Compiling 160 source files to
> > /home/roy/jakarta-lucene/build/classes/java
> > [javac]
> >
>
/home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
>
> > StandardTokenizer.java:15:
> > cannot resolve symbol
> > [javac] symbol : class Reader
> > [javac] location: class
> > org.apache.lucene.analysis.standard.StandardTokenizer
> > [javac] public StandardTokenizer(Reader reader) {
> > [javac] ^
> > [javac]
> >
>
/home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
>
> > StandardTokenizer.java:24:
> > cannot resolve symbol
> > [javac] symbol : class IOException
> > [javac] location: class
> > org.apache.lucene.analysis.standard.StandardTokenizer
> > [javac] final public org.apache.lucene.analysis.Token next()
> > throws ParseException, IOException {
> > [javac]
> > ^
> > [javac]
> >
>
/home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
>
> > StandardTokenizer.java:15:
> > recursive constructor invocation
> > [javac] public StandardTokenizer(Reader reader) {
> > [javac] ^
> > [javac]
> >
>
/home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
>
> > StandardTokenizerTokenManager.java:493:
> > cannot resolve symbol
> > [javac] symbol : method jCheckNAdd (int)
> > [javac] location: class
> > org.apache.lucene.analysis.standard.StandardTokenizerTokenManager
> > [javac] jCheckNAdd(25);
> > [javac] ^
> > [javac]
> >
>
/home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
>
> > StandardTokenizerTokenManager.java:942:
> > cannot resolve symbol
> > [javac] symbol : variable ind
> > [javac] location: class
> > org.apache.lucene.analysis.standard.StandardTokenizerTokenManager
> > [javac] ind = 4;
> > [javac] ^
> > [javac] 5 errors
> >
> > BUILD FAILED
> > /home/roy/jakarta-lucene/build.xml:140: Compile failed; see the
> > compiler error output for details.
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
Re: Is the latest StandardTokenizer.jj corrupted?
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
Roy,
I've committed a fix that fixes the import statement issues (Otis -
what happened? - I guess you didn't try regenerating from the JavaCC
.jj files)
All seems to be well now. Let me know if you still have issues.
Erik
On Aug 2, 2004, at 3:30 PM, Roy wrote:
> The latest StandardTokenizer.jj in cvs repository seems to be
> corrupted.
>
> I used ant javacc-StandardAnalyzer to regenerate the java code, then
> tried to rebuild the lucene package but got some strange errors as
> follows. It seems javacc didn't generated correct code against the
> latest jj file: the last two errors indicate there are some characters
> missing from those symbols. I checked out the 1.4 jj file and did the
> same thing again. It works! So I think there's some problems with the
> latest version. The diff shows the change is minor. I didn't figure
> the cause. Maybe somebody in the core team can look into this?
>
> My purpose is to modify the StandardTokenizer.jj to build my own. But
> I didn't succeed. Then I found out even the vanilla jj has some
> problems.
>
> compile-core:
> [javac] Compiling 160 source files to
> /home/roy/jakarta-lucene/build/classes/java
> [javac]
> /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
> StandardTokenizer.java:15:
> cannot resolve symbol
> [javac] symbol : class Reader
> [javac] location: class
> org.apache.lucene.analysis.standard.StandardTokenizer
> [javac] public StandardTokenizer(Reader reader) {
> [javac] ^
> [javac]
> /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
> StandardTokenizer.java:24:
> cannot resolve symbol
> [javac] symbol : class IOException
> [javac] location: class
> org.apache.lucene.analysis.standard.StandardTokenizer
> [javac] final public org.apache.lucene.analysis.Token next()
> throws ParseException, IOException {
> [javac]
> ^
> [javac]
> /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
> StandardTokenizer.java:15:
> recursive constructor invocation
> [javac] public StandardTokenizer(Reader reader) {
> [javac] ^
> [javac]
> /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
> StandardTokenizerTokenManager.java:493:
> cannot resolve symbol
> [javac] symbol : method jCheckNAdd (int)
> [javac] location: class
> org.apache.lucene.analysis.standard.StandardTokenizerTokenManager
> [javac] jCheckNAdd(25);
> [javac] ^
> [javac]
> /home/roy/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
> StandardTokenizerTokenManager.java:942:
> cannot resolve symbol
> [javac] symbol : variable ind
> [javac] location: class
> org.apache.lucene.analysis.standard.StandardTokenizerTokenManager
> [javac] ind = 4;
> [javac] ^
> [javac] 5 errors
>
> BUILD FAILED
> /home/roy/jakarta-lucene/build.xml:140: Compile failed; see the
> compiler error output for details.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org