You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by "Igal @ getRailo.org" <ig...@getrailo.org> on 2013/01/10 01:53:36 UTC
NPE when adding a Document to an IndexWriter
I keep getting an NPE when trying to add a Doc to an IndexWriter. I've
minimized my code to very basic code. what am I doing wrong? pseudo-code:
Document doc = new Document();
TextField ft;
ft = new TextField( "desc1", "word1", Field.Store.YES );
doc.add( ft );
ft = new TextField( "desc2", "word2", Field.Store.YES );
doc.add( ft ); // if I comment out this line then no NPE
IndexWriter iw = new IndexWriter( luceneDirectory, config );
iw.addDocument( doc ); // <== throws NPE
Exception in thread "main" java.lang.NullPointerException
at
org.apache.lucene.analysis.util.CharacterUtils$Java5CharacterUtils.fill(CharacterUtils.java:191)
at
org.apache.lucene.analysis.util.CharTokenizer.incrementToken(CharTokenizer.java:153)
at
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:102)
at
org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:307)
at
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:244)
at
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:373)
at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1445)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1124)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1105)
at s21waf.text.lucene4.Test1.testIndexWriter(Test1.java:71)
at s21waf.text.lucene4.Test1.main(Test1.java:141)
NPE with StandardTokenizer:
Exception in thread "main" java.lang.NullPointerException
at
org.apache.lucene.analysis.standard.StandardTokenizerImpl.zzRefill(StandardTokenizerImpl.java:921)
at
org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:1128)
at
org.apache.lucene.analysis.standard.StandardTokenizer.incrementToken(StandardTokenizer.java:179)
at
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:102)
at
org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:307)
at
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:244)
at
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:373)
at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1445)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1124)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1105)
at s21waf.text.lucene4.Test1.testIndexWriter(Test1.java:75)
at s21waf.text.lucene4.Test1.main(Test1.java:145)
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: NPE when adding a Document to an IndexWriter
Posted by "Igal @ getRailo.org" <ig...@getrailo.org>.
I think that all I needed to create the components is:
@Override
protected Analyzer.TokenStreamComponents createComponents( String
fieldName, Reader reader ) {
Analyzer.TokenStreamComponents tsc = new
Analyzer.TokenStreamComponents(
getTokenFilterChain( reader, config )
);
return tsc;
}
I'll need to test it to know for sure though.
thanks,
Igal
On 1/9/2013 6:54 PM, Igal @ getRailo.org wrote:
> hi Hoss -- thank you for your time. it looks like you're right (and
> it makes sense if the reader is advanced in two places at the same
> time that it will cause a problem).
>
> I'll try to figure out how to create an Analyzer out of the
> Tokenizer. that's what I was trying to do there and obviously I did
> it wrong.
>
> thanks again,
>
>
> Igal
>
>
> On 1/9/2013 6:28 PM, Chris Hostetter wrote:
>> : thanks for your reply. please see attached. I tried to maintain the
>> : structure of the code that I need to use in the library I'm
>> building. I think
>> : it should work for you as long as you remove the package
>> declaration at the
>> : top.
>>
>> I can't currently try your code, but skimming through it i'd bet
>> money the
>> problem is in your Analyzer. Have you tried simplifying your test down
>> and just using "StandardAnalyzer" to rule that out?
>>
>> In particular i see this...
>>
>>>>> Analyzer.TokenStreamComponents tsc = new
>>>>> Analyzer.TokenStreamComponents(
>>>>> getCharTokenizer( reader )
>>>>> , getTokenFilterChain( reader, config )
>>>>> );
>> ...passing the same Reader to to diff methods there is almost certainly
>> not what you want to do.
>>
>>
>>
>> -Hoss
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: NPE when adding a Document to an IndexWriter
Posted by "Igal @ getRailo.org" <ig...@getrailo.org>.
hi Hoss -- thank you for your time. it looks like you're right (and it
makes sense if the reader is advanced in two places at the same time
that it will cause a problem).
I'll try to figure out how to create an Analyzer out of the Tokenizer.
that's what I was trying to do there and obviously I did it wrong.
thanks again,
Igal
On 1/9/2013 6:28 PM, Chris Hostetter wrote:
> : thanks for your reply. please see attached. I tried to maintain the
> : structure of the code that I need to use in the library I'm building. I think
> : it should work for you as long as you remove the package declaration at the
> : top.
>
> I can't currently try your code, but skimming through it i'd bet money the
> problem is in your Analyzer. Have you tried simplifying your test down
> and just using "StandardAnalyzer" to rule that out?
>
> In particular i see this...
>
>>>> Analyzer.TokenStreamComponents tsc = new Analyzer.TokenStreamComponents(
>>>> getCharTokenizer( reader )
>>>> , getTokenFilterChain( reader, config )
>>>> );
> ...passing the same Reader to to diff methods there is almost certainly
> not what you want to do.
>
>
>
> -Hoss
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: NPE when adding a Document to an IndexWriter
Posted by Chris Hostetter <ho...@fucit.org>.
: thanks for your reply. please see attached. I tried to maintain the
: structure of the code that I need to use in the library I'm building. I think
: it should work for you as long as you remove the package declaration at the
: top.
I can't currently try your code, but skimming through it i'd bet money the
problem is in your Analyzer. Have you tried simplifying your test down
and just using "StandardAnalyzer" to rule that out?
In particular i see this...
>>> Analyzer.TokenStreamComponents tsc = new Analyzer.TokenStreamComponents(
>>> getCharTokenizer( reader )
>>> , getTokenFilterChain( reader, config )
>>> );
...passing the same Reader to to diff methods there is almost certainly
not what you want to do.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: NPE when adding a Document to an IndexWriter
Posted by "Igal @ getRailo.org" <ig...@getrailo.org>.
thanks for your reply. please see attached. I tried to maintain the
structure of the code that I need to use in the library I'm building. I
think it should work for you as long as you remove the package
declaration at the top.
when I run the attached file I get the following output:
debug:
Exception in thread "main" java.lang.NullPointerException
at
org.apache.lucene.analysis.util.CharacterUtils$Java5CharacterUtils.fill(CharacterUtils.java:191)
at
org.apache.lucene.analysis.util.CharTokenizer.incrementToken(CharTokenizer.java:153)
at
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:102)
at
org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:307)
at
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:244)
at
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:373)
at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1445)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1124)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1105)
at s21waf.text.lucene4.TestNPE.testIndexWriter(TestNPE.java:47)
at s21waf.text.lucene4.TestNPE.main(TestNPE.java:111)
Java Result: 1
BUILD SUCCESSFUL (total time: 0 seconds)
thanks,
Igal
On 1/9/2013 5:23 PM, Chris Hostetter wrote:
> : I keep getting an NPE when trying to add a Doc to an IndexWriter. I've
> : minimized my code to very basic code. what am I doing wrong? pseudo-code:
>
> can you post a full test that other people can run to try and reproduce?
>
> it doesn't even have to be a junit test -- just some complete javacode
> people paste into a main method and compile would be enough (right now we
> have no idea what IndexWriterConfig you are using (could easily affect
> things) or what directory you are using (less likeley, but still)
>
>
> -Hoss
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
Re: NPE when adding a Document to an IndexWriter
Posted by Chris Hostetter <ho...@fucit.org>.
: I keep getting an NPE when trying to add a Doc to an IndexWriter. I've
: minimized my code to very basic code. what am I doing wrong? pseudo-code:
can you post a full test that other people can run to try and reproduce?
it doesn't even have to be a junit test -- just some complete javacode
people paste into a main method and compile would be enough (right now we
have no idea what IndexWriterConfig you are using (could easily affect
things) or what directory you are using (less likeley, but still)
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org