You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Yonghui Zhao <zh...@gmail.com> on 2013/08/13 13:41:54 UTC
IllegalStateException in SpanTermQuery
One of my UT is passed In lucene 3.5, but it is failed in lucene4.3.
The exception is:
IllegalStateException("field \"" + term.field() + "\" was indexed without
position data; cannot run SpanTermQuery (term=" + term.text() + ")");
After I change index option of the field from DOCS_ONLY to
DOCS_AND_FREQS_AND_POSITIONS, the test is passed.
I am wondering what's the reason of this limit in lucene 4.3.
Thanks
Re: IllegalStateException in SpanTermQuery
Posted by Michael McCandless <lu...@mikemccandless.com>.
OK I see why your test in 3.5 was passing: if you just run
SpanTermQuery alone, the hit count will be correct, because it never
needs to access positions (I suspect?).
Ie, a SpanTermQuery alone is like running TermQuery.
It's when SpanTermQuery is used inside other SpanQuerys that positions
will be needed, and your 3.5 should fail at that point.
Mike McCandless
http://blog.mikemccandless.com
On Wed, Aug 14, 2013 at 12:22 AM, Yonghui Zhao <zh...@gmail.com> wrote:
> In our old code, we create the filed like this.
>
> Field metaField = new Field(name,strVal,fldDef.store,
> Index.NOT_ANALYZED_NO_NORMS);
> metaField.setOmitNorms(true);
> *metaField.setIndexOptions(IndexOptions.DOCS_ONLY);*
> luceneDoc.add(metaField);
>
>
> The test case is quite simple, it only checks numhits.
>
> public void testSpanTermQuery() throws Exception {
> String req = "{\"query\":{\"span_term\":{\"color\":\"red\"}}}";
> JSONObject res = search(new JSONObject(req));
> assertEquals("numhits is wrong", 2160, res.getInt("numhits"));
> }
>
> The query is interpreted to "return new SpanTermQuery(new Term(field,
> spanterm));"
>
> In lucene 3.5 no exception is found and test is passed.
>
>
>
>
>
>
> 2013/8/13 Michael McCandless <lu...@mikemccandless.com>
>
>> All span queries require positions to work; older Lucene released
>> failed to catch you if you tried to use a span query on a field that
>> did not index positions, but now Lucene 4.x does catch you (this is an
>> improvement).
>>
>> You should double check your unit test: it really should not have been
>> passing in Lucene 3.5 if it did not index positions ...
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Tue, Aug 13, 2013 at 7:41 AM, Yonghui Zhao <zh...@gmail.com>
>> wrote:
>> > One of my UT is passed In lucene 3.5, but it is failed in lucene4.3.
>> > The exception is:
>> >
>> > IllegalStateException("field \"" + term.field() + "\" was indexed without
>> > position data; cannot run SpanTermQuery (term=" + term.text() + ")");
>> >
>> >
>> > After I change index option of the field from DOCS_ONLY to
>> > DOCS_AND_FREQS_AND_POSITIONS, the test is passed.
>> >
>> > I am wondering what's the reason of this limit in lucene 4.3.
>> >
>> > Thanks
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: IllegalStateException in SpanTermQuery
Posted by Erick Erickson <er...@gmail.com>.
As Mike said, this is an intended change. The test
passed in 3.5 because there was no check if
Span queries were working on a field that supported
them. In 4.x this is checked and an error is thrown.
Best
Erick
On Wed, Aug 14, 2013 at 12:22 AM, Yonghui Zhao <zh...@gmail.com>wrote:
> In our old code, we create the filed like this.
>
> Field metaField = new Field(name,strVal,fldDef.store,
> Index.NOT_ANALYZED_NO_NORMS);
> metaField.setOmitNorms(true);
> *metaField.setIndexOptions(IndexOptions.DOCS_ONLY);*
> luceneDoc.add(metaField);
>
>
> The test case is quite simple, it only checks numhits.
>
> public void testSpanTermQuery() throws Exception {
> String req = "{\"query\":{\"span_term\":{\"color\":\"red\"}}}";
> JSONObject res = search(new JSONObject(req));
> assertEquals("numhits is wrong", 2160, res.getInt("numhits"));
> }
>
> The query is interpreted to "return new SpanTermQuery(new Term(field,
> spanterm));"
>
> In lucene 3.5 no exception is found and test is passed.
>
>
>
>
>
>
> 2013/8/13 Michael McCandless <lu...@mikemccandless.com>
>
> > All span queries require positions to work; older Lucene released
> > failed to catch you if you tried to use a span query on a field that
> > did not index positions, but now Lucene 4.x does catch you (this is an
> > improvement).
> >
> > You should double check your unit test: it really should not have been
> > passing in Lucene 3.5 if it did not index positions ...
> >
> > Mike McCandless
> >
> > http://blog.mikemccandless.com
> >
> >
> > On Tue, Aug 13, 2013 at 7:41 AM, Yonghui Zhao <zh...@gmail.com>
> > wrote:
> > > One of my UT is passed In lucene 3.5, but it is failed in lucene4.3.
> > > The exception is:
> > >
> > > IllegalStateException("field \"" + term.field() + "\" was indexed
> without
> > > position data; cannot run SpanTermQuery (term=" + term.text() + ")");
> > >
> > >
> > > After I change index option of the field from DOCS_ONLY to
> > > DOCS_AND_FREQS_AND_POSITIONS, the test is passed.
> > >
> > > I am wondering what's the reason of this limit in lucene 4.3.
> > >
> > > Thanks
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
Re: IllegalStateException in SpanTermQuery
Posted by Yonghui Zhao <zh...@gmail.com>.
In our old code, we create the filed like this.
Field metaField = new Field(name,strVal,fldDef.store,
Index.NOT_ANALYZED_NO_NORMS);
metaField.setOmitNorms(true);
*metaField.setIndexOptions(IndexOptions.DOCS_ONLY);*
luceneDoc.add(metaField);
The test case is quite simple, it only checks numhits.
public void testSpanTermQuery() throws Exception {
String req = "{\"query\":{\"span_term\":{\"color\":\"red\"}}}";
JSONObject res = search(new JSONObject(req));
assertEquals("numhits is wrong", 2160, res.getInt("numhits"));
}
The query is interpreted to "return new SpanTermQuery(new Term(field,
spanterm));"
In lucene 3.5 no exception is found and test is passed.
2013/8/13 Michael McCandless <lu...@mikemccandless.com>
> All span queries require positions to work; older Lucene released
> failed to catch you if you tried to use a span query on a field that
> did not index positions, but now Lucene 4.x does catch you (this is an
> improvement).
>
> You should double check your unit test: it really should not have been
> passing in Lucene 3.5 if it did not index positions ...
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Tue, Aug 13, 2013 at 7:41 AM, Yonghui Zhao <zh...@gmail.com>
> wrote:
> > One of my UT is passed In lucene 3.5, but it is failed in lucene4.3.
> > The exception is:
> >
> > IllegalStateException("field \"" + term.field() + "\" was indexed without
> > position data; cannot run SpanTermQuery (term=" + term.text() + ")");
> >
> >
> > After I change index option of the field from DOCS_ONLY to
> > DOCS_AND_FREQS_AND_POSITIONS, the test is passed.
> >
> > I am wondering what's the reason of this limit in lucene 4.3.
> >
> > Thanks
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: IllegalStateException in SpanTermQuery
Posted by Michael McCandless <lu...@mikemccandless.com>.
All span queries require positions to work; older Lucene released
failed to catch you if you tried to use a span query on a field that
did not index positions, but now Lucene 4.x does catch you (this is an
improvement).
You should double check your unit test: it really should not have been
passing in Lucene 3.5 if it did not index positions ...
Mike McCandless
http://blog.mikemccandless.com
On Tue, Aug 13, 2013 at 7:41 AM, Yonghui Zhao <zh...@gmail.com> wrote:
> One of my UT is passed In lucene 3.5, but it is failed in lucene4.3.
> The exception is:
>
> IllegalStateException("field \"" + term.field() + "\" was indexed without
> position data; cannot run SpanTermQuery (term=" + term.text() + ")");
>
>
> After I change index option of the field from DOCS_ONLY to
> DOCS_AND_FREQS_AND_POSITIONS, the test is passed.
>
> I am wondering what's the reason of this limit in lucene 4.3.
>
> Thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org