You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Bill Tschumy <bi...@otherwise.com> on 2005/04/11 21:27:57 UTC
Strange sort error
In my application, by default I display all documents that are in the
index. I sort them either using a "time modified" or "time created".
If I have a newly created empty index, I find I get an error if I sort
by "time modified" but not "time created". In either case there are
actually no documents that match my query so in reality there is
nothing to sort.
Here is my query:
query = new TermQuery(new Term(MyIndexer.CREATOR_KEY,
MyIndexer.PARSNIPS_VAL));
String fieldName = sortType == Parsnips.SORT_BY_MODIFIED ?
MyIndexer.MODIFIED_KEY : MyIndexer.CREATED_KEY;
Sort sorter = new Sort(new SortField(fieldName, SortField.STRING,
true));
hits = searcher.search(query, sorter);
The error I'm getting when using MyIndexer.MODIFIED_KEY as the sort
field is:
java.lang.RuntimeException: no terms in field modified
at
org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.ja
va:256)
at
org.apache.lucene.search.FieldSortedHitQueue.comparatorString(FieldSorte
dHitQueue.java:265)
at
org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSo
rtedHitQueue.java:180)
at
org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueue.
java:58)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:122)
at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:64)
at org.apache.lucene.search.Hits.<init>(Hits.java:51)
at org.apache.lucene.search.Searcher.search(Searcher.java:41)
at com.otherwise.parsnips.MySearcher.search(MySearcher.java:170)
at com.otherwise.parsnips.MySearcher.search(MySearcher.java:149)
at com.otherwise.parsnips.Parsnips.<init>(Parsnips.java:163)
at com.otherwise.parsnips.Parsnips.main(Parsnips.java:1205)
I can't understand why I would be getting this for one sort field but
not the other given there are 0 hits anyway in a newly created index.
Anyone have any thoughts? I am using Lucene 1.4.2.
--
Bill Tschumy
Otherwise -- Austin, TX
http://www.otherwise.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Daniel Naber <lu...@danielnaber.de>.
On Friday 15 April 2005 01:46, Chris Hostetter wrote:
> Or worse, a query that does work today, stops working tomorow because
> one doc was removed.
You're right, that is not acceptable. I've created a bug report about the
original problem:
http://issues.apache.org/bugzilla/show_bug.cgi?id=34477
Further discussion should probably take place on the development list.
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Chris Hostetter <ho...@fucit.org>.
: one is sorting on doesn't even have to exist in all the documents. I
: think it would be even more confusing for an invalid query suddenly
: becoming a valid query in the future just because someone added a doc
Or worse, a query that does work today, stops working tomorow because one
doc was removed.
: with that field indexed (esp since that doc may not even match the
: query being sorted).
In my mind, thta's the real crux of the issue...
Imagine you have two general classes of Documents: "animals" and "plants"
-- all documents have some fields in common, including a "doctype" field
(which is allways either "plant" or "animal") but there may be some fields
which are exlusive to each type (ie: "number_of_legs", "avg_root_depth").
doing a search on something like "description:africa" sorted by score,
might return you all the plants and animals in africa. sorting by
"number_of_legs" would get you the same results, but all the animals would
come first (since the plants don't have any legs). a search for
"doctype:plant description:africa" would only return you plants, but even
then it would still work if you sorted by "number_of_legs".
so why then should it stop working if all of hte animals gradually become
extinct and get removed from the index.
: In short, I think sorting should act like querying (no exception if
: field doesn't exist yet).
agreed.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Yonik Seeley <ys...@gmail.com>.
Also, it's more flexible. You can easily implement stricter checking
on top of a "lax" model (use a term enumerator to see if the field
exists before you call search), but not vice versa.
-Yonik
On 4/14/05, Yonik Seeley <ys...@gmail.com> wrote:
> Hmmm, that's a great lucene architecture question.
> Should one be allowed to sort on a field that doesn't exist?
> One *can* query on fields that don't exist (and that's correct in my view).
>
> The thing is, lucene field creation is lazy... just because the field
> doesn't exist now doesn't mean that it won't exist later. The field
> one is sorting on doesn't even have to exist in all the documents. I
> think it would be even more confusing for an invalid query suddenly
> becoming a valid query in the future just because someone added a doc
> with that field indexed (esp since that doc may not even match the
> query being sorted).
>
> In short, I think sorting should act like querying (no exception if
> field doesn't exist yet).
>
> -Yonik
>
> On 4/14/05, Daniel Naber <lu...@danielnaber.de> wrote:
> > On Thursday 14 April 2005 16:28, Yonik Seeley wrote:
> >
> > > I haven't tried it, but I think the fix should be easy... never throw
> > > that exception.
> >
> > As Lucene does not have the concept of a "warning" I think it should throw
> > exceptions when someone tries to do something that doesn't make sense
> > (even if it's technically possible). And sorting on a field that doesn't
> > exist doesn't seem to make sense.
> >
> > Well, searching on a field that doesn't exist won't give you an exception
> > either. For debugging it would be useful if you'd get an exception instead
> > of no results.
> >
> > Regards
> > Daniel
> >
> > --
> > http://www.danielnaber.de
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Yonik Seeley <ys...@gmail.com>.
Hmmm, that's a great lucene architecture question.
Should one be allowed to sort on a field that doesn't exist?
One *can* query on fields that don't exist (and that's correct in my view).
The thing is, lucene field creation is lazy... just because the field
doesn't exist now doesn't mean that it won't exist later. The field
one is sorting on doesn't even have to exist in all the documents. I
think it would be even more confusing for an invalid query suddenly
becoming a valid query in the future just because someone added a doc
with that field indexed (esp since that doc may not even match the
query being sorted).
In short, I think sorting should act like querying (no exception if
field doesn't exist yet).
-Yonik
On 4/14/05, Daniel Naber <lu...@danielnaber.de> wrote:
> On Thursday 14 April 2005 16:28, Yonik Seeley wrote:
>
> > I haven't tried it, but I think the fix should be easy... never throw
> > that exception.
>
> As Lucene does not have the concept of a "warning" I think it should throw
> exceptions when someone tries to do something that doesn't make sense
> (even if it's technically possible). And sorting on a field that doesn't
> exist doesn't seem to make sense.
>
> Well, searching on a field that doesn't exist won't give you an exception
> either. For debugging it would be useful if you'd get an exception instead
> of no results.
>
> Regards
> Daniel
>
> --
> http://www.danielnaber.de
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Daniel Naber <lu...@danielnaber.de>.
On Thursday 14 April 2005 16:28, Yonik Seeley wrote:
> I haven't tried it, but I think the fix should be easy... never throw
> that exception.
As Lucene does not have the concept of a "warning" I think it should throw
exceptions when someone tries to do something that doesn't make sense
(even if it's technically possible). And sorting on a field that doesn't
exist doesn't seem to make sense.
Well, searching on a field that doesn't exist won't give you an exception
either. For debugging it would be useful if you'd get an exception instead
of no results.
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Yonik Seeley <ys...@gmail.com>.
> if (termEnum==null || term.field() != field) break; // CHANGE here
Errr, that should be term==null of course.
> if (term==null || term.field() != field) break; // CHANGE here
And it *may* be slightly speedier to check for null just before the
do/while loop instead:
if (termEnum.term() != null) {
do {
[...]
} while(...)
}
-Yonik
On 4/14/05, Yonik Seeley <ys...@gmail.com> wrote:
> I haven't tried it, but I think the fix should be easy... never throw
> that exception. Either check for null before the loop, or in the
> loop.
>
> Original code for native int sorting:
>
> TermEnum termEnum = reader.terms (new Term (field, ""));
> try {
> if (termEnum.term() == null) {
> throw new RuntimeException ("no terms in field " + field);
> }
> do {
> Term term = termEnum.term();
> if (term.field() != field) break;
> int termval = Integer.parseInt (term.text());
> termDocs.seek (termEnum);
> while (termDocs.next()) {
> retArray[termDocs.doc()] = termval;
> }
> } while (termEnum.next());
> } finally {
> termDocs.close();
> termEnum.close();
> }
>
> ------------- possible fix --------------
> TermEnum termEnum = reader.terms (new Term (field, ""));
> try {
> do {
> Term term = termEnum.term();
> if (termEnum==null || term.field() != field) break; // CHANGE here
> int termval = Integer.parseInt (term.text());
> termDocs.seek (termEnum);
> while (termDocs.next()) {
> retArray[termDocs.doc()] = termval;
> }
> } while (termEnum.next());
> } finally {
> termDocs.close();
> termEnum.close();
> }
>
> -Yonik
>
> On 4/13/05, Daniel Naber <lu...@danielnaber.de> wrote:
> > On Tuesday 12 April 2005 20:04, Bill Tschumy wrote:
> >
> > > Here is a small program that will manifest the error. Hopefully
> > > someone can explain the problem. It happens with Lucene 1.4.2 and
> > > 1.4.3.
> >
> > This is the code that throws the exception (from FieldCacheImpl.java):
> >
> > TermEnum termEnum = reader.terms (new Term (field, ""));
> > (...)
> > if (termEnum.term() == null) {
> > throw new RuntimeException ("no terms in field " + field);
> > }
> >
> > The problem is that a TermEnum always returns all terms after a given one,
> > not only terms in the same field. So the check is incomplete. If one
> > changes the if like this, one will always get an exception if there are no
> > terms in the field, as the exception claims:
> >
> > if (termEnum.term() == null || !termEnum.term().field().equals(field)) {
> >
> > The other issue is that you probably expect to not get an exception at all,
> > as there are no matches. Lucene doesn't first search and then sort, these
> > tasks are parallel I think. So this is not that easy to fix (and I doubt
> > if one should try).
> >
> > Could you open a bug report for the problem with the exception that seems
> > to occur only sometimes? The change suggested above needs to be tested
> > before it can be committed and a bug report is useful for that.
> >
> > Regards
> > Daniel
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Yonik Seeley <ys...@gmail.com>.
I haven't tried it, but I think the fix should be easy... never throw
that exception. Either check for null before the loop, or in the
loop.
Original code for native int sorting:
TermEnum termEnum = reader.terms (new Term (field, ""));
try {
if (termEnum.term() == null) {
throw new RuntimeException ("no terms in field " + field);
}
do {
Term term = termEnum.term();
if (term.field() != field) break;
int termval = Integer.parseInt (term.text());
termDocs.seek (termEnum);
while (termDocs.next()) {
retArray[termDocs.doc()] = termval;
}
} while (termEnum.next());
} finally {
termDocs.close();
termEnum.close();
}
------------- possible fix --------------
TermEnum termEnum = reader.terms (new Term (field, ""));
try {
do {
Term term = termEnum.term();
if (termEnum==null || term.field() != field) break; // CHANGE here
int termval = Integer.parseInt (term.text());
termDocs.seek (termEnum);
while (termDocs.next()) {
retArray[termDocs.doc()] = termval;
}
} while (termEnum.next());
} finally {
termDocs.close();
termEnum.close();
}
-Yonik
On 4/13/05, Daniel Naber <lu...@danielnaber.de> wrote:
> On Tuesday 12 April 2005 20:04, Bill Tschumy wrote:
>
> > Here is a small program that will manifest the error. Hopefully
> > someone can explain the problem. It happens with Lucene 1.4.2 and
> > 1.4.3.
>
> This is the code that throws the exception (from FieldCacheImpl.java):
>
> TermEnum termEnum = reader.terms (new Term (field, ""));
> (...)
> if (termEnum.term() == null) {
> throw new RuntimeException ("no terms in field " + field);
> }
>
> The problem is that a TermEnum always returns all terms after a given one,
> not only terms in the same field. So the check is incomplete. If one
> changes the if like this, one will always get an exception if there are no
> terms in the field, as the exception claims:
>
> if (termEnum.term() == null || !termEnum.term().field().equals(field)) {
>
> The other issue is that you probably expect to not get an exception at all,
> as there are no matches. Lucene doesn't first search and then sort, these
> tasks are parallel I think. So this is not that easy to fix (and I doubt
> if one should try).
>
> Could you open a bug report for the problem with the exception that seems
> to occur only sometimes? The change suggested above needs to be tested
> before it can be committed and a bug report is useful for that.
>
> Regards
> Daniel
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Daniel Naber <lu...@danielnaber.de>.
On Tuesday 12 April 2005 20:04, Bill Tschumy wrote:
> Here is a small program that will manifest the error. Hopefully
> someone can explain the problem. It happens with Lucene 1.4.2 and
> 1.4.3.
This is the code that throws the exception (from FieldCacheImpl.java):
TermEnum termEnum = reader.terms (new Term (field, ""));
(...)
if (termEnum.term() == null) {
throw new RuntimeException ("no terms in field " + field);
}
The problem is that a TermEnum always returns all terms after a given one,
not only terms in the same field. So the check is incomplete. If one
changes the if like this, one will always get an exception if there are no
terms in the field, as the exception claims:
if (termEnum.term() == null || !termEnum.term().field().equals(field)) {
The other issue is that you probably expect to not get an exception at all,
as there are no matches. Lucene doesn't first search and then sort, these
tasks are parallel I think. So this is not that easy to fix (and I doubt
if one should try).
Could you open a bug report for the problem with the exception that seems
to occur only sometimes? The change suggested above needs to be tested
before it can be committed and a bug report is useful for that.
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Bill Tschumy <bi...@otherwise.com>.
Great! Thanks for looking at it and thanks for the work around.
On Apr 12, 2005, at 1:50 PM, Yonik Seeley wrote:
> A workaround until this problem is fixed in Lucene would be to add an
> indexed sentinel field to a single doc in the collection that will be
> larger (after) all other fields that you may try a sort on.
>
> Example:
> String sentinel = new String(new char[]{0xffff});
> doc.add(Field.Keyword(sentinel, sentinel));
>
> -Yonik
>
> On Apr 12, 2005 2:32 PM, Yonik Seeley <ys...@gmail.com> wrote:
>>> Any fieldName that starts with "i" or
>>> below (including capitals) works. Can anyone think of what could
>>> possibly be going on here?
>>
>> Looks like you uncovered an obscure sorting bug.
>> The reason that fields >= "j" fail is that your last indexed field
>> (and hence the last indexed term) starts with "i" (specifically
>> "indexVersion").
>>
>>> private static String VERSION_CREATOR_KEY = "creatorVer";
>>> private static String INDEX_VERSION_KEY = "indexVersion";
>>
>> If you changed these to "a" and "aa", then all three tests would fail.
>>
>> -Yonik
>>
>>
>> On Apr 12, 2005 2:04 PM, Bill Tschumy <bi...@otherwise.com> wrote:
>>> On Apr 12, 2005, at 8:38 AM, Erik Hatcher wrote:
>>>
>>>> Could you give us a self-contained test case that reproduces this
>>>> issue?
>>>>
>>>> Erik
>>>>
>>>
>>> Here is a small program that will manifest the error. Hopefully
>>> someone can explain the problem. It happens with Lucene 1.4.2 and
>>> 1.4.3.
>>>
>>> file: SortProblem.java
>>> =========================
>>> import java.io.*;
>>> import org.apache.lucene.search.*;
>>> import org.apache.lucene.index.*;
>>> import org.apache.lucene.store.*;
>>> import org.apache.lucene.document.*;
>>> import org.apache.lucene.analysis.standard.StandardAnalyzer;
>>>
>>> /**
>>> * This program demonstrates a problem I'm having with Lucene. If
>>> I
>>> search for
>>> * documents with a particular field name/value pair and sort them
>>> based upon
>>> * another field, I sometimes get a RuntimeException thrown. This
>>> happens when the
>>> * Hits comes back empty and the sort field name begins with a
>>> letter
>>> < "j". For
>>> * the bug to manifest it appears I also need to have an unrelated
>>> document in the
>>> * index that is storing version information.
>>> **/
>>>
>>> public class SortProblem
>>> {
>>>
>>> private static String INDEX_DIRECTORY = "SortProblemIndex";
>>>
>>> // This is the field name of a field in the dcoument that hold
>>> version
>>> // information. The bug happens if this field name has certain
>>> values.
>>> // "creatorVal" will fail, but "xreatorVal" will not. Very
>>> wierd.
>>> private static String VERSION_CREATOR_KEY = "creatorVer";
>>>
>>> private static String INDEX_VERSION_VAL = "indexVersionVal";
>>> private static String INDEX_VERSION_KEY = "indexVersion";
>>> private static String INDEX_VERSION = "1.1";
>>> private static String CREATOR_KEY = "creator";
>>> private static String PARSNIPS_VAL = "Parsnips";
>>>
>>> public static void main(String[] args)
>>> {
>>> initIndex(INDEX_DIRECTORY);
>>> // The search appears to fail if the fieldName starts with a
>>> letter >= "j".
>>> // The first and last search here will work while the middle
>>> will fail.
>>> search(INDEX_DIRECTORY, "aaa");
>>> search(INDEX_DIRECTORY, "mmm");
>>> search(INDEX_DIRECTORY, "bbb");
>>> }
>>>
>>> private static void initIndex(String directoryName)
>>> {
>>> File indexDir = new File(directoryName);
>>> if (indexDir.exists())
>>> deleteFileOrDirectory(indexDir);
>>> indexDir.mkdir();
>>> try
>>> {
>>> IndexWriter writer = new IndexWriter(indexDir, new
>>> StandardAnalyzer(), true);
>>> // Adding the one document which contains version
>>> information seems
>>> // necessary to cause some search/sorts to fail.
>>> Document doc = new Document();
>>> doc.add(Field.Keyword(VERSION_CREATOR_KEY,
>>> INDEX_VERSION_VAL));
>>> doc.add(Field.Keyword(INDEX_VERSION_KEY,
>>> INDEX_VERSION));
>>> writer.addDocument(doc);
>>> writer.close();
>>> }
>>> catch (IOException ioe)
>>> {
>>> ioe.printStackTrace();
>>> }
>>> }
>>>
>>> private static void search(String directoryName, String
>>> sortFieldName)
>>> {
>>> try
>>> {
>>> File indexDir = new File(directoryName);
>>> Directory fsDir = FSDirectory.getDirectory(indexDir,
>>> false);
>>> IndexSearcher searcher = new IndexSearcher(fsDir);
>>> Hits hits;
>>> Query query = new TermQuery(new Term(CREATOR_KEY,
>>> PARSNIPS_VAL));
>>> Sort sorter = new Sort(new SortField(sortFieldName,
>>> SortField.STRING, true));
>>> try
>>> {
>>> hits = searcher.search(query, sorter);
>>> System.out.println("sort on " + sortFieldName + "
>>> successful.");
>>> }
>>> catch (RuntimeException e)
>>> {
>>> System.out.println("sort on " + sortFieldName + "
>>> failed.");
>>> e.printStackTrace();
>>> }
>>> }
>>> catch (IOException e)
>>> {
>>> e.printStackTrace();
>>> }
>>>
>>> }
>>>
>>> private static boolean deleteFileOrDirectory(File dir)
>>> {
>>> if (dir.isDirectory())
>>> {
>>> String[] children = dir.list();
>>> for (int i = 0; i < children.length; i++)
>>> {
>>> boolean success = deleteFileOrDirectory(new
>>> File(dir,
>>> children[i]));
>>> if (!success)
>>> {
>>> return false;
>>> }
>>> }
>>> }
>>> // The directory is now empty so delete it
>>> return dir.delete();
>>> }
>>> }
>>>
>>>> On Apr 12, 2005, at 9:19 AM, Bill Tschumy wrote:
>>>>
>>>>> This problem is seeming more and more strange. It now looks like
>>>>> if
>>>>> the fieldName I'm sorting on starts is ASCII "j" or above, the
>>>>> RuntimeException is thrown. Any fieldName that starts with "i" or
>>>>> below (including capitals) works. Can anyone think of what could
>>>>> possibly be going on here?
>>>>>
>>>>>
>>>>> On Apr 11, 2005, at 2:27 PM, Bill Tschumy wrote:
>>>>>
>>>>>> In my application, by default I display all documents that are in
>>>>>> the index. I sort them either using a "time modified" or "time
>>>>>> created". If I have a newly created empty index, I find I get an
>>>>>> error if I sort by "time modified" but not "time created". In
>>>>>> either case there are actually no documents that match my query so
>>>>>> in reality there is nothing to sort.
>>>>>>
>>>>>> Here is my query:
>>>>>>
>>>>>> query = new TermQuery(new Term(MyIndexer.CREATOR_KEY,
>>>>>> MyIndexer.PARSNIPS_VAL));
>>>>>> String fieldName = sortType == Parsnips.SORT_BY_MODIFIED ?
>>>>>> MyIndexer.MODIFIED_KEY : MyIndexer.CREATED_KEY;
>>>>>> Sort sorter = new Sort(new SortField(fieldName, SortField.STRING,
>>>>>> true));
>>>>>> hits = searcher.search(query, sorter);
>>>>>>
>>>>>> The error I'm getting when using MyIndexer.MODIFIED_KEY (which is
>>>>>> "modified") as the sort field is:
>>>>>>
>>>>>> java.lang.RuntimeException: no terms in field modified
>>>>>> at
>>>>>> org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheI
>>>>>> mpl
>>>>>> .java:256)
>>>>>> at
>>>>>> org.apache.lucene.search.FieldSortedHitQueue.comparatorString(Fiel
>>>>>> dSo
>>>>>> rtedHitQueue.java:265)
>>>>>> at
>>>>>> org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(F
>>>>>> iel
>>>>>> dSortedHitQueue.java:180)
>>>>>> at
>>>>>> org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHit
>>>>>> Que
>>>>>> ue.java:58)
>>>>>> at
>>>>>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:
>>>>>> 122)
>>>>>> at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:64)
>>>>>> at org.apache.lucene.search.Hits.<init>(Hits.java:51)
>>>>>> at
>>>>>> org.apache.lucene.search.Searcher.search(Searcher.java:41)
>>>>>> at
>>>>>> com.otherwise.parsnips.MySearcher.search(MySearcher.java:170)
>>>>>> at
>>>>>> com.otherwise.parsnips.MySearcher.search(MySearcher.java:149)
>>>>>> at
>>>>>> com.otherwise.parsnips.Parsnips.<init>(Parsnips.java:163)
>>>>>> at
>>>>>> com.otherwise.parsnips.Parsnips.main(Parsnips.java:1205)
>>>>>>
>>>>>> I can't understand why I would be getting this for one sort field
>>>>>> but not the other given there are 0 hits anyway in a newly created
>>>>>> index. Anyone have any thoughts? I am using Lucene 1.4.2.
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Bill Tschumy
Otherwise -- Austin, TX
http://www.otherwise.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Yonik Seeley <ys...@gmail.com>.
A workaround until this problem is fixed in Lucene would be to add an
indexed sentinel field to a single doc in the collection that will be
larger (after) all other fields that you may try a sort on.
Example:
String sentinel = new String(new char[]{0xffff});
doc.add(Field.Keyword(sentinel, sentinel));
-Yonik
On Apr 12, 2005 2:32 PM, Yonik Seeley <ys...@gmail.com> wrote:
> > Any fieldName that starts with "i" or
> > below (including capitals) works. Can anyone think of what could
> > possibly be going on here?
>
> Looks like you uncovered an obscure sorting bug.
> The reason that fields >= "j" fail is that your last indexed field
> (and hence the last indexed term) starts with "i" (specifically
> "indexVersion").
>
> > private static String VERSION_CREATOR_KEY = "creatorVer";
> > private static String INDEX_VERSION_KEY = "indexVersion";
>
> If you changed these to "a" and "aa", then all three tests would fail.
>
> -Yonik
>
>
> On Apr 12, 2005 2:04 PM, Bill Tschumy <bi...@otherwise.com> wrote:
> > On Apr 12, 2005, at 8:38 AM, Erik Hatcher wrote:
> >
> > > Could you give us a self-contained test case that reproduces this
> > > issue?
> > >
> > > Erik
> > >
> >
> > Here is a small program that will manifest the error. Hopefully
> > someone can explain the problem. It happens with Lucene 1.4.2 and
> > 1.4.3.
> >
> > file: SortProblem.java
> > =========================
> > import java.io.*;
> > import org.apache.lucene.search.*;
> > import org.apache.lucene.index.*;
> > import org.apache.lucene.store.*;
> > import org.apache.lucene.document.*;
> > import org.apache.lucene.analysis.standard.StandardAnalyzer;
> >
> > /**
> > * This program demonstrates a problem I'm having with Lucene. If I
> > search for
> > * documents with a particular field name/value pair and sort them
> > based upon
> > * another field, I sometimes get a RuntimeException thrown. This
> > happens when the
> > * Hits comes back empty and the sort field name begins with a letter
> > < "j". For
> > * the bug to manifest it appears I also need to have an unrelated
> > document in the
> > * index that is storing version information.
> > **/
> >
> > public class SortProblem
> > {
> >
> > private static String INDEX_DIRECTORY = "SortProblemIndex";
> >
> > // This is the field name of a field in the dcoument that hold
> > version
> > // information. The bug happens if this field name has certain
> > values.
> > // "creatorVal" will fail, but "xreatorVal" will not. Very wierd.
> > private static String VERSION_CREATOR_KEY = "creatorVer";
> >
> > private static String INDEX_VERSION_VAL = "indexVersionVal";
> > private static String INDEX_VERSION_KEY = "indexVersion";
> > private static String INDEX_VERSION = "1.1";
> > private static String CREATOR_KEY = "creator";
> > private static String PARSNIPS_VAL = "Parsnips";
> >
> > public static void main(String[] args)
> > {
> > initIndex(INDEX_DIRECTORY);
> > // The search appears to fail if the fieldName starts with a
> > letter >= "j".
> > // The first and last search here will work while the middle
> > will fail.
> > search(INDEX_DIRECTORY, "aaa");
> > search(INDEX_DIRECTORY, "mmm");
> > search(INDEX_DIRECTORY, "bbb");
> > }
> >
> > private static void initIndex(String directoryName)
> > {
> > File indexDir = new File(directoryName);
> > if (indexDir.exists())
> > deleteFileOrDirectory(indexDir);
> > indexDir.mkdir();
> > try
> > {
> > IndexWriter writer = new IndexWriter(indexDir, new
> > StandardAnalyzer(), true);
> > // Adding the one document which contains version
> > information seems
> > // necessary to cause some search/sorts to fail.
> > Document doc = new Document();
> > doc.add(Field.Keyword(VERSION_CREATOR_KEY,
> > INDEX_VERSION_VAL));
> > doc.add(Field.Keyword(INDEX_VERSION_KEY, INDEX_VERSION));
> > writer.addDocument(doc);
> > writer.close();
> > }
> > catch (IOException ioe)
> > {
> > ioe.printStackTrace();
> > }
> > }
> >
> > private static void search(String directoryName, String
> > sortFieldName)
> > {
> > try
> > {
> > File indexDir = new File(directoryName);
> > Directory fsDir = FSDirectory.getDirectory(indexDir, false);
> > IndexSearcher searcher = new IndexSearcher(fsDir);
> > Hits hits;
> > Query query = new TermQuery(new Term(CREATOR_KEY,
> > PARSNIPS_VAL));
> > Sort sorter = new Sort(new SortField(sortFieldName,
> > SortField.STRING, true));
> > try
> > {
> > hits = searcher.search(query, sorter);
> > System.out.println("sort on " + sortFieldName + "
> > successful.");
> > }
> > catch (RuntimeException e)
> > {
> > System.out.println("sort on " + sortFieldName + "
> > failed.");
> > e.printStackTrace();
> > }
> > }
> > catch (IOException e)
> > {
> > e.printStackTrace();
> > }
> >
> > }
> >
> > private static boolean deleteFileOrDirectory(File dir)
> > {
> > if (dir.isDirectory())
> > {
> > String[] children = dir.list();
> > for (int i = 0; i < children.length; i++)
> > {
> > boolean success = deleteFileOrDirectory(new File(dir,
> > children[i]));
> > if (!success)
> > {
> > return false;
> > }
> > }
> > }
> > // The directory is now empty so delete it
> > return dir.delete();
> > }
> > }
> >
> > > On Apr 12, 2005, at 9:19 AM, Bill Tschumy wrote:
> > >
> > >> This problem is seeming more and more strange. It now looks like if
> > >> the fieldName I'm sorting on starts is ASCII "j" or above, the
> > >> RuntimeException is thrown. Any fieldName that starts with "i" or
> > >> below (including capitals) works. Can anyone think of what could
> > >> possibly be going on here?
> > >>
> > >>
> > >> On Apr 11, 2005, at 2:27 PM, Bill Tschumy wrote:
> > >>
> > >>> In my application, by default I display all documents that are in
> > >>> the index. I sort them either using a "time modified" or "time
> > >>> created". If I have a newly created empty index, I find I get an
> > >>> error if I sort by "time modified" but not "time created". In
> > >>> either case there are actually no documents that match my query so
> > >>> in reality there is nothing to sort.
> > >>>
> > >>> Here is my query:
> > >>>
> > >>> query = new TermQuery(new Term(MyIndexer.CREATOR_KEY,
> > >>> MyIndexer.PARSNIPS_VAL));
> > >>> String fieldName = sortType == Parsnips.SORT_BY_MODIFIED ?
> > >>> MyIndexer.MODIFIED_KEY : MyIndexer.CREATED_KEY;
> > >>> Sort sorter = new Sort(new SortField(fieldName, SortField.STRING,
> > >>> true));
> > >>> hits = searcher.search(query, sorter);
> > >>>
> > >>> The error I'm getting when using MyIndexer.MODIFIED_KEY (which is
> > >>> "modified") as the sort field is:
> > >>>
> > >>> java.lang.RuntimeException: no terms in field modified
> > >>> at
> > >>> org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl
> > >>> .java:256)
> > >>> at
> > >>> org.apache.lucene.search.FieldSortedHitQueue.comparatorString(FieldSo
> > >>> rtedHitQueue.java:265)
> > >>> at
> > >>> org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(Fiel
> > >>> dSortedHitQueue.java:180)
> > >>> at
> > >>> org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQue
> > >>> ue.java:58)
> > >>> at
> > >>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:
> > >>> 122)
> > >>> at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:64)
> > >>> at org.apache.lucene.search.Hits.<init>(Hits.java:51)
> > >>> at org.apache.lucene.search.Searcher.search(Searcher.java:41)
> > >>> at
> > >>> com.otherwise.parsnips.MySearcher.search(MySearcher.java:170)
> > >>> at
> > >>> com.otherwise.parsnips.MySearcher.search(MySearcher.java:149)
> > >>> at com.otherwise.parsnips.Parsnips.<init>(Parsnips.java:163)
> > >>> at com.otherwise.parsnips.Parsnips.main(Parsnips.java:1205)
> > >>>
> > >>> I can't understand why I would be getting this for one sort field
> > >>> but not the other given there are 0 hits anyway in a newly created
> > >>> index. Anyone have any thoughts? I am using Lucene 1.4.2.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Yonik Seeley <ys...@gmail.com>.
> Any fieldName that starts with "i" or
> below (including capitals) works. Can anyone think of what could
> possibly be going on here?
Looks like you uncovered an obscure sorting bug.
The reason that fields >= "j" fail is that your last indexed field
(and hence the last indexed term) starts with "i" (specifically
"indexVersion").
> private static String VERSION_CREATOR_KEY = "creatorVer";
> private static String INDEX_VERSION_KEY = "indexVersion";
If you changed these to "a" and "aa", then all three tests would fail.
-Yonik
On Apr 12, 2005 2:04 PM, Bill Tschumy <bi...@otherwise.com> wrote:
> On Apr 12, 2005, at 8:38 AM, Erik Hatcher wrote:
>
> > Could you give us a self-contained test case that reproduces this
> > issue?
> >
> > Erik
> >
>
> Here is a small program that will manifest the error. Hopefully
> someone can explain the problem. It happens with Lucene 1.4.2 and
> 1.4.3.
>
> file: SortProblem.java
> =========================
> import java.io.*;
> import org.apache.lucene.search.*;
> import org.apache.lucene.index.*;
> import org.apache.lucene.store.*;
> import org.apache.lucene.document.*;
> import org.apache.lucene.analysis.standard.StandardAnalyzer;
>
> /**
> * This program demonstrates a problem I'm having with Lucene. If I
> search for
> * documents with a particular field name/value pair and sort them
> based upon
> * another field, I sometimes get a RuntimeException thrown. This
> happens when the
> * Hits comes back empty and the sort field name begins with a letter
> < "j". For
> * the bug to manifest it appears I also need to have an unrelated
> document in the
> * index that is storing version information.
> **/
>
> public class SortProblem
> {
>
> private static String INDEX_DIRECTORY = "SortProblemIndex";
>
> // This is the field name of a field in the dcoument that hold
> version
> // information. The bug happens if this field name has certain
> values.
> // "creatorVal" will fail, but "xreatorVal" will not. Very wierd.
> private static String VERSION_CREATOR_KEY = "creatorVer";
>
> private static String INDEX_VERSION_VAL = "indexVersionVal";
> private static String INDEX_VERSION_KEY = "indexVersion";
> private static String INDEX_VERSION = "1.1";
> private static String CREATOR_KEY = "creator";
> private static String PARSNIPS_VAL = "Parsnips";
>
> public static void main(String[] args)
> {
> initIndex(INDEX_DIRECTORY);
> // The search appears to fail if the fieldName starts with a
> letter >= "j".
> // The first and last search here will work while the middle
> will fail.
> search(INDEX_DIRECTORY, "aaa");
> search(INDEX_DIRECTORY, "mmm");
> search(INDEX_DIRECTORY, "bbb");
> }
>
> private static void initIndex(String directoryName)
> {
> File indexDir = new File(directoryName);
> if (indexDir.exists())
> deleteFileOrDirectory(indexDir);
> indexDir.mkdir();
> try
> {
> IndexWriter writer = new IndexWriter(indexDir, new
> StandardAnalyzer(), true);
> // Adding the one document which contains version
> information seems
> // necessary to cause some search/sorts to fail.
> Document doc = new Document();
> doc.add(Field.Keyword(VERSION_CREATOR_KEY,
> INDEX_VERSION_VAL));
> doc.add(Field.Keyword(INDEX_VERSION_KEY, INDEX_VERSION));
> writer.addDocument(doc);
> writer.close();
> }
> catch (IOException ioe)
> {
> ioe.printStackTrace();
> }
> }
>
> private static void search(String directoryName, String
> sortFieldName)
> {
> try
> {
> File indexDir = new File(directoryName);
> Directory fsDir = FSDirectory.getDirectory(indexDir, false);
> IndexSearcher searcher = new IndexSearcher(fsDir);
> Hits hits;
> Query query = new TermQuery(new Term(CREATOR_KEY,
> PARSNIPS_VAL));
> Sort sorter = new Sort(new SortField(sortFieldName,
> SortField.STRING, true));
> try
> {
> hits = searcher.search(query, sorter);
> System.out.println("sort on " + sortFieldName + "
> successful.");
> }
> catch (RuntimeException e)
> {
> System.out.println("sort on " + sortFieldName + "
> failed.");
> e.printStackTrace();
> }
> }
> catch (IOException e)
> {
> e.printStackTrace();
> }
>
> }
>
> private static boolean deleteFileOrDirectory(File dir)
> {
> if (dir.isDirectory())
> {
> String[] children = dir.list();
> for (int i = 0; i < children.length; i++)
> {
> boolean success = deleteFileOrDirectory(new File(dir,
> children[i]));
> if (!success)
> {
> return false;
> }
> }
> }
> // The directory is now empty so delete it
> return dir.delete();
> }
> }
>
> > On Apr 12, 2005, at 9:19 AM, Bill Tschumy wrote:
> >
> >> This problem is seeming more and more strange. It now looks like if
> >> the fieldName I'm sorting on starts is ASCII "j" or above, the
> >> RuntimeException is thrown. Any fieldName that starts with "i" or
> >> below (including capitals) works. Can anyone think of what could
> >> possibly be going on here?
> >>
> >>
> >> On Apr 11, 2005, at 2:27 PM, Bill Tschumy wrote:
> >>
> >>> In my application, by default I display all documents that are in
> >>> the index. I sort them either using a "time modified" or "time
> >>> created". If I have a newly created empty index, I find I get an
> >>> error if I sort by "time modified" but not "time created". In
> >>> either case there are actually no documents that match my query so
> >>> in reality there is nothing to sort.
> >>>
> >>> Here is my query:
> >>>
> >>> query = new TermQuery(new Term(MyIndexer.CREATOR_KEY,
> >>> MyIndexer.PARSNIPS_VAL));
> >>> String fieldName = sortType == Parsnips.SORT_BY_MODIFIED ?
> >>> MyIndexer.MODIFIED_KEY : MyIndexer.CREATED_KEY;
> >>> Sort sorter = new Sort(new SortField(fieldName, SortField.STRING,
> >>> true));
> >>> hits = searcher.search(query, sorter);
> >>>
> >>> The error I'm getting when using MyIndexer.MODIFIED_KEY (which is
> >>> "modified") as the sort field is:
> >>>
> >>> java.lang.RuntimeException: no terms in field modified
> >>> at
> >>> org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl
> >>> .java:256)
> >>> at
> >>> org.apache.lucene.search.FieldSortedHitQueue.comparatorString(FieldSo
> >>> rtedHitQueue.java:265)
> >>> at
> >>> org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(Fiel
> >>> dSortedHitQueue.java:180)
> >>> at
> >>> org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQue
> >>> ue.java:58)
> >>> at
> >>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:
> >>> 122)
> >>> at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:64)
> >>> at org.apache.lucene.search.Hits.<init>(Hits.java:51)
> >>> at org.apache.lucene.search.Searcher.search(Searcher.java:41)
> >>> at
> >>> com.otherwise.parsnips.MySearcher.search(MySearcher.java:170)
> >>> at
> >>> com.otherwise.parsnips.MySearcher.search(MySearcher.java:149)
> >>> at com.otherwise.parsnips.Parsnips.<init>(Parsnips.java:163)
> >>> at com.otherwise.parsnips.Parsnips.main(Parsnips.java:1205)
> >>>
> >>> I can't understand why I would be getting this for one sort field
> >>> but not the other given there are 0 hits anyway in a newly created
> >>> index. Anyone have any thoughts? I am using Lucene 1.4.2.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Bill Tschumy <bi...@otherwise.com>.
On Apr 12, 2005, at 8:38 AM, Erik Hatcher wrote:
> Could you give us a self-contained test case that reproduces this
> issue?
>
> Erik
>
Here is a small program that will manifest the error. Hopefully
someone can explain the problem. It happens with Lucene 1.4.2 and
1.4.3.
file: SortProblem.java
=========================
import java.io.*;
import org.apache.lucene.search.*;
import org.apache.lucene.index.*;
import org.apache.lucene.store.*;
import org.apache.lucene.document.*;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
/**
* This program demonstrates a problem I'm having with Lucene. If I
search for
* documents with a particular field name/value pair and sort them
based upon
* another field, I sometimes get a RuntimeException thrown. This
happens when the
* Hits comes back empty and the sort field name begins with a letter
< "j". For
* the bug to manifest it appears I also need to have an unrelated
document in the
* index that is storing version information.
**/
public class SortProblem
{
private static String INDEX_DIRECTORY = "SortProblemIndex";
// This is the field name of a field in the dcoument that hold
version
// information. The bug happens if this field name has certain
values.
// "creatorVal" will fail, but "xreatorVal" will not. Very wierd.
private static String VERSION_CREATOR_KEY = "creatorVer";
private static String INDEX_VERSION_VAL = "indexVersionVal";
private static String INDEX_VERSION_KEY = "indexVersion";
private static String INDEX_VERSION = "1.1";
private static String CREATOR_KEY = "creator";
private static String PARSNIPS_VAL = "Parsnips";
public static void main(String[] args)
{
initIndex(INDEX_DIRECTORY);
// The search appears to fail if the fieldName starts with a
letter >= "j".
// The first and last search here will work while the middle
will fail.
search(INDEX_DIRECTORY, "aaa");
search(INDEX_DIRECTORY, "mmm");
search(INDEX_DIRECTORY, "bbb");
}
private static void initIndex(String directoryName)
{
File indexDir = new File(directoryName);
if (indexDir.exists())
deleteFileOrDirectory(indexDir);
indexDir.mkdir();
try
{
IndexWriter writer = new IndexWriter(indexDir, new
StandardAnalyzer(), true);
// Adding the one document which contains version
information seems
// necessary to cause some search/sorts to fail.
Document doc = new Document();
doc.add(Field.Keyword(VERSION_CREATOR_KEY,
INDEX_VERSION_VAL));
doc.add(Field.Keyword(INDEX_VERSION_KEY, INDEX_VERSION));
writer.addDocument(doc);
writer.close();
}
catch (IOException ioe)
{
ioe.printStackTrace();
}
}
private static void search(String directoryName, String
sortFieldName)
{
try
{
File indexDir = new File(directoryName);
Directory fsDir = FSDirectory.getDirectory(indexDir, false);
IndexSearcher searcher = new IndexSearcher(fsDir);
Hits hits;
Query query = new TermQuery(new Term(CREATOR_KEY,
PARSNIPS_VAL));
Sort sorter = new Sort(new SortField(sortFieldName,
SortField.STRING, true));
try
{
hits = searcher.search(query, sorter);
System.out.println("sort on " + sortFieldName + "
successful.");
}
catch (RuntimeException e)
{
System.out.println("sort on " + sortFieldName + "
failed.");
e.printStackTrace();
}
}
catch (IOException e)
{
e.printStackTrace();
}
}
private static boolean deleteFileOrDirectory(File dir)
{
if (dir.isDirectory())
{
String[] children = dir.list();
for (int i = 0; i < children.length; i++)
{
boolean success = deleteFileOrDirectory(new File(dir,
children[i]));
if (!success)
{
return false;
}
}
}
// The directory is now empty so delete it
return dir.delete();
}
}
> On Apr 12, 2005, at 9:19 AM, Bill Tschumy wrote:
>
>> This problem is seeming more and more strange. It now looks like if
>> the fieldName I'm sorting on starts is ASCII "j" or above, the
>> RuntimeException is thrown. Any fieldName that starts with "i" or
>> below (including capitals) works. Can anyone think of what could
>> possibly be going on here?
>>
>>
>> On Apr 11, 2005, at 2:27 PM, Bill Tschumy wrote:
>>
>>> In my application, by default I display all documents that are in
>>> the index. I sort them either using a "time modified" or "time
>>> created". If I have a newly created empty index, I find I get an
>>> error if I sort by "time modified" but not "time created". In
>>> either case there are actually no documents that match my query so
>>> in reality there is nothing to sort.
>>>
>>> Here is my query:
>>>
>>> query = new TermQuery(new Term(MyIndexer.CREATOR_KEY,
>>> MyIndexer.PARSNIPS_VAL));
>>> String fieldName = sortType == Parsnips.SORT_BY_MODIFIED ?
>>> MyIndexer.MODIFIED_KEY : MyIndexer.CREATED_KEY;
>>> Sort sorter = new Sort(new SortField(fieldName, SortField.STRING,
>>> true));
>>> hits = searcher.search(query, sorter);
>>>
>>> The error I'm getting when using MyIndexer.MODIFIED_KEY (which is
>>> "modified") as the sort field is:
>>>
>>> java.lang.RuntimeException: no terms in field modified
>>> at
>>> org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl
>>> .java:256)
>>> at
>>> org.apache.lucene.search.FieldSortedHitQueue.comparatorString(FieldSo
>>> rtedHitQueue.java:265)
>>> at
>>> org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(Fiel
>>> dSortedHitQueue.java:180)
>>> at
>>> org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQue
>>> ue.java:58)
>>> at
>>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:
>>> 122)
>>> at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:64)
>>> at org.apache.lucene.search.Hits.<init>(Hits.java:51)
>>> at org.apache.lucene.search.Searcher.search(Searcher.java:41)
>>> at
>>> com.otherwise.parsnips.MySearcher.search(MySearcher.java:170)
>>> at
>>> com.otherwise.parsnips.MySearcher.search(MySearcher.java:149)
>>> at com.otherwise.parsnips.Parsnips.<init>(Parsnips.java:163)
>>> at com.otherwise.parsnips.Parsnips.main(Parsnips.java:1205)
>>>
>>> I can't understand why I would be getting this for one sort field
>>> but not the other given there are 0 hits anyway in a newly created
>>> index. Anyone have any thoughts? I am using Lucene 1.4.2.
>>>
>>> --
>>> Bill Tschumy
>>> Otherwise -- Austin, TX
>>> http://www.otherwise.com
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>> --
>> Bill Tschumy
>> Otherwise -- Austin, TX
>> http://www.otherwise.com
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Bill Tschumy
Otherwise -- Austin, TX
http://www.otherwise.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
Could you give us a self-contained test case that reproduces this issue?
Erik
On Apr 12, 2005, at 9:19 AM, Bill Tschumy wrote:
> This problem is seeming more and more strange. It now looks like if
> the fieldName I'm sorting on starts is ASCII "j" or above, the
> RuntimeException is thrown. Any fieldName that starts with "i" or
> below (including capitals) works. Can anyone think of what could
> possibly be going on here?
>
>
> On Apr 11, 2005, at 2:27 PM, Bill Tschumy wrote:
>
>> In my application, by default I display all documents that are in the
>> index. I sort them either using a "time modified" or "time created".
>> If I have a newly created empty index, I find I get an error if I
>> sort by "time modified" but not "time created". In either case there
>> are actually no documents that match my query so in reality there is
>> nothing to sort.
>>
>> Here is my query:
>>
>> query = new TermQuery(new Term(MyIndexer.CREATOR_KEY,
>> MyIndexer.PARSNIPS_VAL));
>> String fieldName = sortType == Parsnips.SORT_BY_MODIFIED ?
>> MyIndexer.MODIFIED_KEY : MyIndexer.CREATED_KEY;
>> Sort sorter = new Sort(new SortField(fieldName, SortField.STRING,
>> true));
>> hits = searcher.search(query, sorter);
>>
>> The error I'm getting when using MyIndexer.MODIFIED_KEY (which is
>> "modified") as the sort field is:
>>
>> java.lang.RuntimeException: no terms in field modified
>> at
>> org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.
>> java:256)
>> at
>> org.apache.lucene.search.FieldSortedHitQueue.comparatorString(FieldSor
>> tedHitQueue.java:265)
>> at
>> org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(Field
>> SortedHitQueue.java:180)
>> at
>> org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueu
>> e.java:58)
>> at
>> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:122)
>> at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:64)
>> at org.apache.lucene.search.Hits.<init>(Hits.java:51)
>> at org.apache.lucene.search.Searcher.search(Searcher.java:41)
>> at
>> com.otherwise.parsnips.MySearcher.search(MySearcher.java:170)
>> at
>> com.otherwise.parsnips.MySearcher.search(MySearcher.java:149)
>> at com.otherwise.parsnips.Parsnips.<init>(Parsnips.java:163)
>> at com.otherwise.parsnips.Parsnips.main(Parsnips.java:1205)
>>
>> I can't understand why I would be getting this for one sort field but
>> not the other given there are 0 hits anyway in a newly created index.
>> Anyone have any thoughts? I am using Lucene 1.4.2.
>>
>> --
>> Bill Tschumy
>> Otherwise -- Austin, TX
>> http://www.otherwise.com
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
> --
> Bill Tschumy
> Otherwise -- Austin, TX
> http://www.otherwise.com
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange sort error
Posted by Bill Tschumy <bi...@otherwise.com>.
This problem is seeming more and more strange. It now looks like if
the fieldName I'm sorting on starts is ASCII "j" or above, the
RuntimeException is thrown. Any fieldName that starts with "i" or
below (including capitals) works. Can anyone think of what could
possibly be going on here?
On Apr 11, 2005, at 2:27 PM, Bill Tschumy wrote:
> In my application, by default I display all documents that are in the
> index. I sort them either using a "time modified" or "time created".
> If I have a newly created empty index, I find I get an error if I sort
> by "time modified" but not "time created". In either case there are
> actually no documents that match my query so in reality there is
> nothing to sort.
>
> Here is my query:
>
> query = new TermQuery(new Term(MyIndexer.CREATOR_KEY,
> MyIndexer.PARSNIPS_VAL));
> String fieldName = sortType == Parsnips.SORT_BY_MODIFIED ?
> MyIndexer.MODIFIED_KEY : MyIndexer.CREATED_KEY;
> Sort sorter = new Sort(new SortField(fieldName, SortField.STRING,
> true));
> hits = searcher.search(query, sorter);
>
> The error I'm getting when using MyIndexer.MODIFIED_KEY (which is
> "modified") as the sort field is:
>
> java.lang.RuntimeException: no terms in field modified
> at
> org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.j
> ava:256)
> at
> org.apache.lucene.search.FieldSortedHitQueue.comparatorString(FieldSort
> edHitQueue.java:265)
> at
> org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldS
> ortedHitQueue.java:180)
> at
> org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueue
> .java:58)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:122)
> at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:64)
> at org.apache.lucene.search.Hits.<init>(Hits.java:51)
> at org.apache.lucene.search.Searcher.search(Searcher.java:41)
> at
> com.otherwise.parsnips.MySearcher.search(MySearcher.java:170)
> at
> com.otherwise.parsnips.MySearcher.search(MySearcher.java:149)
> at com.otherwise.parsnips.Parsnips.<init>(Parsnips.java:163)
> at com.otherwise.parsnips.Parsnips.main(Parsnips.java:1205)
>
> I can't understand why I would be getting this for one sort field but
> not the other given there are 0 hits anyway in a newly created index.
> Anyone have any thoughts? I am using Lucene 1.4.2.
>
> --
> Bill Tschumy
> Otherwise -- Austin, TX
> http://www.otherwise.com
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Bill Tschumy
Otherwise -- Austin, TX
http://www.otherwise.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org