You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Li Li <fa...@gmail.com> on 2012/03/23 05:27:05 UTC
ToParentBlockJoinQuery query loop finitely
hi all,
I read these two articles
http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html,
http://blog.mikemccandless.com/2012/01/tochildblockjoinquery-in-lucene.htmland
wrote a test program. But it seems there is some problem. it ends with
endless loop.
Here is my program, almost copy from that article, the only
modification is from BlockJoinQuery to ToParentBlockJoinQuery because I
can't find BlockJoinQuery in latest 4.0 dev trunk source tree.
another modification is from SocreMode.none to ScoreMode.Avg because
when using SocreMode.None, an exception throws:
Exception in thread "main" java.lang.IllegalStateException: ScoreMode is
None
at
org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.swapChildScores(ToParentBlockJoinQuery.java:270)
at
org.apache.lucene.search.join.ToParentBlockJoinCollector.copyGroups(ToParentBlockJoinCollector.java:251)
So I change ScoreMode.None to Avg
// BlockJoinQuery skuJoinQuery = new BlockJoinQuery(skuQuery, shirts,
// ScoreMode.None);
ToParentBlockJoinQuery skuJoinQuery = new
ToParentBlockJoinQuery(skuQuery, shirts,
ScoreMode.Avg);
Then I run again, it ends up with endless loop in BooleanScorer2:
public void score(Collector collector) throws IOException {
collector.setScorer(this);
while ((doc = countingSumScorer.nextDoc()) != NO_MORE_DOCS) {
collector.collect(doc);
}
}
current doc is -1, and countingSumScorer.nextDoc() always return -1!
following is my program. btw, what's the status of query time join? what's
the relationship of solr's join and lucene's join?
-------------my codes-------------
public class TestJoin {
/**
* @param args
* @throws Exception
*/
public static void main(String[] args) throws Exception {
File f=new File("./testindex");
if(!f.exists()){
f.mkdirs();
}
Directory dir=FSDirectory.open(f);
for(String name:dir.listAll()){
dir.deleteFile(name);
}
IndexWriterConfig conf=new IndexWriterConfig(Version.LUCENE_40,new
WhitespaceAnalyzer(Version.LUCENE_40));
IndexWriter writer=new IndexWriter(dir, conf);
FieldType fType=new FieldType();
fType.setIndexed(true);
fType.setStored(true);
fType.setIndexOptions(IndexOptions.DOCS_AND_FREQS);
fType.setTokenized(true);
Document shirt=new Document();
shirt.add(new Field("name","three wolf",fType));
shirt.add(new StringField("type", "shirt"));
Document sku1=new Document();
sku1.add(new StringField("size","small"));
sku1.add(new StringField("color","blue"));
Document sku2=new Document();
sku2.add(new StringField("size","small"));
sku2.add(new StringField("color","black"));
Document sku3=new Document();
sku3.add(new StringField("size","medium"));
sku3.add(new StringField("color","black"));
Document sku4=new Document();
sku4.add(new StringField("size","large"));
sku4.add(new StringField("color","gray"));
List<Document> shirtGroup=new ArrayList<Document>();
shirtGroup.add(shirt);
shirtGroup.add(sku1);
shirtGroup.add(sku2);
shirtGroup.add(sku3);
shirtGroup.add(sku4);
writer.addDocuments(shirtGroup);
shirt=new Document();
shirt.add(new Field("name","seven wolf",fType));
shirt.add(new StringField("type", "shirt"));
sku1=new Document();
sku1.add(new StringField("size","large"));
sku1.add(new StringField("color","blue"));
sku2=new Document();
sku2.add(new StringField("size","small"));
sku2.add(new StringField("color","black"));
shirtGroup=new ArrayList<Document>();
shirtGroup.add(shirt);
shirtGroup.add(sku1);
shirtGroup.add(sku2);
writer.addDocuments(shirtGroup);
writer.close();
IndexSearcher searcher=new IndexSearcher(DirectoryReader.open(dir));
Filter shirts = new CachingWrapperFilter(new QueryWrapperFilter(
new TermQuery(new Term("type", "shirt"))));
BooleanQuery skuQuery = new BooleanQuery();
skuQuery.add(new TermQuery(new Term("size", "small")), Occur.MUST);
skuQuery.add(new TermQuery(new Term("color", "black")), Occur.MUST);
// BlockJoinQuery skuJoinQuery = new BlockJoinQuery(skuQuery, shirts,
// ScoreMode.None);
ToParentBlockJoinQuery skuJoinQuery = new
ToParentBlockJoinQuery(skuQuery, shirts,
ScoreMode.Avg);
BooleanQuery query = new BooleanQuery();
query.add(new TermQuery(new Term("name", "wolf")), Occur.MUST);
query.add(skuJoinQuery, Occur.MUST);
ToParentBlockJoinCollector c = new ToParentBlockJoinCollector(
Sort.RELEVANCE, // sort
10, // numHits
true, // trackScores
false // trackMaxScore
);
searcher.search(query, c);
Sort skuSort=new Sort(new SortField("size",Type.STRING));
TopGroups hits = c.getTopGroups(skuJoinQuery, skuSort, 0, // offset
10, // maxDocsPerGroup
0, // withinGroupOffset
true // fillSortFields
);
System.out.println("totalHitCount: " + hits.totalHitCount);
System.out.println("totalGroupedHitCount: " +
hits.totalGroupedHitCount);
for(GroupDocs group:hits.groups){
System.out.println("totalHits: "+group.totalHits);
for(ScoreDoc doc:group.scoreDocs){
System.out.println(doc.doc+", "+doc.score);
}
}
}
}
Re: ToParentBlockJoinQuery query loop finitely
Posted by Li Li <fa...@gmail.com>.
that's fine. thank you.
On Fri, Mar 23, 2012 at 7:30 PM, Martijn v Groningen <
martijn.v.groningen@gmail.com> wrote:
> Hi,
>
> There is an example usage in the package.html.
> Something to keep in mind is that this search time join is different than
> block join query. The JoinUtil can be used to do join filtering and the
> block join is more meant for parent / child search.
>
> Martijn
>
> On 23 March 2012 11:58, Li Li <fa...@gmail.com> wrote:
>
> > thank you. is there any the search time join example?
> > I can only find a JoinUtil in package org.apache.lucene.search.join and a
> > TestJoinUtil in test package.
> >
> >
> > On Fri, Mar 23, 2012 at 6:24 PM, Michael McCandless <
> > lucene@mikemccandless.com> wrote:
> >
> > > I think you're hitting the exception because you passed
> > > trackScores=true to ToParentBlockJoinCollector. If you do that, the
> > > ScoreMode cannot be None... I'll update the javadocs to make this
> > > clear, and I'll also fix the exception message.
> > >
> > > I think you're hitting the infinite loop because your parent document
> > > is first, not last, in the group. Try putting the parent (shirt)
> > > document last in each case instead...
> > >
> > > Query-time join is already committed to trunk and 3.x, so it'll be in
> > > 3.6.0/4.0.
> > >
> > > Mike McCandless
> > >
> > > http://blog.mikemccandless.com
> > >
> > > On Fri, Mar 23, 2012 at 12:27 AM, Li Li <fa...@gmail.com> wrote:
> > > > hi all,
> > > > I read these two articles
> > > >
> > >
> >
> http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html
> > > ,
> > > >
> > >
> >
> http://blog.mikemccandless.com/2012/01/tochildblockjoinquery-in-lucene.htmland
> > > > wrote a test program. But it seems there is some problem. it ends
> with
> > > > endless loop.
> > > > Here is my program, almost copy from that article, the only
> > > > modification is from BlockJoinQuery to ToParentBlockJoinQuery
> because I
> > > > can't find BlockJoinQuery in latest 4.0 dev trunk source tree.
> > > > another modification is from SocreMode.none to ScoreMode.Avg
> because
> > > > when using SocreMode.None, an exception throws:
> > > > Exception in thread "main" java.lang.IllegalStateException: ScoreMode
> > is
> > > > None
> > > > at
> > > >
> > >
> >
> org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.swapChildScores(ToParentBlockJoinQuery.java:270)
> > > > at
> > > >
> > >
> >
> org.apache.lucene.search.join.ToParentBlockJoinCollector.copyGroups(ToParentBlockJoinCollector.java:251)
> > > > So I change ScoreMode.None to Avg
> > > > // BlockJoinQuery skuJoinQuery = new BlockJoinQuery(skuQuery,
> > > shirts,
> > > > // ScoreMode.None);
> > > > ToParentBlockJoinQuery skuJoinQuery = new
> > > > ToParentBlockJoinQuery(skuQuery, shirts,
> > > > ScoreMode.Avg);
> > > >
> > > >
> > > > Then I run again, it ends up with endless loop in BooleanScorer2:
> > > > public void score(Collector collector) throws IOException {
> > > > collector.setScorer(this);
> > > > while ((doc = countingSumScorer.nextDoc()) != NO_MORE_DOCS) {
> > > > collector.collect(doc);
> > > > }
> > > > }
> > > >
> > > > current doc is -1, and countingSumScorer.nextDoc() always return
> -1!
> > > > following is my program. btw, what's the status of query time join?
> > > what's
> > > > the relationship of solr's join and lucene's join?
> > > > -------------my codes-------------
> > > > public class TestJoin {
> > > >
> > > > /**
> > > > * @param args
> > > > * @throws Exception
> > > > */
> > > > public static void main(String[] args) throws Exception {
> > > > File f=new File("./testindex");
> > > > if(!f.exists()){
> > > > f.mkdirs();
> > > > }
> > > > Directory dir=FSDirectory.open(f);
> > > > for(String name:dir.listAll()){
> > > > dir.deleteFile(name);
> > > > }
> > > > IndexWriterConfig conf=new
> > IndexWriterConfig(Version.LUCENE_40,new
> > > > WhitespaceAnalyzer(Version.LUCENE_40));
> > > > IndexWriter writer=new IndexWriter(dir, conf);
> > > > FieldType fType=new FieldType();
> > > > fType.setIndexed(true);
> > > > fType.setStored(true);
> > > > fType.setIndexOptions(IndexOptions.DOCS_AND_FREQS);
> > > > fType.setTokenized(true);
> > > > Document shirt=new Document();
> > > > shirt.add(new Field("name","three wolf",fType));
> > > > shirt.add(new StringField("type", "shirt"));
> > > > Document sku1=new Document();
> > > > sku1.add(new StringField("size","small"));
> > > > sku1.add(new StringField("color","blue"));
> > > > Document sku2=new Document();
> > > > sku2.add(new StringField("size","small"));
> > > > sku2.add(new StringField("color","black"));
> > > > Document sku3=new Document();
> > > > sku3.add(new StringField("size","medium"));
> > > > sku3.add(new StringField("color","black"));
> > > > Document sku4=new Document();
> > > > sku4.add(new StringField("size","large"));
> > > > sku4.add(new StringField("color","gray"));
> > > > List<Document> shirtGroup=new ArrayList<Document>();
> > > > shirtGroup.add(shirt);
> > > > shirtGroup.add(sku1);
> > > > shirtGroup.add(sku2);
> > > > shirtGroup.add(sku3);
> > > > shirtGroup.add(sku4);
> > > > writer.addDocuments(shirtGroup);
> > > >
> > > > shirt=new Document();
> > > > shirt.add(new Field("name","seven wolf",fType));
> > > > shirt.add(new StringField("type", "shirt"));
> > > > sku1=new Document();
> > > > sku1.add(new StringField("size","large"));
> > > > sku1.add(new StringField("color","blue"));
> > > > sku2=new Document();
> > > > sku2.add(new StringField("size","small"));
> > > > sku2.add(new StringField("color","black"));
> > > >
> > > > shirtGroup=new ArrayList<Document>();
> > > > shirtGroup.add(shirt);
> > > > shirtGroup.add(sku1);
> > > > shirtGroup.add(sku2);
> > > > writer.addDocuments(shirtGroup);
> > > > writer.close();
> > > >
> > > > IndexSearcher searcher=new
> > > IndexSearcher(DirectoryReader.open(dir));
> > > > Filter shirts = new CachingWrapperFilter(new
> QueryWrapperFilter(
> > > > new TermQuery(new Term("type", "shirt"))));
> > > > BooleanQuery skuQuery = new BooleanQuery();
> > > > skuQuery.add(new TermQuery(new Term("size", "small")),
> > > Occur.MUST);
> > > > skuQuery.add(new TermQuery(new Term("color", "black")),
> > > Occur.MUST);
> > > > // BlockJoinQuery skuJoinQuery = new BlockJoinQuery(skuQuery,
> > > shirts,
> > > > // ScoreMode.None);
> > > > ToParentBlockJoinQuery skuJoinQuery = new
> > > > ToParentBlockJoinQuery(skuQuery, shirts,
> > > > ScoreMode.Avg);
> > > >
> > > > BooleanQuery query = new BooleanQuery();
> > > > query.add(new TermQuery(new Term("name", "wolf")),
> Occur.MUST);
> > > > query.add(skuJoinQuery, Occur.MUST);
> > > >
> > > > ToParentBlockJoinCollector c = new ToParentBlockJoinCollector(
> > > > Sort.RELEVANCE, // sort
> > > > 10, // numHits
> > > > true, // trackScores
> > > > false // trackMaxScore
> > > > );
> > > > searcher.search(query, c);
> > > >
> > > > Sort skuSort=new Sort(new SortField("size",Type.STRING));
> > > > TopGroups hits = c.getTopGroups(skuJoinQuery, skuSort, 0, //
> > > offset
> > > > 10, // maxDocsPerGroup
> > > > 0, // withinGroupOffset
> > > > true // fillSortFields
> > > > );
> > > > System.out.println("totalHitCount: " + hits.totalHitCount);
> > > > System.out.println("totalGroupedHitCount: " +
> > > > hits.totalGroupedHitCount);
> > > > for(GroupDocs group:hits.groups){
> > > > System.out.println("totalHits: "+group.totalHits);
> > > > for(ScoreDoc doc:group.scoreDocs){
> > > > System.out.println(doc.doc+", "+doc.score);
> > > > }
> > > > }
> > > > }
> > > >
> > > > }
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> >
>
>
>
> --
> Met vriendelijke groet,
>
> Martijn van Groningen
>
Re: ToParentBlockJoinQuery query loop finitely
Posted by Martijn v Groningen <ma...@gmail.com>.
Hi,
There is an example usage in the package.html.
Something to keep in mind is that this search time join is different than
block join query. The JoinUtil can be used to do join filtering and the
block join is more meant for parent / child search.
Martijn
On 23 March 2012 11:58, Li Li <fa...@gmail.com> wrote:
> thank you. is there any the search time join example?
> I can only find a JoinUtil in package org.apache.lucene.search.join and a
> TestJoinUtil in test package.
>
>
> On Fri, Mar 23, 2012 at 6:24 PM, Michael McCandless <
> lucene@mikemccandless.com> wrote:
>
> > I think you're hitting the exception because you passed
> > trackScores=true to ToParentBlockJoinCollector. If you do that, the
> > ScoreMode cannot be None... I'll update the javadocs to make this
> > clear, and I'll also fix the exception message.
> >
> > I think you're hitting the infinite loop because your parent document
> > is first, not last, in the group. Try putting the parent (shirt)
> > document last in each case instead...
> >
> > Query-time join is already committed to trunk and 3.x, so it'll be in
> > 3.6.0/4.0.
> >
> > Mike McCandless
> >
> > http://blog.mikemccandless.com
> >
> > On Fri, Mar 23, 2012 at 12:27 AM, Li Li <fa...@gmail.com> wrote:
> > > hi all,
> > > I read these two articles
> > >
> >
> http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html
> > ,
> > >
> >
> http://blog.mikemccandless.com/2012/01/tochildblockjoinquery-in-lucene.htmland
> > > wrote a test program. But it seems there is some problem. it ends with
> > > endless loop.
> > > Here is my program, almost copy from that article, the only
> > > modification is from BlockJoinQuery to ToParentBlockJoinQuery because I
> > > can't find BlockJoinQuery in latest 4.0 dev trunk source tree.
> > > another modification is from SocreMode.none to ScoreMode.Avg because
> > > when using SocreMode.None, an exception throws:
> > > Exception in thread "main" java.lang.IllegalStateException: ScoreMode
> is
> > > None
> > > at
> > >
> >
> org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.swapChildScores(ToParentBlockJoinQuery.java:270)
> > > at
> > >
> >
> org.apache.lucene.search.join.ToParentBlockJoinCollector.copyGroups(ToParentBlockJoinCollector.java:251)
> > > So I change ScoreMode.None to Avg
> > > // BlockJoinQuery skuJoinQuery = new BlockJoinQuery(skuQuery,
> > shirts,
> > > // ScoreMode.None);
> > > ToParentBlockJoinQuery skuJoinQuery = new
> > > ToParentBlockJoinQuery(skuQuery, shirts,
> > > ScoreMode.Avg);
> > >
> > >
> > > Then I run again, it ends up with endless loop in BooleanScorer2:
> > > public void score(Collector collector) throws IOException {
> > > collector.setScorer(this);
> > > while ((doc = countingSumScorer.nextDoc()) != NO_MORE_DOCS) {
> > > collector.collect(doc);
> > > }
> > > }
> > >
> > > current doc is -1, and countingSumScorer.nextDoc() always return -1!
> > > following is my program. btw, what's the status of query time join?
> > what's
> > > the relationship of solr's join and lucene's join?
> > > -------------my codes-------------
> > > public class TestJoin {
> > >
> > > /**
> > > * @param args
> > > * @throws Exception
> > > */
> > > public static void main(String[] args) throws Exception {
> > > File f=new File("./testindex");
> > > if(!f.exists()){
> > > f.mkdirs();
> > > }
> > > Directory dir=FSDirectory.open(f);
> > > for(String name:dir.listAll()){
> > > dir.deleteFile(name);
> > > }
> > > IndexWriterConfig conf=new
> IndexWriterConfig(Version.LUCENE_40,new
> > > WhitespaceAnalyzer(Version.LUCENE_40));
> > > IndexWriter writer=new IndexWriter(dir, conf);
> > > FieldType fType=new FieldType();
> > > fType.setIndexed(true);
> > > fType.setStored(true);
> > > fType.setIndexOptions(IndexOptions.DOCS_AND_FREQS);
> > > fType.setTokenized(true);
> > > Document shirt=new Document();
> > > shirt.add(new Field("name","three wolf",fType));
> > > shirt.add(new StringField("type", "shirt"));
> > > Document sku1=new Document();
> > > sku1.add(new StringField("size","small"));
> > > sku1.add(new StringField("color","blue"));
> > > Document sku2=new Document();
> > > sku2.add(new StringField("size","small"));
> > > sku2.add(new StringField("color","black"));
> > > Document sku3=new Document();
> > > sku3.add(new StringField("size","medium"));
> > > sku3.add(new StringField("color","black"));
> > > Document sku4=new Document();
> > > sku4.add(new StringField("size","large"));
> > > sku4.add(new StringField("color","gray"));
> > > List<Document> shirtGroup=new ArrayList<Document>();
> > > shirtGroup.add(shirt);
> > > shirtGroup.add(sku1);
> > > shirtGroup.add(sku2);
> > > shirtGroup.add(sku3);
> > > shirtGroup.add(sku4);
> > > writer.addDocuments(shirtGroup);
> > >
> > > shirt=new Document();
> > > shirt.add(new Field("name","seven wolf",fType));
> > > shirt.add(new StringField("type", "shirt"));
> > > sku1=new Document();
> > > sku1.add(new StringField("size","large"));
> > > sku1.add(new StringField("color","blue"));
> > > sku2=new Document();
> > > sku2.add(new StringField("size","small"));
> > > sku2.add(new StringField("color","black"));
> > >
> > > shirtGroup=new ArrayList<Document>();
> > > shirtGroup.add(shirt);
> > > shirtGroup.add(sku1);
> > > shirtGroup.add(sku2);
> > > writer.addDocuments(shirtGroup);
> > > writer.close();
> > >
> > > IndexSearcher searcher=new
> > IndexSearcher(DirectoryReader.open(dir));
> > > Filter shirts = new CachingWrapperFilter(new QueryWrapperFilter(
> > > new TermQuery(new Term("type", "shirt"))));
> > > BooleanQuery skuQuery = new BooleanQuery();
> > > skuQuery.add(new TermQuery(new Term("size", "small")),
> > Occur.MUST);
> > > skuQuery.add(new TermQuery(new Term("color", "black")),
> > Occur.MUST);
> > > // BlockJoinQuery skuJoinQuery = new BlockJoinQuery(skuQuery,
> > shirts,
> > > // ScoreMode.None);
> > > ToParentBlockJoinQuery skuJoinQuery = new
> > > ToParentBlockJoinQuery(skuQuery, shirts,
> > > ScoreMode.Avg);
> > >
> > > BooleanQuery query = new BooleanQuery();
> > > query.add(new TermQuery(new Term("name", "wolf")), Occur.MUST);
> > > query.add(skuJoinQuery, Occur.MUST);
> > >
> > > ToParentBlockJoinCollector c = new ToParentBlockJoinCollector(
> > > Sort.RELEVANCE, // sort
> > > 10, // numHits
> > > true, // trackScores
> > > false // trackMaxScore
> > > );
> > > searcher.search(query, c);
> > >
> > > Sort skuSort=new Sort(new SortField("size",Type.STRING));
> > > TopGroups hits = c.getTopGroups(skuJoinQuery, skuSort, 0, //
> > offset
> > > 10, // maxDocsPerGroup
> > > 0, // withinGroupOffset
> > > true // fillSortFields
> > > );
> > > System.out.println("totalHitCount: " + hits.totalHitCount);
> > > System.out.println("totalGroupedHitCount: " +
> > > hits.totalGroupedHitCount);
> > > for(GroupDocs group:hits.groups){
> > > System.out.println("totalHits: "+group.totalHits);
> > > for(ScoreDoc doc:group.scoreDocs){
> > > System.out.println(doc.doc+", "+doc.score);
> > > }
> > > }
> > > }
> > >
> > > }
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
--
Met vriendelijke groet,
Martijn van Groningen
Re: ToParentBlockJoinQuery query loop finitely
Posted by Li Li <fa...@gmail.com>.
thank you. is there any the search time join example?
I can only find a JoinUtil in package org.apache.lucene.search.join and a
TestJoinUtil in test package.
On Fri, Mar 23, 2012 at 6:24 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:
> I think you're hitting the exception because you passed
> trackScores=true to ToParentBlockJoinCollector. If you do that, the
> ScoreMode cannot be None... I'll update the javadocs to make this
> clear, and I'll also fix the exception message.
>
> I think you're hitting the infinite loop because your parent document
> is first, not last, in the group. Try putting the parent (shirt)
> document last in each case instead...
>
> Query-time join is already committed to trunk and 3.x, so it'll be in
> 3.6.0/4.0.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Fri, Mar 23, 2012 at 12:27 AM, Li Li <fa...@gmail.com> wrote:
> > hi all,
> > I read these two articles
> >
> http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html
> ,
> >
> http://blog.mikemccandless.com/2012/01/tochildblockjoinquery-in-lucene.htmland
> > wrote a test program. But it seems there is some problem. it ends with
> > endless loop.
> > Here is my program, almost copy from that article, the only
> > modification is from BlockJoinQuery to ToParentBlockJoinQuery because I
> > can't find BlockJoinQuery in latest 4.0 dev trunk source tree.
> > another modification is from SocreMode.none to ScoreMode.Avg because
> > when using SocreMode.None, an exception throws:
> > Exception in thread "main" java.lang.IllegalStateException: ScoreMode is
> > None
> > at
> >
> org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.swapChildScores(ToParentBlockJoinQuery.java:270)
> > at
> >
> org.apache.lucene.search.join.ToParentBlockJoinCollector.copyGroups(ToParentBlockJoinCollector.java:251)
> > So I change ScoreMode.None to Avg
> > // BlockJoinQuery skuJoinQuery = new BlockJoinQuery(skuQuery,
> shirts,
> > // ScoreMode.None);
> > ToParentBlockJoinQuery skuJoinQuery = new
> > ToParentBlockJoinQuery(skuQuery, shirts,
> > ScoreMode.Avg);
> >
> >
> > Then I run again, it ends up with endless loop in BooleanScorer2:
> > public void score(Collector collector) throws IOException {
> > collector.setScorer(this);
> > while ((doc = countingSumScorer.nextDoc()) != NO_MORE_DOCS) {
> > collector.collect(doc);
> > }
> > }
> >
> > current doc is -1, and countingSumScorer.nextDoc() always return -1!
> > following is my program. btw, what's the status of query time join?
> what's
> > the relationship of solr's join and lucene's join?
> > -------------my codes-------------
> > public class TestJoin {
> >
> > /**
> > * @param args
> > * @throws Exception
> > */
> > public static void main(String[] args) throws Exception {
> > File f=new File("./testindex");
> > if(!f.exists()){
> > f.mkdirs();
> > }
> > Directory dir=FSDirectory.open(f);
> > for(String name:dir.listAll()){
> > dir.deleteFile(name);
> > }
> > IndexWriterConfig conf=new IndexWriterConfig(Version.LUCENE_40,new
> > WhitespaceAnalyzer(Version.LUCENE_40));
> > IndexWriter writer=new IndexWriter(dir, conf);
> > FieldType fType=new FieldType();
> > fType.setIndexed(true);
> > fType.setStored(true);
> > fType.setIndexOptions(IndexOptions.DOCS_AND_FREQS);
> > fType.setTokenized(true);
> > Document shirt=new Document();
> > shirt.add(new Field("name","three wolf",fType));
> > shirt.add(new StringField("type", "shirt"));
> > Document sku1=new Document();
> > sku1.add(new StringField("size","small"));
> > sku1.add(new StringField("color","blue"));
> > Document sku2=new Document();
> > sku2.add(new StringField("size","small"));
> > sku2.add(new StringField("color","black"));
> > Document sku3=new Document();
> > sku3.add(new StringField("size","medium"));
> > sku3.add(new StringField("color","black"));
> > Document sku4=new Document();
> > sku4.add(new StringField("size","large"));
> > sku4.add(new StringField("color","gray"));
> > List<Document> shirtGroup=new ArrayList<Document>();
> > shirtGroup.add(shirt);
> > shirtGroup.add(sku1);
> > shirtGroup.add(sku2);
> > shirtGroup.add(sku3);
> > shirtGroup.add(sku4);
> > writer.addDocuments(shirtGroup);
> >
> > shirt=new Document();
> > shirt.add(new Field("name","seven wolf",fType));
> > shirt.add(new StringField("type", "shirt"));
> > sku1=new Document();
> > sku1.add(new StringField("size","large"));
> > sku1.add(new StringField("color","blue"));
> > sku2=new Document();
> > sku2.add(new StringField("size","small"));
> > sku2.add(new StringField("color","black"));
> >
> > shirtGroup=new ArrayList<Document>();
> > shirtGroup.add(shirt);
> > shirtGroup.add(sku1);
> > shirtGroup.add(sku2);
> > writer.addDocuments(shirtGroup);
> > writer.close();
> >
> > IndexSearcher searcher=new
> IndexSearcher(DirectoryReader.open(dir));
> > Filter shirts = new CachingWrapperFilter(new QueryWrapperFilter(
> > new TermQuery(new Term("type", "shirt"))));
> > BooleanQuery skuQuery = new BooleanQuery();
> > skuQuery.add(new TermQuery(new Term("size", "small")),
> Occur.MUST);
> > skuQuery.add(new TermQuery(new Term("color", "black")),
> Occur.MUST);
> > // BlockJoinQuery skuJoinQuery = new BlockJoinQuery(skuQuery,
> shirts,
> > // ScoreMode.None);
> > ToParentBlockJoinQuery skuJoinQuery = new
> > ToParentBlockJoinQuery(skuQuery, shirts,
> > ScoreMode.Avg);
> >
> > BooleanQuery query = new BooleanQuery();
> > query.add(new TermQuery(new Term("name", "wolf")), Occur.MUST);
> > query.add(skuJoinQuery, Occur.MUST);
> >
> > ToParentBlockJoinCollector c = new ToParentBlockJoinCollector(
> > Sort.RELEVANCE, // sort
> > 10, // numHits
> > true, // trackScores
> > false // trackMaxScore
> > );
> > searcher.search(query, c);
> >
> > Sort skuSort=new Sort(new SortField("size",Type.STRING));
> > TopGroups hits = c.getTopGroups(skuJoinQuery, skuSort, 0, //
> offset
> > 10, // maxDocsPerGroup
> > 0, // withinGroupOffset
> > true // fillSortFields
> > );
> > System.out.println("totalHitCount: " + hits.totalHitCount);
> > System.out.println("totalGroupedHitCount: " +
> > hits.totalGroupedHitCount);
> > for(GroupDocs group:hits.groups){
> > System.out.println("totalHits: "+group.totalHits);
> > for(ScoreDoc doc:group.scoreDocs){
> > System.out.println(doc.doc+", "+doc.score);
> > }
> > }
> > }
> >
> > }
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: ToParentBlockJoinQuery query loop finitely
Posted by Michael McCandless <lu...@mikemccandless.com>.
I think you're hitting the exception because you passed
trackScores=true to ToParentBlockJoinCollector. If you do that, the
ScoreMode cannot be None... I'll update the javadocs to make this
clear, and I'll also fix the exception message.
I think you're hitting the infinite loop because your parent document
is first, not last, in the group. Try putting the parent (shirt)
document last in each case instead...
Query-time join is already committed to trunk and 3.x, so it'll be in 3.6.0/4.0.
Mike McCandless
http://blog.mikemccandless.com
On Fri, Mar 23, 2012 at 12:27 AM, Li Li <fa...@gmail.com> wrote:
> hi all,
> I read these two articles
> http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html,
> http://blog.mikemccandless.com/2012/01/tochildblockjoinquery-in-lucene.htmland
> wrote a test program. But it seems there is some problem. it ends with
> endless loop.
> Here is my program, almost copy from that article, the only
> modification is from BlockJoinQuery to ToParentBlockJoinQuery because I
> can't find BlockJoinQuery in latest 4.0 dev trunk source tree.
> another modification is from SocreMode.none to ScoreMode.Avg because
> when using SocreMode.None, an exception throws:
> Exception in thread "main" java.lang.IllegalStateException: ScoreMode is
> None
> at
> org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.swapChildScores(ToParentBlockJoinQuery.java:270)
> at
> org.apache.lucene.search.join.ToParentBlockJoinCollector.copyGroups(ToParentBlockJoinCollector.java:251)
> So I change ScoreMode.None to Avg
> // BlockJoinQuery skuJoinQuery = new BlockJoinQuery(skuQuery, shirts,
> // ScoreMode.None);
> ToParentBlockJoinQuery skuJoinQuery = new
> ToParentBlockJoinQuery(skuQuery, shirts,
> ScoreMode.Avg);
>
>
> Then I run again, it ends up with endless loop in BooleanScorer2:
> public void score(Collector collector) throws IOException {
> collector.setScorer(this);
> while ((doc = countingSumScorer.nextDoc()) != NO_MORE_DOCS) {
> collector.collect(doc);
> }
> }
>
> current doc is -1, and countingSumScorer.nextDoc() always return -1!
> following is my program. btw, what's the status of query time join? what's
> the relationship of solr's join and lucene's join?
> -------------my codes-------------
> public class TestJoin {
>
> /**
> * @param args
> * @throws Exception
> */
> public static void main(String[] args) throws Exception {
> File f=new File("./testindex");
> if(!f.exists()){
> f.mkdirs();
> }
> Directory dir=FSDirectory.open(f);
> for(String name:dir.listAll()){
> dir.deleteFile(name);
> }
> IndexWriterConfig conf=new IndexWriterConfig(Version.LUCENE_40,new
> WhitespaceAnalyzer(Version.LUCENE_40));
> IndexWriter writer=new IndexWriter(dir, conf);
> FieldType fType=new FieldType();
> fType.setIndexed(true);
> fType.setStored(true);
> fType.setIndexOptions(IndexOptions.DOCS_AND_FREQS);
> fType.setTokenized(true);
> Document shirt=new Document();
> shirt.add(new Field("name","three wolf",fType));
> shirt.add(new StringField("type", "shirt"));
> Document sku1=new Document();
> sku1.add(new StringField("size","small"));
> sku1.add(new StringField("color","blue"));
> Document sku2=new Document();
> sku2.add(new StringField("size","small"));
> sku2.add(new StringField("color","black"));
> Document sku3=new Document();
> sku3.add(new StringField("size","medium"));
> sku3.add(new StringField("color","black"));
> Document sku4=new Document();
> sku4.add(new StringField("size","large"));
> sku4.add(new StringField("color","gray"));
> List<Document> shirtGroup=new ArrayList<Document>();
> shirtGroup.add(shirt);
> shirtGroup.add(sku1);
> shirtGroup.add(sku2);
> shirtGroup.add(sku3);
> shirtGroup.add(sku4);
> writer.addDocuments(shirtGroup);
>
> shirt=new Document();
> shirt.add(new Field("name","seven wolf",fType));
> shirt.add(new StringField("type", "shirt"));
> sku1=new Document();
> sku1.add(new StringField("size","large"));
> sku1.add(new StringField("color","blue"));
> sku2=new Document();
> sku2.add(new StringField("size","small"));
> sku2.add(new StringField("color","black"));
>
> shirtGroup=new ArrayList<Document>();
> shirtGroup.add(shirt);
> shirtGroup.add(sku1);
> shirtGroup.add(sku2);
> writer.addDocuments(shirtGroup);
> writer.close();
>
> IndexSearcher searcher=new IndexSearcher(DirectoryReader.open(dir));
> Filter shirts = new CachingWrapperFilter(new QueryWrapperFilter(
> new TermQuery(new Term("type", "shirt"))));
> BooleanQuery skuQuery = new BooleanQuery();
> skuQuery.add(new TermQuery(new Term("size", "small")), Occur.MUST);
> skuQuery.add(new TermQuery(new Term("color", "black")), Occur.MUST);
> // BlockJoinQuery skuJoinQuery = new BlockJoinQuery(skuQuery, shirts,
> // ScoreMode.None);
> ToParentBlockJoinQuery skuJoinQuery = new
> ToParentBlockJoinQuery(skuQuery, shirts,
> ScoreMode.Avg);
>
> BooleanQuery query = new BooleanQuery();
> query.add(new TermQuery(new Term("name", "wolf")), Occur.MUST);
> query.add(skuJoinQuery, Occur.MUST);
>
> ToParentBlockJoinCollector c = new ToParentBlockJoinCollector(
> Sort.RELEVANCE, // sort
> 10, // numHits
> true, // trackScores
> false // trackMaxScore
> );
> searcher.search(query, c);
>
> Sort skuSort=new Sort(new SortField("size",Type.STRING));
> TopGroups hits = c.getTopGroups(skuJoinQuery, skuSort, 0, // offset
> 10, // maxDocsPerGroup
> 0, // withinGroupOffset
> true // fillSortFields
> );
> System.out.println("totalHitCount: " + hits.totalHitCount);
> System.out.println("totalGroupedHitCount: " +
> hits.totalGroupedHitCount);
> for(GroupDocs group:hits.groups){
> System.out.println("totalHits: "+group.totalHits);
> for(ScoreDoc doc:group.scoreDocs){
> System.out.println(doc.doc+", "+doc.score);
> }
> }
> }
>
> }
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org