You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Shamik Bandopadhyay <sh...@gmail.com> on 2016/09/22 17:16:14 UTC
How to retrieve parent documents without a nested structure (block-join)
Hi,
I have a set of documents indexed which has a pseudo parent-child
relationship. Each child document has a reference to the parent document
through an ID. As the documents are not available to the crawler in order,
I'm not able to index them in a nested structure to support
block-join.Here's an example of a dataset in index right now.
<doc>
<field name="id">1</field>
<field name="title">Parent title</field>
<field name="doc_id">123</field>
</doc>
<doc>
<field name="id">2</field>
<field name="title">Child title1</field>
<field name="parent_doc_id">123</field>
</doc>
<doc>
<field name="id">3</field>
<field name="title">Child title2</field>
<field name="parent_doc_id">123</field>
</doc>
<doc>
<field name="id">4</field>
<field name="title">Misc title2</field>
</doc>
As per my requirement, if I search on "title2", the result should bring
back the following result, the parent document (id=1) and non-related
document (id=4).
<doc>
<field name="id">1</field>
<field name="title">Parent title</field>
<field name="doc_id">123</field>
</doc>
<doc>
<field name="id">4</field>
<field name="title">Misc title2</field>
</doc>
This is similar in lines with Block Join Parent Query Parser where I could
have fired a query like : q={!parent
which="content_type:parentDocument"}title:title2
Not sure if the Graph Query Parser can be a relevant solution in this
regard. The problem I see there is I'm running on 5.5 with 2 shard and n
number of replicas. The graph query parser seems to be designed for a
single node/single shard.
This is tad urgent for me as I'm trying to come up with an approach to deal
with this. Any pointers will be highly appreciated.
Thanks,
Shamik
Re: How to retrieve parent documents without a nested structure
(block-join)
Posted by shamik <sh...@gmail.com>.
Thanks again Alex.
I should have clarified the use of browse request handler. The reason I'm
simulating the request handler parameters of my production system using
browse. I used a separate request handler, stripped down all properties to
match "select". I finally narrowed down the issue to Minimum match (mm)
parameter. I had specified it as "mm=100%". That was preventing the query
from returning any results.
Once again, appreciate all your help.
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-retrieve-parent-documents-without-a-nested-structure-block-join-tp4297510p4298304.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to retrieve parent documents without a nested structure (block-join)
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
I am not sure why you are even trying this against /browse, but the
easy way to compare handlers is to add echoParams=all to both requests
and compare all parameters side-by-side.
Regards,
Alex.
----
Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/
On 27 September 2016 at 20:52, shamik <sh...@gmail.com> wrote:
> Sorry to bump this up, but can someone please explain the parsing behaviour
> of a join query (show above) in respect to different request handler ?
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-retrieve-parent-documents-without-a-nested-structure-block-join-tp4297510p4298249.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to retrieve parent documents without a nested structure
(block-join)
Posted by shamik <sh...@gmail.com>.
Sorry to bump this up, but can someone please explain the parsing behaviour
of a join query (show above) in respect to different request handler ?
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-retrieve-parent-documents-without-a-nested-structure-block-join-tp4297510p4298249.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to retrieve parent documents without a nested structure
(block-join)
Posted by shamik <sh...@gmail.com>.
Thanks Alex, this has been extremely helpful. There's one doubt though.
The query returns expected result if I use "select" or "query" request
handler, but fails for others. Here's the debug output from "/select" using
edismax.
http://localhost:8983/solr/techproducts/query?q=({!join%20from=manu_id_s%20to=id}ipod)(name:GB18030%20-manu_id_s:*)&fl=id,title&debug=query&wt=xml&hl=false&spellcheck=false&defType=edismax
*<str name="parsedquery">(+(JoinQuery({!join from=manu_id_s to=id}text:ipod)
(name:gb18030 -manu_id_s:*)))/no_coord</str>
<str name="parsedquery_toString">+({!join from=manu_id_s to=id}text:ipod
(name:gb18030 -manu_id_s:*))</str>
*
Now, if I use "/browse", I don't get any results back. Here's a snippet from
browse request handler config.
<requestHandler name="/browse" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<str name="wt">velocity</str>
<str name="v.template">browse</str>
<str name="v.layout">layout</str>
<str name="title">Solritas</str>
<str name="defType">edismax</str>
<str name="qf">
text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
title^10.0 description^5.0 keywords^5.0 author^2.0
resourcename^1.0 subject^0.5
</str>
<str name="mm">100%</str>
<str name="q.alt">*:*</str>
<str name="rows">10</str>
<str name="fl">*,score</str>
As you can see, I've defined the "qf" fields with defType as edismax.
Here's the query:
http://localhost:8983/solr/techproducts/browse?q=({!join%20from=manu_id_s%20to=id}ipod)(name:GB18030%20-manu_id_s:*)&debug=query&wt=xml&hl=false&spellcheck=false
Output:
*<str name="parsedquery">(+((JoinQuery({!join from=manu_id_s
to=id}text:ipod) (DisjunctionMaxQuery((keywords:name:gb18030^5.0 |
author:name:gb18030^2.0 | ((subject:name subject:gb subject:18030)~3)^0.5 |
manu:name:gb18030^1.1 | ((description:name description:gb
description:18030)~3)^5.0 | ((title:name title:gb title:18030)~3)^10.0 |
features:name:gb18030 | cat:name:GB18030^1.4 | name:name:gb18030^1.2 |
text:name:gb18030^0.5 | id:name:GB18030^10.0 | resourcename:name:gb18030 |
sku:"namegb 18030"^1.5)) -manu_id_s:*))~2))/no_coord</str>
<str name="parsedquery_toString">+(({!join from=manu_id_s to=id}text:ipod
((keywords:name:gb18030^5.0 | author:name:gb18030^2.0 | ((subject:name
subject:gb subject:18030)~3)^0.5 | manu:name:gb18030^1.1 |
((description:name description:gb description:18030)~3)^5.0 | ((title:name
title:gb title:18030)~3)^10.0 | features:name:gb18030 | cat:name:GB18030^1.4
| name:name:gb18030^1.2 | text:name:gb18030^0.5 | id:name:GB18030^10.0 |
resourcename:name:gb18030 | sku:"namegb 18030"^1.5) -manu_id_s:*))~2)</str>*
If I remove the join query condition ({!join from=manu_id_s to=id}ipod) ,
the query returns the result based on the second condition.
The other doubt I've is why "text" is getting picked as a default field in
the join condition? I've defined the "df" fields in "browse" which are being
used in the second condition. Do I need to explicitly set the df fields
inside the join condition?
The other thing I've noticed is the difference in parsed query if I add a
space in between the two clause. For e.g. *q=({!join from=manu_id_s
to=id}ipod) (name:GB18030 -manu_id_s:*)* results in
*<str name="parsedquery">(+((JoinQuery({!join from=manu_id_s
to=id}text:ipod) (name:gb18030 -manu_id_s:*))~2))/no_coord</str>
<str name="parsedquery_toString">+(({!join from=manu_id_s to=id}text:ipod
(name:gb18030 -manu_id_s:*))~2)</str>*
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-retrieve-parent-documents-without-a-nested-structure-block-join-tp4297510p4298115.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to retrieve parent documents without a nested structure (block-join)
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
This seems to work against the techproducts example in 6.2:
({!join from=manu_id_s to=id}ipod) (name:GB18030 -manu_id_s:*)
Two clauses, first one does join and parent mapping. The second one
looks at the records that don't have the mapping key at all and run
the match against that. In your case, the second query's keyword is
probably the same as in the first query.
The brackets are there to ensure the queries are split correctly. Use
debugFlag to confirm what happens.
Regards
Alex.
----
Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/
On 26 September 2016 at 10:02, shamik <sh...@gmail.com> wrote:
> Thanks for getting back on this. I was trying to formulate a query in similar
> lines but not able to construct it (multiple clauses) correctly so far. That
> can be attributed to my inexperience with Solr queries as well. Can you
> please point to any documentation / example for my reference ?
>
> Appreciate your help.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-retrieve-parent-documents-without-a-nested-structure-block-join-tp4297510p4297951.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to retrieve parent documents without a nested structure
(block-join)
Posted by shamik <sh...@gmail.com>.
Thanks for getting back on this. I was trying to formulate a query in similar
lines but not able to construct it (multiple clauses) correctly so far. That
can be attributed to my inexperience with Solr queries as well. Can you
please point to any documentation / example for my reference ?
Appreciate your help.
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-retrieve-parent-documents-without-a-nested-structure-block-join-tp4297510p4297951.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to retrieve parent documents without a nested structure (block-join)
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Can't you just have two clauses in the query, one that does join and one
that does the same search again but with additional rule of not having a
parentID? Then nested documents match first clause and get "joined" into
the parent record. And non nested documents would match by themselves. I
think you could even use local parameter variables to avoid repeating the
common query.
Regards,
Alex
On 26 Sep 2016 3:59 AM, "shamik" <sh...@gmail.com> wrote:
> Thanks Alex. With the conventional join query I'm able to return the parent
> document based on a query match on the child. But, it filters out any other
> documents which are outside the scope of join condition. For e.g. in my
> case, I would expect the query to return :
>
> <doc>
> <field name="id">1</field>
> <field name="title">Parent title</field>
> <field name="doc_id">123</field>
> </doc>
> <doc>
> <field name="id">4</field>
> <field name="title">Misc title2</field>
> </doc>
>
> I'm only getting back id=1 with the following join query :
>
> *{!join from=parent_doc_id to=doc_id}title2*
>
> Is there a way to get the document with id=4 ?
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/How-to-retrieve-parent-documents-without-a-
> nested-structure-block-join-tp4297510p4297935.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: How to retrieve parent documents without a nested structure
(block-join)
Posted by shamik <sh...@gmail.com>.
Thanks Alex. With the conventional join query I'm able to return the parent
document based on a query match on the child. But, it filters out any other
documents which are outside the scope of join condition. For e.g. in my
case, I would expect the query to return :
<doc>
<field name="id">1</field>
<field name="title">Parent title</field>
<field name="doc_id">123</field>
</doc>
<doc>
<field name="id">4</field>
<field name="title">Misc title2</field>
</doc>
I'm only getting back id=1 with the following join query :
*{!join from=parent_doc_id to=doc_id}title2*
Is there a way to get the document with id=4 ?
--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-retrieve-parent-documents-without-a-nested-structure-block-join-tp4297510p4297935.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to retrieve parent documents without a nested structure (block-join)
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
Why not a traditional join?
https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-JoinQueryParser
Regards,
Alex.
----
Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/
On 23 September 2016 at 00:16, Shamik Bandopadhyay <sh...@gmail.com> wrote:
> Hi,
>
> I have a set of documents indexed which has a pseudo parent-child
> relationship. Each child document has a reference to the parent document
> through an ID. As the documents are not available to the crawler in order,
> I'm not able to index them in a nested structure to support
> block-join.Here's an example of a dataset in index right now.
>
> <doc>
> <field name="id">1</field>
> <field name="title">Parent title</field>
> <field name="doc_id">123</field>
> </doc>
> <doc>
> <field name="id">2</field>
> <field name="title">Child title1</field>
> <field name="parent_doc_id">123</field>
> </doc>
> <doc>
> <field name="id">3</field>
> <field name="title">Child title2</field>
> <field name="parent_doc_id">123</field>
> </doc>
> <doc>
> <field name="id">4</field>
> <field name="title">Misc title2</field>
> </doc>
>
> As per my requirement, if I search on "title2", the result should bring
> back the following result, the parent document (id=1) and non-related
> document (id=4).
>
> <doc>
> <field name="id">1</field>
> <field name="title">Parent title</field>
> <field name="doc_id">123</field>
> </doc>
> <doc>
> <field name="id">4</field>
> <field name="title">Misc title2</field>
> </doc>
>
> This is similar in lines with Block Join Parent Query Parser where I could
> have fired a query like : q={!parent
> which="content_type:parentDocument"}title:title2
>
> Not sure if the Graph Query Parser can be a relevant solution in this
> regard. The problem I see there is I'm running on 5.5 with 2 shard and n
> number of replicas. The graph query parser seems to be designed for a
> single node/single shard.
>
> This is tad urgent for me as I'm trying to come up with an approach to deal
> with this. Any pointers will be highly appreciated.
>
> Thanks,
> Shamik