You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Robert Yu <RO...@morningstar.com> on 2012/06/27 13:27:08 UTC
how Solr/Lucene can support standard join operation
The ability of join operation supported as what http://wiki.apache.org/solr/Join says is so limited.
I'm thinking how to support standard join operation in Solr/Lucene because not all can be de-normalized efficiently.
Take 2 schemas below as an example:
(1) Student
sid
name
cid // class id
(2) class
cid
name
major
In SQL, it will be easy to get all students' name and its class name where student's name start with 'p' and class's major is "CS".
Select s.name, c.name from student s, class c where s.name like 'p%' and c.major = "CS".
How Solr/Lucene support the above query? It seems they do not.
Thanks,
________________________________
Robert Yu
Application Service - Backend
Morningstar Shenzhen Ltd.
Morningstar. Illuminating investing worldwide.
+86 755 3311-0223 voice
+86 137-2377-0925 mobile
+86 755 0000-0000 fax
robert.yu@morningstar.com<ma...@morningstar.com>
8FL, Tower A, Donghai International Center ( or East Pacific International Center)
7888 Shennan Road, Futian district,
Shenzhen, Guangdong province, China 518040
http://cn.morningstar.com<http://cn.morningstar.com/>
This e-mail contains privileged and confidential information and is intended only for the use of the person(s) named above. Any dissemination, distribution, or duplication of this communication without prior written consent from Morningstar is strictly prohibited. If you have received this message in error, please contact the sender immediately and delete the materials from any computer.
RE: how Solr/Lucene can support standard join operation
Posted by Robert Yu <RO...@morningstar.com>.
The point is sometimes data after de-normalization will be huge, in some case, it's even impossible.
Thanks,
-----Original Message-----
From: Lee Carroll [mailto:lee.a.carroll@googlemail.com]
Sent: Wednesday, June 27, 2012 7:38 PM
To: solr-user@lucene.apache.org
Subject: Re: how Solr/Lucene can support standard join operation
In your example de-normalising would be fine in a vast number of use-cases. multi value fields are fine.
If you really want to, see http://wiki.apache.org/solr/Join but make sure you loose the default relational dba world view first and only go down that route if you need to.
On 27 June 2012 12:27, Robert Yu <RO...@morningstar.com> wrote:
> The ability of join operation supported as what http://wiki.apache.org/solr/Join says is so limited.
> I'm thinking how to support standard join operation in Solr/Lucene because not all can be de-normalized efficiently.
>
> Take 2 schemas below as an example:
>
> (1) Student
> sid
> name
> cid // class id
>
> (2) class
>
> cid
>
> name
>
> major
> In SQL, it will be easy to get all students' name and its class name where student's name start with 'p' and class's major is "CS".
> Select s.name, c.name from student s, class c where s.name like 'p%' and c.major = "CS".
>
> How Solr/Lucene support the above query? It seems they do not.
>
> Thanks,
> ________________________________
> Robert Yu
> Application Service - Backend
> Morningstar Shenzhen Ltd.
> Morningstar. Illuminating investing worldwide.
>
> +86 755 3311-0223 voice
> +86 137-2377-0925 mobile
> +86 755 0000-0000 fax
> robert.yu@morningstar.com<ma...@morningstar.com>
> 8FL, Tower A, Donghai International Center ( or East Pacific
> International Center)
> 7888 Shennan Road, Futian district,
> Shenzhen, Guangdong province, China 518040
>
> http://cn.morningstar.com<http://cn.morningstar.com/>
>
> This e-mail contains privileged and confidential information and is intended only for the use of the person(s) named above. Any dissemination, distribution, or duplication of this communication without prior written consent from Morningstar is strictly prohibited. If you have received this message in error, please contact the sender immediately and delete the materials from any computer.
>
Re: how Solr/Lucene can support standard join operation
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
FYI,
If denormalization doesn't work for you, check index time join
http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html.
here is the scratch for query and index time support:
https://issues.apache.org/jira/browse/SOLR-3076
https://issues.apache.org/jira/browse/SOLR-3535
On Wed, Jun 27, 2012 at 3:47 PM, Lee Carroll
<le...@googlemail.com>wrote:
> Sorry you have that link! and I did not see the question - apols
>
> index schema could look something like:
>
> id
> name
> classList -> multi value
> majorClassList -> multi value
>
> a standard query would do the equivalent of your sql
>
> again apols for not seeing the link
>
> lee c
>
>
>
> On 27 June 2012 12:37, Lee Carroll <le...@googlemail.com> wrote:
> > In your example de-normalising would be fine in a vast number of
> > use-cases. multi value fields are fine.
> >
> > If you really want to, see http://wiki.apache.org/solr/Join but make
> > sure you loose the default relational dba world view first
> > and only go down that route if you need to.
> >
> >
> >
> > On 27 June 2012 12:27, Robert Yu <RO...@morningstar.com> wrote:
> >> The ability of join operation supported as what
> http://wiki.apache.org/solr/Join says is so limited.
> >> I'm thinking how to support standard join operation in Solr/Lucene
> because not all can be de-normalized efficiently.
> >>
> >> Take 2 schemas below as an example:
> >>
> >> (1) Student
> >> sid
> >> name
> >> cid // class id
> >>
> >> (2) class
> >>
> >> cid
> >>
> >> name
> >>
> >> major
> >> In SQL, it will be easy to get all students' name and its class name
> where student's name start with 'p' and class's major is "CS".
> >> Select s.name, c.name from student s, class c where s.namelike 'p%' and c.major = "CS".
> >>
> >> How Solr/Lucene support the above query? It seems they do not.
> >>
> >> Thanks,
> >> ________________________________
> >> Robert Yu
> >> Application Service - Backend
> >> Morningstar Shenzhen Ltd.
> >> Morningstar. Illuminating investing worldwide.
> >>
> >> +86 755 3311-0223 voice
> >> +86 137-2377-0925 mobile
> >> +86 755 0000-0000 fax
> >> robert.yu@morningstar.com<ma...@morningstar.com>
> >> 8FL, Tower A, Donghai International Center ( or East Pacific
> International Center)
> >> 7888 Shennan Road, Futian district,
> >> Shenzhen, Guangdong province, China 518040
> >>
> >> http://cn.morningstar.com<http://cn.morningstar.com/>
> >>
> >> This e-mail contains privileged and confidential information and is
> intended only for the use of the person(s) named above. Any dissemination,
> distribution, or duplication of this communication without prior written
> consent from Morningstar is strictly prohibited. If you have received this
> message in error, please contact the sender immediately and delete the
> materials from any computer.
> >>
>
--
Sincerely yours
Mikhail Khludnev
Tech Lead
Grid Dynamics
<http://www.griddynamics.com>
<mk...@griddynamics.com>
Re: how Solr/Lucene can support standard join operation
Posted by Lee Carroll <le...@googlemail.com>.
Sorry you have that link! and I did not see the question - apols
index schema could look something like:
id
name
classList -> multi value
majorClassList -> multi value
a standard query would do the equivalent of your sql
again apols for not seeing the link
lee c
On 27 June 2012 12:37, Lee Carroll <le...@googlemail.com> wrote:
> In your example de-normalising would be fine in a vast number of
> use-cases. multi value fields are fine.
>
> If you really want to, see http://wiki.apache.org/solr/Join but make
> sure you loose the default relational dba world view first
> and only go down that route if you need to.
>
>
>
> On 27 June 2012 12:27, Robert Yu <RO...@morningstar.com> wrote:
>> The ability of join operation supported as what http://wiki.apache.org/solr/Join says is so limited.
>> I'm thinking how to support standard join operation in Solr/Lucene because not all can be de-normalized efficiently.
>>
>> Take 2 schemas below as an example:
>>
>> (1) Student
>> sid
>> name
>> cid // class id
>>
>> (2) class
>>
>> cid
>>
>> name
>>
>> major
>> In SQL, it will be easy to get all students' name and its class name where student's name start with 'p' and class's major is "CS".
>> Select s.name, c.name from student s, class c where s.name like 'p%' and c.major = "CS".
>>
>> How Solr/Lucene support the above query? It seems they do not.
>>
>> Thanks,
>> ________________________________
>> Robert Yu
>> Application Service - Backend
>> Morningstar Shenzhen Ltd.
>> Morningstar. Illuminating investing worldwide.
>>
>> +86 755 3311-0223 voice
>> +86 137-2377-0925 mobile
>> +86 755 0000-0000 fax
>> robert.yu@morningstar.com<ma...@morningstar.com>
>> 8FL, Tower A, Donghai International Center ( or East Pacific International Center)
>> 7888 Shennan Road, Futian district,
>> Shenzhen, Guangdong province, China 518040
>>
>> http://cn.morningstar.com<http://cn.morningstar.com/>
>>
>> This e-mail contains privileged and confidential information and is intended only for the use of the person(s) named above. Any dissemination, distribution, or duplication of this communication without prior written consent from Morningstar is strictly prohibited. If you have received this message in error, please contact the sender immediately and delete the materials from any computer.
>>
Re: how Solr/Lucene can support standard join operation
Posted by Lee Carroll <le...@googlemail.com>.
In your example de-normalising would be fine in a vast number of
use-cases. multi value fields are fine.
If you really want to, see http://wiki.apache.org/solr/Join but make
sure you loose the default relational dba world view first
and only go down that route if you need to.
On 27 June 2012 12:27, Robert Yu <RO...@morningstar.com> wrote:
> The ability of join operation supported as what http://wiki.apache.org/solr/Join says is so limited.
> I'm thinking how to support standard join operation in Solr/Lucene because not all can be de-normalized efficiently.
>
> Take 2 schemas below as an example:
>
> (1) Student
> sid
> name
> cid // class id
>
> (2) class
>
> cid
>
> name
>
> major
> In SQL, it will be easy to get all students' name and its class name where student's name start with 'p' and class's major is "CS".
> Select s.name, c.name from student s, class c where s.name like 'p%' and c.major = "CS".
>
> How Solr/Lucene support the above query? It seems they do not.
>
> Thanks,
> ________________________________
> Robert Yu
> Application Service - Backend
> Morningstar Shenzhen Ltd.
> Morningstar. Illuminating investing worldwide.
>
> +86 755 3311-0223 voice
> +86 137-2377-0925 mobile
> +86 755 0000-0000 fax
> robert.yu@morningstar.com<ma...@morningstar.com>
> 8FL, Tower A, Donghai International Center ( or East Pacific International Center)
> 7888 Shennan Road, Futian district,
> Shenzhen, Guangdong province, China 518040
>
> http://cn.morningstar.com<http://cn.morningstar.com/>
>
> This e-mail contains privileged and confidential information and is intended only for the use of the person(s) named above. Any dissemination, distribution, or duplication of this communication without prior written consent from Morningstar is strictly prohibited. If you have received this message in error, please contact the sender immediately and delete the materials from any computer.
>