You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Deepthi P <vl...@gmail.com> on 2018/02/08 06:21:18 UTC
Design Question
I have a dictionary of 20000 ID's and their description which is in a
collection. There is another solr collection in which each document have 10
or more ID's(multi valued field). I would like to text search in the
dictionary and bring back the matched ID's and search these ID's in solr
collection. For example if I search for 'fox' in the dictionary collection
and it returns 40 ID's, I would like to sent these 40 ID's to solr
collection and bring back documents that have these ID's.Please suggest the
best way to implement this.
Re: Design Question
Posted by Emir Arnautović <em...@sematext.com>.
Hi Deepthi,
I never tried case with multivalue field.
One note about join - your dictionary index must be single shard index and must be replicated on all Solr nodes where your other index is.
Let us know how it worked out for you.
Thanks,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 8 Feb 2018, at 14:59, Deepthi P <vl...@gmail.com> wrote:
>
> Hi Emir,
>
> Thank you. The dictionary is static but the descriptions are long text. If
> i denormalize each document will have 100+ descriptions and I have 8
> million records in collection. There is lot of repetition of descriptions
> and the index becomes large. I am trying to avoid that. Also data load time
> adds to it as I have to iterate over each ID and add description.
>
> Will join work if ID is multivalued field in Solr collection and single
> value field in dictionary ? Please advise.
>
> Thanks
> Deepthi
>
> On Thu, Feb 8, 2018 at 2:55 AM, Emir Arnautović <
> emir.arnautovic@sematext.com> wrote:
>
>> Hi Deepthi,
>> Is dictionary static? Can value for some id change? If static, and if
>> query performance matters to you, the best and also the simplest solution
>> is to denormalise data and store dictionary values with docs.
>>
>> Alternative is to use join query parser: https://lucene.apache.org/
>> solr/guide/6_6/other-parsers.html#OtherParsers-JoiningAcrossCollections <
>> https://lucene.apache.org/solr/guide/6_6/other-parsers.html#OtherParsers-
>> JoiningAcrossCollections>
>>
>> HTH,
>> Emir
>> --
>> Monitoring - Log Management - Alerting - Anomaly Detection
>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>
>>
>>
>>> On 8 Feb 2018, at 07:21, Deepthi P <vl...@gmail.com> wrote:
>>>
>>> I have a dictionary of 20000 ID's and their description which is in a
>>> collection. There is another solr collection in which each document have
>> 10
>>> or more ID's(multi valued field). I would like to text search in the
>>> dictionary and bring back the matched ID's and search these ID's in solr
>>> collection. For example if I search for 'fox' in the dictionary
>> collection
>>> and it returns 40 ID's, I would like to sent these 40 ID's to solr
>>> collection and bring back documents that have these ID's.Please suggest
>> the
>>> best way to implement this.
>>
>>
Re: Design Question
Posted by Deepthi P <vl...@gmail.com>.
Hi Emir,
Thank you. The dictionary is static but the descriptions are long text. If
i denormalize each document will have 100+ descriptions and I have 8
million records in collection. There is lot of repetition of descriptions
and the index becomes large. I am trying to avoid that. Also data load time
adds to it as I have to iterate over each ID and add description.
Will join work if ID is multivalued field in Solr collection and single
value field in dictionary ? Please advise.
Thanks
Deepthi
On Thu, Feb 8, 2018 at 2:55 AM, Emir Arnautović <
emir.arnautovic@sematext.com> wrote:
> Hi Deepthi,
> Is dictionary static? Can value for some id change? If static, and if
> query performance matters to you, the best and also the simplest solution
> is to denormalise data and store dictionary values with docs.
>
> Alternative is to use join query parser: https://lucene.apache.org/
> solr/guide/6_6/other-parsers.html#OtherParsers-JoiningAcrossCollections <
> https://lucene.apache.org/solr/guide/6_6/other-parsers.html#OtherParsers-
> JoiningAcrossCollections>
>
> HTH,
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
> > On 8 Feb 2018, at 07:21, Deepthi P <vl...@gmail.com> wrote:
> >
> > I have a dictionary of 20000 ID's and their description which is in a
> > collection. There is another solr collection in which each document have
> 10
> > or more ID's(multi valued field). I would like to text search in the
> > dictionary and bring back the matched ID's and search these ID's in solr
> > collection. For example if I search for 'fox' in the dictionary
> collection
> > and it returns 40 ID's, I would like to sent these 40 ID's to solr
> > collection and bring back documents that have these ID's.Please suggest
> the
> > best way to implement this.
>
>
Re: Design Question
Posted by Emir Arnautović <em...@sematext.com>.
Hi Deepthi,
Is dictionary static? Can value for some id change? If static, and if query performance matters to you, the best and also the simplest solution is to denormalise data and store dictionary values with docs.
Alternative is to use join query parser: https://lucene.apache.org/solr/guide/6_6/other-parsers.html#OtherParsers-JoiningAcrossCollections <https://lucene.apache.org/solr/guide/6_6/other-parsers.html#OtherParsers-JoiningAcrossCollections>
HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 8 Feb 2018, at 07:21, Deepthi P <vl...@gmail.com> wrote:
>
> I have a dictionary of 20000 ID's and their description which is in a
> collection. There is another solr collection in which each document have 10
> or more ID's(multi valued field). I would like to text search in the
> dictionary and bring back the matched ID's and search these ID's in solr
> collection. For example if I search for 'fox' in the dictionary collection
> and it returns 40 ID's, I would like to sent these 40 ID's to solr
> collection and bring back documents that have these ID's.Please suggest the
> best way to implement this.
Fwd: Design Question
Posted by Deepthi P <vl...@gmail.com>.
I have a dictionary of 20000 ID's and their description which is in a
collection. There is another solr collection in which each document have 10
or more ID's(multi valued field). I would like to text search in the
dictionary and bring back the matched ID's and search these ID's in solr
collection. For example if I search for 'fox' in the dictionary collection
and it returns 40 ID's, I would like to sent these 40 ID's to solr
collection and bring back documents that have these ID's.Please suggest the
best way to implement this.