You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Lewin Joy (TMNA)" <Le...@toyota.com> on 2018/01/02 21:35:47 UTC
Solr Issue
** PROTECTED 関係者外秘
Hi,
I am using Solr 6.1 and am facing an issue with a complex scenario.
Could you help figure out how this can be achieved in Solr?
We have items: A, B, C . There will be multiple record entries for each items.
For our understanding, let’s say the fields for these records are: primary_key,item_name,status.
I need to retrieve all records with status= ‘N’ and filter out items which has any of it’s record matching status: ‘Y’
For record set below, the query should only return me records 1 and 2.
Primary_key
Item_Name
status
1
A
N
2
A
N
3
B
N
4
B
Y
5
B
N
6
C
Y
7
C
N
Currently, I am using Streaming Query expressions to do complement() operation.
But the number of records with status= ‘Y’ is too huge and causes performance Problems.
And secondly, streaming query exports with Joins and complements can’t be used properly for bringing out paginated outputs.
Is there anyway, we can group the results and do a query on the group to filter out such records?
Or any other approach which could give my results paginated?
Thanks,
Lewin
Re: Solr Issue
Posted by Erick Erickson <er...@gmail.com>.
Ok, I missed that you had multiple records.
Pagination does not mesh well with streaming, they address quite different
use cases. You might have to re-think your indexing scheme to somehow make
the selection more amenable to standard Solr queries. Perhaps something
with the join query parser or indexing parent/child docs.
Best,
Erick
On Fri, Jan 5, 2018 at 11:57 AM, Lewin Joy (TMNA) <Le...@toyota.com>
wrote:
> Hi Erick,
>
> I just didn't know how to handle the logic without using streaming query.
> What I want is to apply the logic without streaming expressions.
>
> I want to eliminate "item_name"s in the result set which has any of it's
> record's status='N'
>
> Example:
> Id:1, Item Name: A , status: Y
> Id:2, Item Name: A, status: Y
> Id:3, Item Name: B, status: Y
> Id:4, Item Name: B, status: N
> Id:5, Item Name: B, status: Y
>
> For this data, If I apply fq=status:Y -status:N, I would still get Ids:
> 1,2,3, and 5.
> I would like to eliminate B completely as it has one record with status:N
>
> Is this possible to do in Solr without using streaming expressions?
>
> -Lewin
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Tuesday, January 02, 2018 6:34 PM
> To: solr-user <so...@lucene.apache.org>
> Subject: Re: Solr Issue
>
> wait, what is it you want to do? Streaming is built expressly to handle
> very large result sets. It is _not_ really designed to deliver pages at a
> time. Why do you want to use it at all?
>
> Why not just use straight Solr, perhaps with cursorMark if you want to
> page deeply. The query is something like:
> q=status:‘N’ -status: ‘Y’
> or
> q=status:‘N’&fq:-status: ‘Y’
>
> Best,
> Erick
>
> On Tue, Jan 2, 2018 at 1:35 PM, Lewin Joy (TMNA) <Le...@toyota.com>
> wrote:
> > ** PROTECTED 関係者外秘
> > Hi,
> >
> > I am using Solr 6.1 and am facing an issue with a complex scenario.
> > Could you help figure out how this can be achieved in Solr?
> >
> > We have items: A, B, C . There will be multiple record entries for each
> items.
> > For our understanding, let’s say the fields for these records are:
> primary_key,item_name,status.
> >
> > I need to retrieve all records with status= ‘N’ and filter out items
> which has any of it’s record matching status: ‘Y’
> >
> > For record set below, the query should only return me records 1 and 2.
> > Primary_key
> >
> > Item_Name
> >
> > status
> >
> > 1
> >
> > A
> >
> > N
> >
> > 2
> >
> > A
> >
> > N
> >
> > 3
> >
> > B
> >
> > N
> >
> > 4
> >
> > B
> >
> > Y
> >
> > 5
> >
> > B
> >
> > N
> >
> > 6
> >
> > C
> >
> > Y
> >
> > 7
> >
> > C
> >
> > N
> >
> >
> >
> > Currently, I am using Streaming Query expressions to do complement()
> operation.
> > But the number of records with status= ‘Y’ is too huge and causes
> performance Problems.
> > And secondly, streaming query exports with Joins and complements can’t
> be used properly for bringing out paginated outputs.
> >
> > Is there anyway, we can group the results and do a query on the group to
> filter out such records?
> > Or any other approach which could give my results paginated?
> >
> > Thanks,
> > Lewin
>
RE: Solr Issue
Posted by "Lewin Joy (TMNA)" <Le...@toyota.com>.
Hi Erick,
I just didn't know how to handle the logic without using streaming query.
What I want is to apply the logic without streaming expressions.
I want to eliminate "item_name"s in the result set which has any of it's record's status='N'
Example:
Id:1, Item Name: A , status: Y
Id:2, Item Name: A, status: Y
Id:3, Item Name: B, status: Y
Id:4, Item Name: B, status: N
Id:5, Item Name: B, status: Y
For this data, If I apply fq=status:Y -status:N, I would still get Ids: 1,2,3, and 5.
I would like to eliminate B completely as it has one record with status:N
Is this possible to do in Solr without using streaming expressions?
-Lewin
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Tuesday, January 02, 2018 6:34 PM
To: solr-user <so...@lucene.apache.org>
Subject: Re: Solr Issue
wait, what is it you want to do? Streaming is built expressly to handle very large result sets. It is _not_ really designed to deliver pages at a time. Why do you want to use it at all?
Why not just use straight Solr, perhaps with cursorMark if you want to page deeply. The query is something like:
q=status:‘N’ -status: ‘Y’
or
q=status:‘N’&fq:-status: ‘Y’
Best,
Erick
On Tue, Jan 2, 2018 at 1:35 PM, Lewin Joy (TMNA) <Le...@toyota.com> wrote:
> ** PROTECTED 関係者外秘
> Hi,
>
> I am using Solr 6.1 and am facing an issue with a complex scenario.
> Could you help figure out how this can be achieved in Solr?
>
> We have items: A, B, C . There will be multiple record entries for each items.
> For our understanding, let’s say the fields for these records are: primary_key,item_name,status.
>
> I need to retrieve all records with status= ‘N’ and filter out items which has any of it’s record matching status: ‘Y’
>
> For record set below, the query should only return me records 1 and 2.
> Primary_key
>
> Item_Name
>
> status
>
> 1
>
> A
>
> N
>
> 2
>
> A
>
> N
>
> 3
>
> B
>
> N
>
> 4
>
> B
>
> Y
>
> 5
>
> B
>
> N
>
> 6
>
> C
>
> Y
>
> 7
>
> C
>
> N
>
>
>
> Currently, I am using Streaming Query expressions to do complement() operation.
> But the number of records with status= ‘Y’ is too huge and causes performance Problems.
> And secondly, streaming query exports with Joins and complements can’t be used properly for bringing out paginated outputs.
>
> Is there anyway, we can group the results and do a query on the group to filter out such records?
> Or any other approach which could give my results paginated?
>
> Thanks,
> Lewin
Re: Solr Issue
Posted by Erick Erickson <er...@gmail.com>.
wait, what is it you want to do? Streaming is built expressly to
handle very large result sets. It is _not_ really designed to deliver
pages at a time. Why do you want to use it at all?
Why not just use straight Solr, perhaps with cursorMark if you want to
page deeply. The query is something like:
q=status:‘N’ -status: ‘Y’
or
q=status:‘N’&fq:-status: ‘Y’
Best,
Erick
On Tue, Jan 2, 2018 at 1:35 PM, Lewin Joy (TMNA) <Le...@toyota.com> wrote:
> ** PROTECTED 関係者外秘
> Hi,
>
> I am using Solr 6.1 and am facing an issue with a complex scenario.
> Could you help figure out how this can be achieved in Solr?
>
> We have items: A, B, C . There will be multiple record entries for each items.
> For our understanding, let’s say the fields for these records are: primary_key,item_name,status.
>
> I need to retrieve all records with status= ‘N’ and filter out items which has any of it’s record matching status: ‘Y’
>
> For record set below, the query should only return me records 1 and 2.
> Primary_key
>
> Item_Name
>
> status
>
> 1
>
> A
>
> N
>
> 2
>
> A
>
> N
>
> 3
>
> B
>
> N
>
> 4
>
> B
>
> Y
>
> 5
>
> B
>
> N
>
> 6
>
> C
>
> Y
>
> 7
>
> C
>
> N
>
>
>
> Currently, I am using Streaming Query expressions to do complement() operation.
> But the number of records with status= ‘Y’ is too huge and causes performance Problems.
> And secondly, streaming query exports with Joins and complements can’t be used properly for bringing out paginated outputs.
>
> Is there anyway, we can group the results and do a query on the group to filter out such records?
> Or any other approach which could give my results paginated?
>
> Thanks,
> Lewin
Re: Solr Issue
Posted by Rick Leir <rl...@leirtech.com>.
Lewin
Is this not a job for a database like MySQL? Solr is a search engine, which can be used as a DB with some effort. Choose the right tool for the job . Cheers -- Rick
On January 2, 2018 4:35:47 PM EST, "Lewin Joy (TMNA)" <Le...@toyota.com> wrote:
>** PROTECTED 関係者外秘
>Hi,
>
>I am using Solr 6.1 and am facing an issue with a complex scenario.
>Could you help figure out how this can be achieved in Solr?
>
>We have items: A, B, C . There will be multiple record entries for
>each items.
>For our understanding, let’s say the fields for these records are:
>primary_key,item_name,status.
>
>I need to retrieve all records with status= ‘N’ and filter out items
>which has any of it’s record matching status: ‘Y’
>
>For record set below, the query should only return me records 1 and 2.
>Primary_key
>
>Item_Name
>
>status
>
>1
>
>A
>
>N
>
>2
>
>A
>
>N
>
>3
>
>B
>
>N
>
>4
>
>B
>
>Y
>
>5
>
>B
>
>N
>
>6
>
>C
>
>Y
>
>7
>
>C
>
>N
>
>
>
>Currently, I am using Streaming Query expressions to do complement()
>operation.
>But the number of records with status= ‘Y’ is too huge and causes
>performance Problems.
>And secondly, streaming query exports with Joins and complements can’t
>be used properly for bringing out paginated outputs.
>
>Is there anyway, we can group the results and do a query on the group
>to filter out such records?
>Or any other approach which could give my results paginated?
>
>Thanks,
>Lewin
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com