You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by caddmngr <cg...@gmail.com> on 2012/08/08 00:59:03 UTC
exclusions by query and many values
we have Solr docs for manufacturer parts, where each part is available from
100's of suppliers. those suppliers are stored within each Mfg part document
in a multi-field.
customers searching our parts by keyword against part titles and
descriptions, each have a unique list of what suppliers they are not allowed
to view. For example:
solr docs:
doc_id = 1, mfg_part = abc, suppliers = s1, s2, s3, s4, s5, s6, s7
doc_id = 2, mfg_part = def, suppliers = s4, s5, s6, s7
doc_id = 3, mfg_part = ghi, suppliers = s4
doc_id = 4, mfg_part = jkl, suppliers = s1, s2, s3, s4
doc_id = 5, mfg_part = mno, suppliers = s1, s2, s3, s5
customer A: exclude suppliers: s4, s5, s6, s7
customer B: exclude suppliers: s1, s2, s3, s4
when customer A searches, documents 2 & 3 should not be returned in any
result set
when customer B searches, documents 3 & 4 should not be returned in any
result set
one thought we have is to restructure our docs so that there is one doc per
supplier mfg part, instead of per mfg part, but the result would be an index
of 3000 times the size!!! Many of our mfg parts have 1000 or more suppliers.
Currently we shove exclude lists into the filter query, but its getting to
be quite
we haved looked at Solr "join" in Solr 4, but being this is a production
site generating millions of dollars per week, we cannot afford to use alpha
or beta versions of software.
--
View this message in context: http://lucene.472066.n3.nabble.com/exclusions-by-query-and-many-values-tp3999672.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: exclusions by query and many values
Posted by Jack Krupansky <ja...@basetechnology.com>.
We'll have to see if anybody else has a better idea.
-- Jack Krupansky
-----Original Message-----
From: caddmngr
Sent: Thursday, August 09, 2012 3:49 PM
To: solr-user@lucene.apache.org
Subject: Re: exclusions by query and many values
Thanks for the response, Jack...but as I mentioned, we are currently doing
pretty much what you suggest. When customers login, we pull their list of
exceptions and create the filter query to use on all queries within their
session.
This works good, but as I also mentioned, its getting hard to manage and
the query is awfully long. I'm expecting it to start hurting performance,
so I was hoping there may be another technique or method to do this aside
from filter queries.
Chris
On Aug 7, 2012 8:55 PM, "Jack Krupansky-2 [via Lucene]" <
ml-node+s472066n3999683h91@n3.nabble.com> wrote:
> The usual technique is via filter queries that constrain what documents a
> user query can "see", either by OR-ing the doc classes it can see or
> starting with "*:*" and NOT-ing the doc classes it can't see, or a
> combination of the two techniques.
>
> The filter queries could either be supplied as query request parameters or
> added to the base query by a custom search component. For example, if you
> have some authorization system that you need to communicate with to
> determine the access authorization for a given user.
>
> -- Jack Krupansky
>
> -----Original Message-----
> From: caddmngr
> Sent: Tuesday, August 07, 2012 6:59 PM
> To: [hidden email] <http://user/SendEmail.jtp?type=node&node=3999683&i=0>
> Subject: exclusions by query and many values
>
> we have Solr docs for manufacturer parts, where each part is available
> from
> 100's of suppliers. those suppliers are stored within each Mfg part
> document
> in a multi-field.
>
> customers searching our parts by keyword against part titles and
> descriptions, each have a unique list of what suppliers they are not
> allowed
> to view. For example:
>
> solr docs:
> doc_id = 1, mfg_part = abc, suppliers = s1, s2, s3, s4, s5, s6, s7
> doc_id = 2, mfg_part = def, suppliers = s4, s5, s6, s7
> doc_id = 3, mfg_part = ghi, suppliers = s4
> doc_id = 4, mfg_part = jkl, suppliers = s1, s2, s3, s4
> doc_id = 5, mfg_part = mno, suppliers = s1, s2, s3, s5
>
> customer A: exclude suppliers: s4, s5, s6, s7
> customer B: exclude suppliers: s1, s2, s3, s4
>
> when customer A searches, documents 2 & 3 should not be returned in any
> result set
> when customer B searches, documents 3 & 4 should not be returned in any
> result set
>
> one thought we have is to restructure our docs so that there is one doc
> per
> supplier mfg part, instead of per mfg part, but the result would be an
> index
> of 3000 times the size!!! Many of our mfg parts have 1000 or more
> suppliers.
> Currently we shove exclude lists into the filter query, but its getting to
> be quite
>
> we haved looked at Solr "join" in Solr 4, but being this is a production
> site generating millions of dollars per week, we cannot afford to use
> alpha
> or beta versions of software.
>
>
>
>
>
>
> --
> View this message in context:
>
> http://lucene.472066.n3.nabble.com/exclusions-by-query-and-many-values-tp3999672.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/exclusions-by-query-and-many-values-tp3999672p3999683.html
> To unsubscribe from exclusions by query and many values, click
> here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3999672&code=Y2d1d2ViQGdtYWlsLmNvbXwzOTk5NjcyfDIzMTk2NTU0NA==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context:
http://lucene.472066.n3.nabble.com/exclusions-by-query-and-many-values-tp3999672p4000277.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: exclusions by query and many values
Posted by caddmngr <cg...@gmail.com>.
Thanks for the response, Jack...but as I mentioned, we are currently doing
pretty much what you suggest. When customers login, we pull their list of
exceptions and create the filter query to use on all queries within their
session.
This works good, but as I also mentioned, its getting hard to manage and
the query is awfully long. I'm expecting it to start hurting performance,
so I was hoping there may be another technique or method to do this aside
from filter queries.
Chris
On Aug 7, 2012 8:55 PM, "Jack Krupansky-2 [via Lucene]" <
ml-node+s472066n3999683h91@n3.nabble.com> wrote:
> The usual technique is via filter queries that constrain what documents a
> user query can "see", either by OR-ing the doc classes it can see or
> starting with "*:*" and NOT-ing the doc classes it can't see, or a
> combination of the two techniques.
>
> The filter queries could either be supplied as query request parameters or
> added to the base query by a custom search component. For example, if you
> have some authorization system that you need to communicate with to
> determine the access authorization for a given user.
>
> -- Jack Krupansky
>
> -----Original Message-----
> From: caddmngr
> Sent: Tuesday, August 07, 2012 6:59 PM
> To: [hidden email] <http://user/SendEmail.jtp?type=node&node=3999683&i=0>
> Subject: exclusions by query and many values
>
> we have Solr docs for manufacturer parts, where each part is available
> from
> 100's of suppliers. those suppliers are stored within each Mfg part
> document
> in a multi-field.
>
> customers searching our parts by keyword against part titles and
> descriptions, each have a unique list of what suppliers they are not
> allowed
> to view. For example:
>
> solr docs:
> doc_id = 1, mfg_part = abc, suppliers = s1, s2, s3, s4, s5, s6, s7
> doc_id = 2, mfg_part = def, suppliers = s4, s5, s6, s7
> doc_id = 3, mfg_part = ghi, suppliers = s4
> doc_id = 4, mfg_part = jkl, suppliers = s1, s2, s3, s4
> doc_id = 5, mfg_part = mno, suppliers = s1, s2, s3, s5
>
> customer A: exclude suppliers: s4, s5, s6, s7
> customer B: exclude suppliers: s1, s2, s3, s4
>
> when customer A searches, documents 2 & 3 should not be returned in any
> result set
> when customer B searches, documents 3 & 4 should not be returned in any
> result set
>
> one thought we have is to restructure our docs so that there is one doc
> per
> supplier mfg part, instead of per mfg part, but the result would be an
> index
> of 3000 times the size!!! Many of our mfg parts have 1000 or more
> suppliers.
> Currently we shove exclude lists into the filter query, but its getting to
> be quite
>
> we haved looked at Solr "join" in Solr 4, but being this is a production
> site generating millions of dollars per week, we cannot afford to use
> alpha
> or beta versions of software.
>
>
>
>
>
>
> --
> View this message in context:
>
> http://lucene.472066.n3.nabble.com/exclusions-by-query-and-many-values-tp3999672.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/exclusions-by-query-and-many-values-tp3999672p3999683.html
> To unsubscribe from exclusions by query and many values, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3999672&code=Y2d1d2ViQGdtYWlsLmNvbXwzOTk5NjcyfDIzMTk2NTU0NA==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/exclusions-by-query-and-many-values-tp3999672p4000277.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: exclusions by query and many values
Posted by Jack Krupansky <ja...@basetechnology.com>.
The usual technique is via filter queries that constrain what documents a
user query can "see", either by OR-ing the doc classes it can see or
starting with "*:*" and NOT-ing the doc classes it can't see, or a
combination of the two techniques.
The filter queries could either be supplied as query request parameters or
added to the base query by a custom search component. For example, if you
have some authorization system that you need to communicate with to
determine the access authorization for a given user.
-- Jack Krupansky
-----Original Message-----
From: caddmngr
Sent: Tuesday, August 07, 2012 6:59 PM
To: solr-user@lucene.apache.org
Subject: exclusions by query and many values
we have Solr docs for manufacturer parts, where each part is available from
100's of suppliers. those suppliers are stored within each Mfg part document
in a multi-field.
customers searching our parts by keyword against part titles and
descriptions, each have a unique list of what suppliers they are not allowed
to view. For example:
solr docs:
doc_id = 1, mfg_part = abc, suppliers = s1, s2, s3, s4, s5, s6, s7
doc_id = 2, mfg_part = def, suppliers = s4, s5, s6, s7
doc_id = 3, mfg_part = ghi, suppliers = s4
doc_id = 4, mfg_part = jkl, suppliers = s1, s2, s3, s4
doc_id = 5, mfg_part = mno, suppliers = s1, s2, s3, s5
customer A: exclude suppliers: s4, s5, s6, s7
customer B: exclude suppliers: s1, s2, s3, s4
when customer A searches, documents 2 & 3 should not be returned in any
result set
when customer B searches, documents 3 & 4 should not be returned in any
result set
one thought we have is to restructure our docs so that there is one doc per
supplier mfg part, instead of per mfg part, but the result would be an index
of 3000 times the size!!! Many of our mfg parts have 1000 or more suppliers.
Currently we shove exclude lists into the filter query, but its getting to
be quite
we haved looked at Solr "join" in Solr 4, but being this is a production
site generating millions of dollars per week, we cannot afford to use alpha
or beta versions of software.
--
View this message in context:
http://lucene.472066.n3.nabble.com/exclusions-by-query-and-many-values-tp3999672.html
Sent from the Solr - User mailing list archive at Nabble.com.