You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by gdeconto <ge...@topproducer.com> on 2009/09/03 19:05:33 UTC
how to scan dynamic field without specifying each field in query
say I have a dynamic field called Foo* (where * can be in the hundreds) and
want to search Foo* for a value of 3 (for example)
I know I can do this via this:
http://localhost:8994/solr/select?q=(Foo1:3 OR Foo2:3 OR Foo3:3 OR …
Foo999:3)
However, is there a better way? i.e. is there some way to query by a
function I create, possibly something like this:
http://localhost:8994/solr/select?q=myfunction(‘Foo’, 3)
where myfunction itself iterates thru all the instances of Foo*
any help appreciated
--
View this message in context: http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25280228.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to scan dynamic field without specifying each field in query
Posted by Avlesh Singh <av...@gmail.com>.
A query parser, may be.
But that would not help either. End of the day, someone has to create those
many boolean queries in your case.
Cheers
Avlesh
On Thu, Sep 3, 2009 at 10:59 PM, gdeconto <ge...@topproducer.com>wrote:
>
> thx for the reply.
>
> you mean into a multivalue field? possible, but was wondering if there was
> something more flexible than that. the ability to use a function (ie
> myfunction) would open up some possibilities for more complex searching and
> search syntax.
>
> I could write my own query parser with special extended syntax, but that is
> farther than I wanted to go.
>
>
>
> Manepalli, Kalyan wrote:
> >
> > You can copy the dynamic fields value into a different field and query on
> > that field.
> >
> > Thanks,
> > Kalyan Manepalli
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25280669.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Re: how to scan dynamic field without specifying each field in query
Posted by Renaud Delbru <re...@deri.org>.
Hi,
maybe SIREn [1] can help you for this task. SIREn is a Lucene plugin
that allows to index and query tabular data. You can for example create
a SIREn field "foo", index n values in n cells, and then query a
specific cell or a range of cells. Unfortunately, the Solr plugin is not
yet available, and therefore you will have to write your own query
syntax and parser for this task.
Regards,
[1] http://siren.sindice.com
--
Renaud Delbru
gdeconto wrote:
> thx for the reply.
>
> you mean into a multivalue field? possible, but was wondering if there was
> something more flexible than that. the ability to use a function (ie
> myfunction) would open up some possibilities for more complex searching and
> search syntax.
>
> I could write my own query parser with special extended syntax, but that is
> farther than I wanted to go.
>
>
>
> Manepalli, Kalyan wrote:
>
>> You can copy the dynamic fields value into a different field and query on
>> that field.
>>
>> Thanks,
>> Kalyan Manepalli
>>
>>
>>
>
>
RE: how to scan dynamic field without specifying each field in
query
Posted by gdeconto <ge...@topproducer.com>.
thx for the reply.
you mean into a multivalue field? possible, but was wondering if there was
something more flexible than that. the ability to use a function (ie
myfunction) would open up some possibilities for more complex searching and
search syntax.
I could write my own query parser with special extended syntax, but that is
farther than I wanted to go.
Manepalli, Kalyan wrote:
>
> You can copy the dynamic fields value into a different field and query on
> that field.
>
> Thanks,
> Kalyan Manepalli
>
>
--
View this message in context: http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25280669.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to scan dynamic field without specifying each field in query
Posted by Avlesh Singh <av...@gmail.com>.
>
> I know I can do this via this: http://localhost:8994/solr/select?q=(Foo1:3OR Foo2:3 OR Foo3:3 OR ... Foo999:3)
>
Careful! You may hit the upper limit for MAX_BOOLEAN_CLAUSES this way.
> You can copy the dynamic fields value into a different field and query on
> that field.
>
Good idea!
Cheers
**Avlesh
On Thu, Sep 3, 2009 at 10:47 PM, Manepalli, Kalyan <
KALYAN.MANEPALLI@orbitz.com> wrote:
> You can copy the dynamic fields value into a different field and query on
> that field.
>
> Thanks,
> Kalyan Manepalli
>
> -----Original Message-----
> From: gdeconto [mailto:gerald.deconto@topproducer.com]
> Sent: Thursday, September 03, 2009 12:06 PM
> To: solr-user@lucene.apache.org
> Subject: how to scan dynamic field without specifying each field in query
>
>
> say I have a dynamic field called Foo* (where * can be in the hundreds) and
> want to search Foo* for a value of 3 (for example)
>
> I know I can do this via this:
>
> http://localhost:8994/solr/select?q=(Foo1:3<http://localhost:8994/solr/select?q=%28Foo1:3>OR Foo2:3 OR Foo3:3 OR ...
> Foo999:3)
>
> However, is there a better way? i.e. is there some way to query by a
> function I create, possibly something like this:
>
> http://localhost:8994/solr/select?q=myfunction('Foo<http://localhost:8994/solr/select?q=myfunction%28%27Foo>',
> 3)
>
> where myfunction itself iterates thru all the instances of Foo*
>
> any help appreciated
>
> --
> View this message in context:
> http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25280228.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
RE: how to scan dynamic field without specifying each field in query
Posted by "Manepalli, Kalyan" <KA...@orbitz.com>.
You can copy the dynamic fields value into a different field and query on that field.
Thanks,
Kalyan Manepalli
-----Original Message-----
From: gdeconto [mailto:gerald.deconto@topproducer.com]
Sent: Thursday, September 03, 2009 12:06 PM
To: solr-user@lucene.apache.org
Subject: how to scan dynamic field without specifying each field in query
say I have a dynamic field called Foo* (where * can be in the hundreds) and
want to search Foo* for a value of 3 (for example)
I know I can do this via this:
http://localhost:8994/solr/select?q=(Foo1:3 OR Foo2:3 OR Foo3:3 OR ...
Foo999:3)
However, is there a better way? i.e. is there some way to query by a
function I create, possibly something like this:
http://localhost:8994/solr/select?q=myfunction('Foo', 3)
where myfunction itself iterates thru all the instances of Foo*
any help appreciated
--
View this message in context: http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25280228.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to scan dynamic field without specifying each field in
query
Posted by gdeconto <ge...@topproducer.com>.
Thanks Avlesh.
I was only thinking of something 'like' function queries (since they
appeared to have similar behavior).
Agree that custom QueryParser is looking like my only choice. Now have to
figure out how to do that :-)
Avlesh Singh wrote:
>
> I don't think you need function queries here. Function queries are
> supposed
> to return score for a document based on their ValueSource. What you
> probably
> need is a custom QueryParser.
>
>
--
View this message in context: http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25300855.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to scan dynamic field without specifying each field in query
Posted by Avlesh Singh <av...@gmail.com>.
>
> I dont have that answer as I was asking a general question, not one for a
> specific situation I am encountering).
>
I can understand :)
what I am essentially asking for is: is there a short, simple and generic
> method/technique to deal with large numbers of dynamic fields (rather than
> having to specify each and every test on each and every dynamic field) in a
> query
>
Not as of now. There are a lot of open issues in Solr aiming to handle
dynamic fields in an intuitive way. SolrJ has already been made capable of
binding dynamic field content into Java beans (
https://issues.apache.org/jira/browse/SOLR-1129). Faceting on "myField_*" (
https://issues.apache.org/jira/browse/SOLR-1387) and adding SolrDocuments
with "Map<String, String> myField_*" (
https://issues.apache.org/jira/browse/SOLR-1357) are just some of the
enhancements on the way.
what originally prompted this question is I was looking at FunctionQueries (
> http://wiki.apache.org/solr/FunctionQuery) and started to wonder if there
> was some way to create my own functions to handle dynamic fields.
>
I don't think you need function queries here. Function queries are supposed
to return score for a document based on their ValueSource. What you probably
need is a custom QueryParser.
Cheers
Avlesh
On Fri, Sep 4, 2009 at 9:48 PM, gdeconto <ge...@topproducer.com>wrote:
>
> I dont have that answer as I was asking a general question, not one for a
> specific situation I am encountering).
>
> what I am essentially asking for is: is there a short, simple and generic
> method/technique to deal with large numbers of dynamic fields (rather than
> having to specify each and every test on each and every dynamic field) in a
> query
>
> what originally prompted this question is I was looking at FunctionQueries
> (http://wiki.apache.org/solr/FunctionQuery) and started to wonder if there
> was some way to create my own functions to handle dynamic fields.
>
>
> Aakash Dharmadhikari wrote:
> >
> > what all other searches you would like to perform on these fields?
> >
> > ...
> >
>
> --
> View this message in context:
> http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25297439.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Re: how to scan dynamic field without specifying each field in
query
Posted by gdeconto <ge...@topproducer.com>.
I dont have that answer as I was asking a general question, not one for a
specific situation I am encountering).
what I am essentially asking for is: is there a short, simple and generic
method/technique to deal with large numbers of dynamic fields (rather than
having to specify each and every test on each and every dynamic field) in a
query
what originally prompted this question is I was looking at FunctionQueries
(http://wiki.apache.org/solr/FunctionQuery) and started to wonder if there
was some way to create my own functions to handle dynamic fields.
Aakash Dharmadhikari wrote:
>
> what all other searches you would like to perform on these fields?
>
> ...
>
--
View this message in context: http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25297439.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: how to scan dynamic field without specifying each field in query
Posted by Aakash Dharmadhikari <aa...@gmail.com>.
what all other searches you would like to perform on these fields?
>From the proposed function definition I believe that when foo*:3 is to be
searched all foo* would be searched and none are to be excluded. Assuming
that this is the only search that are to be performed on these fields, we
might declare the dynamic field foo* and rather than constructing fields
actual name with property key we can construct it with the "to be searched
value" and store key as the value.
So assume we want to search fooA:X fooB:X fooC:X.... I would rather store
fooX as the multivalued field and store all A, B, C as its values.
The search query can be FooX:*, that is if field fooX exists, get all the
values for the same.
But again as I asked first, it would depend on what kind of other queries
you want to perform.
regards,
aakash
2009/9/4 gdeconto <ge...@topproducer.com>
>
> I am thinking that my example was too simple/generic :-U. It is possible
> for
> more several dynamic fields to exist and other functionality to be
> required.
> i.e. what about if my example had read:
>
> http://localhost:8994/solr/select?q=((Foo1:3<http://localhost:8994/solr/select?q=%28%28Foo1:3>OR Foo2:3 OR Foo3:3 OR …
> Foo999:3) AND (Bar1:1 OR Bar2:1 OR Bar3:1...Bar999:1) AND (Etc1:7 OR Etc2:7
> OR Etc3:7...Etc:999:7)
>
> obviously a nasty query (and care would be needed for MAX_BOOLEAN_CLAUSES).
> that said, are there other mechanisms to better handle that type of query,
> i.e.:
>
> http://localhost:8994/solr/select?q=(myfunction(<http://localhost:8994/solr/select?q=%28myfunction%28>‘Foo’,
> 3) AND
> myfunction('Bar', 1) AND (myfunction('Etc', 7))
>
>
> gdeconto wrote:
> >
> > say I have a dynamic field called Foo* (where * can be in the hundreds)
> > and want to search Foo* for a value of 3 (for example)
> >
> > I know I can do this via this:
> >
> > http://localhost:8994/solr/select?q=(Foo1:3<http://localhost:8994/solr/select?q=%28Foo1:3>OR Foo2:3 OR Foo3:3 OR …
> > Foo999:3)
> >
> > However, is there a better way? i.e. is there some way to query by a
> > function I create, possibly something like this:
> >
> > http://localhost:8994/solr/select?q=myfunction(<http://localhost:8994/solr/select?q=myfunction%28>‘Foo’,
> 3)
> >
> > where myfunction itself iterates thru all the instances of Foo*
> >
> > any help appreciated
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25283094.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Re: how to scan dynamic field without specifying each field in
query
Posted by gdeconto <ge...@topproducer.com>.
I am thinking that my example was too simple/generic :-U. It is possible for
more several dynamic fields to exist and other functionality to be required.
i.e. what about if my example had read:
http://localhost:8994/solr/select?q=((Foo1:3 OR Foo2:3 OR Foo3:3 OR …
Foo999:3) AND (Bar1:1 OR Bar2:1 OR Bar3:1...Bar999:1) AND (Etc1:7 OR Etc2:7
OR Etc3:7...Etc:999:7)
obviously a nasty query (and care would be needed for MAX_BOOLEAN_CLAUSES).
that said, are there other mechanisms to better handle that type of query,
i.e.:
http://localhost:8994/solr/select?q=(myfunction(‘Foo’, 3) AND
myfunction('Bar', 1) AND (myfunction('Etc', 7))
gdeconto wrote:
>
> say I have a dynamic field called Foo* (where * can be in the hundreds)
> and want to search Foo* for a value of 3 (for example)
>
> I know I can do this via this:
>
> http://localhost:8994/solr/select?q=(Foo1:3 OR Foo2:3 OR Foo3:3 OR …
> Foo999:3)
>
> However, is there a better way? i.e. is there some way to query by a
> function I create, possibly something like this:
>
> http://localhost:8994/solr/select?q=myfunction(‘Foo’, 3)
>
> where myfunction itself iterates thru all the instances of Foo*
>
> any help appreciated
>
>
--
View this message in context: http://www.nabble.com/how-to-scan-dynamic-field-without-specifying-each-field-in-query-tp25280228p25283094.html
Sent from the Solr - User mailing list archive at Nabble.com.