You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ikhsvaku S <ik...@gmail.com> on 2011/10/11 21:47:24 UTC
Pls help :-) ! calling external ws/db to fetch field instead of own index?
Help pls sirs :)
On Tue, Oct 11, 2011 at 1:29 PM, Ikhsvaku S <ik...@gmail.com> wrote:
> Hi,
>
> We were recently investigating use of Solr for querying & indexing our bug
> database. We are very happy and most of the fields could be indexed
> straightforward. But there are some of the fields that cant be indexed as
> they are changed all the time and we want to incorporate that too in solr
> query. Even if solr doesnt index that external field it instead accepts the
> query and calls the external script/db/webservice to get value of the field
> and proceed with rest...
>
> Like it can allow queries with these external fields but instead of
> checking it in its own indexes - queries external database/web-service and
> then return results...?
>
> Eg. userid_s:a* AND my_external_field:a* AND title_s:a*
>
>
> Here my_external_field is fetched from webservice/rdbms calls and returns
> the set of unique doc ids
>
>
> Thanks
>
Re: Pls help :-) ! calling external ws/db to fetch field instead of own index?
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hi,
No, Solr out of the box cannot call external services - it searches its own index and returns documents indexed/stored there that match a query.
You may be able to write a custom SearchComponent that also pulls in data from external data sources and incorporate those in the results, essentially doing federated search.
See http://search-lucene.com/?q=federated+search&fc_project=Solr
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/
----- Original Message -----
> From: Ikhsvaku S <ik...@gmail.com>
> To: solr-user@lucene.apache.org
> Cc:
> Sent: Tuesday, October 11, 2011 3:47 PM
> Subject: Pls help :-) ! calling external ws/db to fetch field instead of own index?
>
> Help pls sirs :)
>
> On Tue, Oct 11, 2011 at 1:29 PM, Ikhsvaku S <ik...@gmail.com> wrote:
>
>> Hi,
>>
>> We were recently investigating use of Solr for querying & indexing our
> bug
>> database. We are very happy and most of the fields could be indexed
>> straightforward. But there are some of the fields that cant be indexed as
>> they are changed all the time and we want to incorporate that too in solr
>> query. Even if solr doesnt index that external field it instead accepts the
>> query and calls the external script/db/webservice to get value of the field
>> and proceed with rest...
>>
>> Like it can allow queries with these external fields but instead of
>> checking it in its own indexes - queries external database/web-service and
>> then return results...?
>>
>> Eg. userid_s:a* AND my_external_field:a* AND title_s:a*
>>
>>
>> Here my_external_field is fetched from webservice/rdbms calls and returns
>> the set of unique doc ids
>>
>>
>> Thanks
>>
>
Re: Pls help :-) ! calling external ws/db to fetch field instead of own index?
Posted by Jan Høydahl <ja...@cominvent.com>.
Hi,
I would first of all try the most simple solution, namely re-index bug documents on every change, even if they occur frequently. Solr can handle quite some update load, so I would not call it impossible before I've tried.
If all you need the external value for is boosting score, then you could use ExternalFileField which is a .txt file which you can update when you want without reindexing. But these cannot be queried or returned in results, only used for boosting and sorting.
The next option is Join. Solr may perform very simple join on a "foreign key", in which case you may update the frequently-changing fields in their own separate small documents, and join them in at query time. There are limitations to this approach though.
Finally, in 4.0 we get DocValues/Column Stride Fields, which is meant for "external" values which change frequently. The idea is that these will be updateable without reindexing, but I don't think they are yet. See http://www.slideshare.net/lucenerevolution/willnauer-simon-doc-values-column-stride-fields-in-lucene for more.
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com
On 11. okt. 2011, at 21:47, Ikhsvaku S wrote:
> Help pls sirs :)
>
> On Tue, Oct 11, 2011 at 1:29 PM, Ikhsvaku S <ik...@gmail.com> wrote:
>
>> Hi,
>>
>> We were recently investigating use of Solr for querying & indexing our bug
>> database. We are very happy and most of the fields could be indexed
>> straightforward. But there are some of the fields that cant be indexed as
>> they are changed all the time and we want to incorporate that too in solr
>> query. Even if solr doesnt index that external field it instead accepts the
>> query and calls the external script/db/webservice to get value of the field
>> and proceed with rest...
>>
>> Like it can allow queries with these external fields but instead of
>> checking it in its own indexes - queries external database/web-service and
>> then return results...?
>>
>> Eg. userid_s:a* AND my_external_field:a* AND title_s:a*
>>
>>
>> Here my_external_field is fetched from webservice/rdbms calls and returns
>> the set of unique doc ids
>>
>>
>> Thanks
>>
RE: Pls help :-) ! calling external ws/db to fetch field instead of
own index?
Posted by "Jaeger, Jay - DOT" <Ja...@dot.wi.gov>.
Perhaps integrate this using a javascript or other application front end to query solr, get the key to the database, and then run off to get the data?
-----Original Message-----
From: Ikhsvaku S [mailto:ikhsvaku@gmail.com]
Sent: Tuesday, October 11, 2011 2:47 PM
To: solr-user@lucene.apache.org
Subject: Pls help :-) ! calling external ws/db to fetch field instead of own index?
Help pls sirs :)
On Tue, Oct 11, 2011 at 1:29 PM, Ikhsvaku S <ik...@gmail.com> wrote:
> Hi,
>
> We were recently investigating use of Solr for querying & indexing our bug
> database. We are very happy and most of the fields could be indexed
> straightforward. But there are some of the fields that cant be indexed as
> they are changed all the time and we want to incorporate that too in solr
> query. Even if solr doesnt index that external field it instead accepts the
> query and calls the external script/db/webservice to get value of the field
> and proceed with rest...
>
> Like it can allow queries with these external fields but instead of
> checking it in its own indexes - queries external database/web-service and
> then return results...?
>
> Eg. userid_s:a* AND my_external_field:a* AND title_s:a*
>
>
> Here my_external_field is fetched from webservice/rdbms calls and returns
> the set of unique doc ids
>
>
> Thanks
>