You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Peyman Faratin <pe...@robustlinks.com> on 2012/06/28 22:02:39 UTC
index writer in searchComponent
Hi
Is it possible to add a new document to the index in a custom SearchComponent (that also implements a SolrCoreAware)? I can get a reference to the indexReader via the ResponseBuilder parameter of the process() method using
rb.req.getSearcher().getReader()
But is it possible to actually add a new document to the index _after_ searching the index? I.e accessing the indexWriter?
thank you
Peyman
Re: index writer in searchComponent
Posted by Dmitry Kan <dm...@gmail.com>.
Hi Peyman,
It is, at least from your perspective and probably system design. In our
case for example, we have slightly different approach, where a user query
is formed on a client side and then caught and pre-processed on a "backend"
side before being sent over to solr. That "backend" side would be then
ideal for any check of a query.
But back to your setup. Could share more thoughts about where do you get
the queries from, why do they inter-mix with your indexing processes? Could
you separate gathering of queries and indexing? If you could, I can't see,
why, prior to indexing you wouldn't be able to query your index and figure
out item 3.
Could be some X Y problem, but I don't insist.
Regards,
Dmitry
On Mon, Jul 2, 2012 at 12:32 AM, Peyman Faratin <pe...@robustlinks.com>wrote:
> Hi Dmitry
> Which SolrJ API would I use to receive the user query? I was under the
> impression the request handler mechanism was the (RESTFUL) interface
> between user query and the index/s.
> thank you
> Peyman
>
> On Jul 1, 2012, at 10:11 AM, Dmitry Kan wrote:
>
> > Hi Peyman,
> >
> > Could you just use solrj api for this purpose? That is, ask via solrj api
> > 1-2 and perform 3 if entity (assuming you mean document or some field
> value
> > by X) didn't exist, i.e. add it to the index.
> >
> > // Dmitry
> >
> > On Sun, Jul 1, 2012 at 6:03 AM, Peyman Faratin <peyman@robustlinks.com
> >wrote:
> >
> >> Hi Erik
> >>
> >> The workflow I'd like to implement is
> >>
> >> 1- search the index using the incoming query
> >> 2- the query is of the type "does entity X exist"
> >> 3- if X does not exist in the index then I'd like to add X to the index
> >>
> >> Currently I am using a custom search component to achieve this by
> creating
> >> a solrserver within the init (or inform) method of the search component
> and
> >> using that instance to update (and commit) the index. I am not sure
> this is
> >> the best approach either and thought using the IndexReader of the search
> >> component itself maybe better.
> >>
> >> Is there a better approach in your opinion?
> >>
> >> thank you Erik
> >>
> >> Peyman
> >>
> >> On Jun 30, 2012, at 8:13 PM, Erick Erickson wrote:
> >>
> >>> Lots of the index modification (all of it?) has been removed in 4.0
> >>> from IndexReaders...
> >>>
> >>> It seems like you could always get the directory and open a
> >>> SolrIndexWriter wherever you wanted,
> >>> but I'm not sure it's a good idea, are there other processes that will
> >>> be writing to the index at the
> >>> same time?
> >>>
> >>> What's the purpose here anyway? There might be a better approach....
> >>>
> >>> Best
> >>> Erick
> >>>
> >>> On Thu, Jun 28, 2012 at 4:02 PM, Peyman Faratin <
> peyman@robustlinks.com>
> >> wrote:
> >>>> Hi
> >>>>
> >>>> Is it possible to add a new document to the index in a custom
> >> SearchComponent (that also implements a SolrCoreAware)? I can get a
> >> reference to the indexReader via the ResponseBuilder parameter of the
> >> process() method using
> >>>>
> >>>> rb.req.getSearcher().getReader()
> >>>>
> >>>> But is it possible to actually add a new document to the index _after_
> >> searching the index? I.e accessing the indexWriter?
> >>>>
> >>>> thank you
> >>>>
> >>>> Peyman
> >>
> >>
> >
> >
> > --
> > Regards,
> >
> > Dmitry Kan
>
>
--
Regards,
Dmitry Kan
Re: index writer in searchComponent
Posted by Peyman Faratin <pe...@robustlinks.com>.
Hi Dmitry
Which SolrJ API would I use to receive the user query? I was under the impression the request handler mechanism was the (RESTFUL) interface between user query and the index/s.
thank you
Peyman
On Jul 1, 2012, at 10:11 AM, Dmitry Kan wrote:
> Hi Peyman,
>
> Could you just use solrj api for this purpose? That is, ask via solrj api
> 1-2 and perform 3 if entity (assuming you mean document or some field value
> by X) didn't exist, i.e. add it to the index.
>
> // Dmitry
>
> On Sun, Jul 1, 2012 at 6:03 AM, Peyman Faratin <pe...@robustlinks.com>wrote:
>
>> Hi Erik
>>
>> The workflow I'd like to implement is
>>
>> 1- search the index using the incoming query
>> 2- the query is of the type "does entity X exist"
>> 3- if X does not exist in the index then I'd like to add X to the index
>>
>> Currently I am using a custom search component to achieve this by creating
>> a solrserver within the init (or inform) method of the search component and
>> using that instance to update (and commit) the index. I am not sure this is
>> the best approach either and thought using the IndexReader of the search
>> component itself maybe better.
>>
>> Is there a better approach in your opinion?
>>
>> thank you Erik
>>
>> Peyman
>>
>> On Jun 30, 2012, at 8:13 PM, Erick Erickson wrote:
>>
>>> Lots of the index modification (all of it?) has been removed in 4.0
>>> from IndexReaders...
>>>
>>> It seems like you could always get the directory and open a
>>> SolrIndexWriter wherever you wanted,
>>> but I'm not sure it's a good idea, are there other processes that will
>>> be writing to the index at the
>>> same time?
>>>
>>> What's the purpose here anyway? There might be a better approach....
>>>
>>> Best
>>> Erick
>>>
>>> On Thu, Jun 28, 2012 at 4:02 PM, Peyman Faratin <pe...@robustlinks.com>
>> wrote:
>>>> Hi
>>>>
>>>> Is it possible to add a new document to the index in a custom
>> SearchComponent (that also implements a SolrCoreAware)? I can get a
>> reference to the indexReader via the ResponseBuilder parameter of the
>> process() method using
>>>>
>>>> rb.req.getSearcher().getReader()
>>>>
>>>> But is it possible to actually add a new document to the index _after_
>> searching the index? I.e accessing the indexWriter?
>>>>
>>>> thank you
>>>>
>>>> Peyman
>>
>>
>
>
> --
> Regards,
>
> Dmitry Kan
Re: index writer in searchComponent
Posted by Dmitry Kan <dm...@gmail.com>.
Hi Peyman,
Could you just use solrj api for this purpose? That is, ask via solrj api
1-2 and perform 3 if entity (assuming you mean document or some field value
by X) didn't exist, i.e. add it to the index.
// Dmitry
On Sun, Jul 1, 2012 at 6:03 AM, Peyman Faratin <pe...@robustlinks.com>wrote:
> Hi Erik
>
> The workflow I'd like to implement is
>
> 1- search the index using the incoming query
> 2- the query is of the type "does entity X exist"
> 3- if X does not exist in the index then I'd like to add X to the index
>
> Currently I am using a custom search component to achieve this by creating
> a solrserver within the init (or inform) method of the search component and
> using that instance to update (and commit) the index. I am not sure this is
> the best approach either and thought using the IndexReader of the search
> component itself maybe better.
>
> Is there a better approach in your opinion?
>
> thank you Erik
>
> Peyman
>
> On Jun 30, 2012, at 8:13 PM, Erick Erickson wrote:
>
> > Lots of the index modification (all of it?) has been removed in 4.0
> > from IndexReaders...
> >
> > It seems like you could always get the directory and open a
> > SolrIndexWriter wherever you wanted,
> > but I'm not sure it's a good idea, are there other processes that will
> > be writing to the index at the
> > same time?
> >
> > What's the purpose here anyway? There might be a better approach....
> >
> > Best
> > Erick
> >
> > On Thu, Jun 28, 2012 at 4:02 PM, Peyman Faratin <pe...@robustlinks.com>
> wrote:
> >> Hi
> >>
> >> Is it possible to add a new document to the index in a custom
> SearchComponent (that also implements a SolrCoreAware)? I can get a
> reference to the indexReader via the ResponseBuilder parameter of the
> process() method using
> >>
> >> rb.req.getSearcher().getReader()
> >>
> >> But is it possible to actually add a new document to the index _after_
> searching the index? I.e accessing the indexWriter?
> >>
> >> thank you
> >>
> >> Peyman
>
>
--
Regards,
Dmitry Kan
Re: index writer in searchComponent
Posted by Peyman Faratin <pe...@robustlinks.com>.
Hi Erik
The workflow I'd like to implement is
1- search the index using the incoming query
2- the query is of the type "does entity X exist"
3- if X does not exist in the index then I'd like to add X to the index
Currently I am using a custom search component to achieve this by creating a solrserver within the init (or inform) method of the search component and using that instance to update (and commit) the index. I am not sure this is the best approach either and thought using the IndexReader of the search component itself maybe better.
Is there a better approach in your opinion?
thank you Erik
Peyman
On Jun 30, 2012, at 8:13 PM, Erick Erickson wrote:
> Lots of the index modification (all of it?) has been removed in 4.0
> from IndexReaders...
>
> It seems like you could always get the directory and open a
> SolrIndexWriter wherever you wanted,
> but I'm not sure it's a good idea, are there other processes that will
> be writing to the index at the
> same time?
>
> What's the purpose here anyway? There might be a better approach....
>
> Best
> Erick
>
> On Thu, Jun 28, 2012 at 4:02 PM, Peyman Faratin <pe...@robustlinks.com> wrote:
>> Hi
>>
>> Is it possible to add a new document to the index in a custom SearchComponent (that also implements a SolrCoreAware)? I can get a reference to the indexReader via the ResponseBuilder parameter of the process() method using
>>
>> rb.req.getSearcher().getReader()
>>
>> But is it possible to actually add a new document to the index _after_ searching the index? I.e accessing the indexWriter?
>>
>> thank you
>>
>> Peyman
Re: index writer in searchComponent
Posted by Erick Erickson <er...@gmail.com>.
Lots of the index modification (all of it?) has been removed in 4.0
from IndexReaders...
It seems like you could always get the directory and open a
SolrIndexWriter wherever you wanted,
but I'm not sure it's a good idea, are there other processes that will
be writing to the index at the
same time?
What's the purpose here anyway? There might be a better approach....
Best
Erick
On Thu, Jun 28, 2012 at 4:02 PM, Peyman Faratin <pe...@robustlinks.com> wrote:
> Hi
>
> Is it possible to add a new document to the index in a custom SearchComponent (that also implements a SolrCoreAware)? I can get a reference to the indexReader via the ResponseBuilder parameter of the process() method using
>
> rb.req.getSearcher().getReader()
>
> But is it possible to actually add a new document to the index _after_ searching the index? I.e accessing the indexWriter?
>
> thank you
>
> Peyman