You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Claudio Atzori <cl...@isti.cnr.it> on 2010/10/11 09:38:10 UTC
deleteByQuery issue
Hi everybody,
in my application I use an instance of EmbeddedSolrServer (solr 1.4.1),
the following snippet shows how I am instantiating it:
> File home = new File(indexDataPath(solrDataDir, indexName));
>
> container = new CoreContainer(indexDataPath(solrDataDir,
> indexName));
> container.load(indexDataPath(solrDataDir, indexName), new
> File(home, "solr.xml"));
>
> return new EmbeddedSolrServer(container, indexName);
and I'm going through some issues using deleteByQuery method, in fact,
when I try to delete a subset of documents, or even all the documents
from the index, I see as they are correctly marked for deletion on the
luke inspector (http://code.google.com/p/luke/), but after a commit I
can still retrieve them, just like they haven't been removed...
I can see the difference and see the documents disappear only when I
restart my jetty application, but obviously this cannot be a feature...
any idea?
Re: deleteByQuery issue
Posted by Claudio Atzori <cl...@isti.cnr.it>.
Hi Eick, thanks for replying.
Yes I do commit after deleting-by-query, but since the IndexReader is an
internal (at lest for my app point of view), is there a way to reopen it
(or a new one) when I do a commit?
Claudio
On 10/11/2010 10:08 PM, Erick Erickson wrote:
> I'd guess that after you delete your documents and commit, you're still
> using
> an IndexReader that you haven't reopened when you search. WARNING:
> I'm not all that familiar with EmbeddedSolrServer, so this may be waaaaay
> off
> base.
>
> HTH
> Erick
>
> On Mon, Oct 11, 2010 at 12:04 PM, Claudio Atzori<claudio.atzori@isti.cnr.it
>> wrote:
>
>> On 10/11/2010 04:06 PM, Ahmet Arslan wrote:
>>
>>>
>>> --- On Mon, 10/11/10, Claudio Atzori<cl...@isti.cnr.it> wrote:
>>>
>>> From: Claudio Atzori<cl...@isti.cnr.it>
>>>> Subject: deleteByQuery issue
>>>> To: solr-user@lucene.apache.org
>>>> Date: Monday, October 11, 2010, 10:38 AM
>>>> Hi everybody,
>>>> in my application I use an instance of EmbeddedSolrServer
>>>> (solr 1.4.1), the following snippet shows how I am
>>>> instantiating it:
>>>>
>>>> File home = new
>>>>>
>>>> File(indexDataPath(solrDataDir, indexName));
>>>>
>>>>> container = new
>>>>>
>>>> CoreContainer(indexDataPath(solrDataDir, indexName));
>>>>
>>>>>
>>>>> container.load(indexDataPath(solrDataDir,
>>>> indexName), new File(home, "solr.xml"));
>>>>
>>>>> return new
>>>>>
>>>> EmbeddedSolrServer(container, indexName);
>>>>
>>>> and I'm going through some issues using deleteByQuery
>>>> method, in fact, when I try to delete a subset of documents,
>>>> or even all the documents from the index, I see as they are
>>>> correctly marked for deletion on the luke inspector (
>>>> http://code.google.com/p/luke/), but after a commit I
>>>> can still retrieve them, just like they haven't been
>>>> removed...
>>>>
>>>> I can see the difference and see the documents disappear
>>>> only when I restart my jetty application, but obviously this
>>>> cannot be a feature... any idea?
>>>>
>>> I think you are accessing same solr index using both embedded server and
>>> http.
>>> The changes that you made using embedded server won't be reflected to http
>>> until a commit issued from http. I mean if you hit this url:
>>>
>>> http://localhost:8983/solr/update?commit=true
>>>
>>> the deleted documents won't be retrieved anymore.
>>>
>>> P.s. if you want to expunge deleted docs completely you can either
>>> optimize or commit with expungeDeletes = "true".
>>>
>>>
>> Thanks for your reply.
>> Alright I'll better explain my scenario. I'm not exposing any http
>> interface of the index. I handle the whole index 'life cycle' via java code
>> with the EmbeddedSolrServer instance, so I'm handling commits,
>> optimizations, feedings, index creation, all through that instance, moreover
>> my client application calls embeddedSolrServerInstance.commit() after
>> deleteByQuery, but the documents are still there....
>>
>>
>
Re: deleteByQuery issue
Posted by Erick Erickson <er...@gmail.com>.
I'd guess that after you delete your documents and commit, you're still
using
an IndexReader that you haven't reopened when you search. WARNING:
I'm not all that familiar with EmbeddedSolrServer, so this may be waaaaay
off
base.
HTH
Erick
On Mon, Oct 11, 2010 at 12:04 PM, Claudio Atzori <claudio.atzori@isti.cnr.it
> wrote:
> On 10/11/2010 04:06 PM, Ahmet Arslan wrote:
>
>>
>> --- On Mon, 10/11/10, Claudio Atzori<cl...@isti.cnr.it> wrote:
>>
>> From: Claudio Atzori<cl...@isti.cnr.it>
>>> Subject: deleteByQuery issue
>>> To: solr-user@lucene.apache.org
>>> Date: Monday, October 11, 2010, 10:38 AM
>>> Hi everybody,
>>> in my application I use an instance of EmbeddedSolrServer
>>> (solr 1.4.1), the following snippet shows how I am
>>> instantiating it:
>>>
>>> File home = new
>>>>
>>> File(indexDataPath(solrDataDir, indexName));
>>>
>>>> container = new
>>>>
>>> CoreContainer(indexDataPath(solrDataDir, indexName));
>>>
>>>>
>>>> container.load(indexDataPath(solrDataDir,
>>> indexName), new File(home, "solr.xml"));
>>>
>>>> return new
>>>>
>>> EmbeddedSolrServer(container, indexName);
>>>
>>> and I'm going through some issues using deleteByQuery
>>> method, in fact, when I try to delete a subset of documents,
>>> or even all the documents from the index, I see as they are
>>> correctly marked for deletion on the luke inspector (
>>> http://code.google.com/p/luke/), but after a commit I
>>> can still retrieve them, just like they haven't been
>>> removed...
>>>
>>> I can see the difference and see the documents disappear
>>> only when I restart my jetty application, but obviously this
>>> cannot be a feature... any idea?
>>>
>> I think you are accessing same solr index using both embedded server and
>> http.
>> The changes that you made using embedded server won't be reflected to http
>> until a commit issued from http. I mean if you hit this url:
>>
>> http://localhost:8983/solr/update?commit=true
>>
>> the deleted documents won't be retrieved anymore.
>>
>> P.s. if you want to expunge deleted docs completely you can either
>> optimize or commit with expungeDeletes = "true".
>>
>>
> Thanks for your reply.
> Alright I'll better explain my scenario. I'm not exposing any http
> interface of the index. I handle the whole index 'life cycle' via java code
> with the EmbeddedSolrServer instance, so I'm handling commits,
> optimizations, feedings, index creation, all through that instance, moreover
> my client application calls embeddedSolrServerInstance.commit() after
> deleteByQuery, but the documents are still there....
>
>
Re: deleteByQuery issue
Posted by Claudio Atzori <cl...@isti.cnr.it>.
On 10/11/2010 04:06 PM, Ahmet Arslan wrote:
>
> --- On Mon, 10/11/10, Claudio Atzori<cl...@isti.cnr.it> wrote:
>
>> From: Claudio Atzori<cl...@isti.cnr.it>
>> Subject: deleteByQuery issue
>> To: solr-user@lucene.apache.org
>> Date: Monday, October 11, 2010, 10:38 AM
>> Hi everybody,
>> in my application I use an instance of EmbeddedSolrServer
>> (solr 1.4.1), the following snippet shows how I am
>> instantiating it:
>>
>>> File home = new
>> File(indexDataPath(solrDataDir, indexName));
>>> container = new
>> CoreContainer(indexDataPath(solrDataDir, indexName));
>>>
>> container.load(indexDataPath(solrDataDir,
>> indexName), new File(home, "solr.xml"));
>>> return new
>> EmbeddedSolrServer(container, indexName);
>>
>> and I'm going through some issues using deleteByQuery
>> method, in fact, when I try to delete a subset of documents,
>> or even all the documents from the index, I see as they are
>> correctly marked for deletion on the luke inspector (http://code.google.com/p/luke/), but after a commit I
>> can still retrieve them, just like they haven't been
>> removed...
>>
>> I can see the difference and see the documents disappear
>> only when I restart my jetty application, but obviously this
>> cannot be a feature... any idea?
> I think you are accessing same solr index using both embedded server and http.
> The changes that you made using embedded server won't be reflected to http until a commit issued from http. I mean if you hit this url:
>
> http://localhost:8983/solr/update?commit=true
>
> the deleted documents won't be retrieved anymore.
>
> P.s. if you want to expunge deleted docs completely you can either optimize or commit with expungeDeletes = "true".
>
Thanks for your reply.
Alright I'll better explain my scenario. I'm not exposing any http
interface of the index. I handle the whole index 'life cycle' via java
code with the EmbeddedSolrServer instance, so I'm handling commits,
optimizations, feedings, index creation, all through that instance,
moreover my client application calls embeddedSolrServerInstance.commit()
after deleteByQuery, but the documents are still there....
Re: deleteByQuery issue
Posted by Ahmet Arslan <io...@yahoo.com>.
--- On Mon, 10/11/10, Claudio Atzori <cl...@isti.cnr.it> wrote:
> From: Claudio Atzori <cl...@isti.cnr.it>
> Subject: deleteByQuery issue
> To: solr-user@lucene.apache.org
> Date: Monday, October 11, 2010, 10:38 AM
> Hi everybody,
> in my application I use an instance of EmbeddedSolrServer
> (solr 1.4.1), the following snippet shows how I am
> instantiating it:
>
> > File home = new
> File(indexDataPath(solrDataDir, indexName));
> >
> > container = new
> CoreContainer(indexDataPath(solrDataDir, indexName));
> >
> container.load(indexDataPath(solrDataDir,
> indexName), new File(home, "solr.xml"));
> >
> > return new
> EmbeddedSolrServer(container, indexName);
>
> and I'm going through some issues using deleteByQuery
> method, in fact, when I try to delete a subset of documents,
> or even all the documents from the index, I see as they are
> correctly marked for deletion on the luke inspector (http://code.google.com/p/luke/), but after a commit I
> can still retrieve them, just like they haven't been
> removed...
>
> I can see the difference and see the documents disappear
> only when I restart my jetty application, but obviously this
> cannot be a feature... any idea?
I think you are accessing same solr index using both embedded server and http.
The changes that you made using embedded server won't be reflected to http until a commit issued from http. I mean if you hit this url:
http://localhost:8983/solr/update?commit=true
the deleted documents won't be retrieved anymore.
P.s. if you want to expunge deleted docs completely you can either optimize or commit with expungeDeletes = "true".