Solr is deleting newly created index from index folder

[vishalgupta084 <vi...@steria.co.in>]

I am runnig cron job for indexing and commiting documents for solr search.
Earlier everything was fine. But from some time it is deleting indexes from
index folder. Whenever I update any document or create any new document, it
gets indexed and commited and appear in search but after some hour later
when i search for the same document it gets disappear from the search and
when i cheked index folder size then noticed that it gets reduced to its
original size that was before updating the document 
Query is:

 <delete fromPending="true" fromCommitted="true"><query>(endtime:[* TO NOW]
AND -endtime:"1970-01-01T01:00:00Z")</query></delete>

 Could anyone please let me know why it is deleting only newly created
indexes not the old indexes. Old indexes appear in search.

 How can i stop this deletion process.

 Although I checked for deletion policy also but in my solrsonfig.xml it is
commented.

 My solr was running fine on production but now it is creating above
mentioned issue. So urgent help require.




--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-is-deleting-newly-created-index-from-index-folder-tp4102104.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr is deleting newly created index from index folder

[Erick Erickson <er...@gmail.com>]

I cannot imagine that Solr suddenly starts deleting indexes without you
changing anything, although all things are possible. Sanity check:
do you have at least as much free space on your disk as the total size
of your index on disk?

Your query will delete everything from your index with an endtime, except
some very specific dates (i.e. a Unix time of 0).

First thing I'd do is stop the cron job to see if that fixes your problem.
My
bet is that this query is not doing what you expect.

Then submit the query to Solr from a URL or from the admin page, turn
query debugging on and examine the results. Do this with just the query,
not the delete parts, i.e.
q=(endtime:[* TO NOW] AND -endtime:"1970-01-01T01:00:00Z")

BTW, the AND is unnecessary, but I don't think that's relevant anyway...

If that doesn't show the problem, please show us the results of running
the query with debug info so we see the parsed query.

Best,
Erick


On Wed, Nov 20, 2013 at 7:31 AM, vishalgupta084
<vi...@steria.co.in>wrote:

> I am runnig cron job for indexing and commiting documents for solr search.
> Earlier everything was fine. But from some time it is deleting indexes from
> index folder. Whenever I update any document or create any new document, it
> gets indexed and commited and appear in search but after some hour later
> when i search for the same document it gets disappear from the search and
> when i cheked index folder size then noticed that it gets reduced to its
> original size that was before updating the document
> Query is:
>
>  <delete fromPending="true" fromCommitted="true"><query>(endtime:[* TO NOW]
> AND -endtime:"1970-01-01T01:00:00Z")</query></delete>
>
>  Could anyone please let me know why it is deleting only newly created
> indexes not the old indexes. Old indexes appear in search.
>
>  How can i stop this deletion process.
>
>  Although I checked for deletion policy also but in my solrsonfig.xml it is
> commented.
>
>  My solr was running fine on production but now it is creating above
> mentioned issue. So urgent help require.
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-is-deleting-newly-created-index-from-index-folder-tp4102104.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Solr is deleting newly created index from index folder

[Jack Krupansky <ja...@basetechnology.com>]

You may be hitting a query parser bug/nuance, that a purely negative 
sub-query needs to have a *:* added so that it is not purely negative.

So, replace:

AND -endtime:"1970-01-01T01:00:00Z"

with

AND (*:* -endtime:"1970-01-01T01:00:00Z")

Or, as Erick mentioned in his reply, you don't really not to AND with a 
sub-query, so make it:

-endtime:"1970-01-01T01:00:00Z"

And then it is simply a clause of the Boolean query.


-- Jack Krupansky
-----Original Message----- 
From: vishalgupta084
Sent: Wednesday, November 20, 2013 7:31 AM
To: solr-user@lucene.apache.org
Subject: Solr is deleting newly created index from index folder

I am runnig cron job for indexing and commiting documents for solr search.
Earlier everything was fine. But from some time it is deleting indexes from
index folder. Whenever I update any document or create any new document, it
gets indexed and commited and appear in search but after some hour later
when i search for the same document it gets disappear from the search and
when i cheked index folder size then noticed that it gets reduced to its
original size that was before updating the document
Query is:

<delete fromPending="true" fromCommitted="true"><query>(endtime:[* TO NOW]
AND -endtime:"1970-01-01T01:00:00Z")</query></delete>

Could anyone please let me know why it is deleting only newly created
indexes not the old indexes. Old indexes appear in search.

How can i stop this deletion process.

Although I checked for deletion policy also but in my solrsonfig.xml it is
commented.

My solr was running fine on production but now it is creating above
mentioned issue. So urgent help require.




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-is-deleting-newly-created-index-from-index-folder-tp4102104.html
Sent from the Solr - User mailing list archive at Nabble.com.