delete solr data and index older than 3 days

[ppeddi <ph...@gmail.com>]

hi, 
Can anyone please post the syntax or some examples for deleting solr data
and index older than 3 days based on datetime field in solr collection. 

I have field data_Start_time which stores the date value in and is of type
'date' in my solr collection. I want to delete the index/data older than 3
days and then optimize the collection.

Please let me know the syntax. 

Thanks,
Ppeddi




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: delete solr data and index older than 3 days

[Alessandro Hoss <al...@gmail.com>]

My bad... my answer is wrong. It's deleting only the data from the last
three days, which is the opposite that you want..

Please check Erick's answer.

Sorry.

On Thu, Dec 28, 2017 at 6:39 PM Alessandro Hoss <al...@gmail.com> wrote:

> Hello,
>
> You can use the same behavior as the delete all
> <https://wiki.apache.org/solr/FAQ#How_can_I_delete_all_documents_from_my_index.3F>,
> but instead of querying for "*:*", you should query for something like
> "yourdatefield:[NOW-3DAYS TO NOW]"
>
> If you need to round to the start of the third day before today and the
> end of today you can use "[NOW/DAY-3DAYS TO NOW/DAY+1DAY-1SECOND]"
>
> See also working with dates
> <https://lucene.apache.org/solr/guide/7_2/working-with-dates.html>
>
> --
> Alessandro Hoss
>
> On Thu, Dec 28, 2017 at 5:02 PM ppeddi <ph...@gmail.com> wrote:
>
>> hi,
>> Can anyone please post the syntax or some examples for deleting solr data
>> and index older than 3 days based on datetime field in solr collection.
>>
>> I have field data_Start_time which stores the date value in and is of type
>> 'date' in my solr collection. I want to delete the index/data older than 3
>> days and then optimize the collection.
>>
>> Please let me know the syntax.
>>
>> Thanks,
>> Ppeddi
>>
>>
>>
>>
>> --
>> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>>
>

Re: delete solr data and index older than 3 days

[Alessandro Hoss <al...@gmail.com>]

Hello,

You can use the same behavior as the delete all
<https://wiki.apache.org/solr/FAQ#How_can_I_delete_all_documents_from_my_index.3F>,
but instead of querying for "*:*", you should query for something like
"yourdatefield:[NOW-3DAYS TO NOW]"

If you need to round to the start of the third day before today and the end
of today you can use "[NOW/DAY-3DAYS TO NOW/DAY+1DAY-1SECOND]"

See also working with dates
<https://lucene.apache.org/solr/guide/7_2/working-with-dates.html>

--
Alessandro Hoss

On Thu, Dec 28, 2017 at 5:02 PM ppeddi <ph...@gmail.com> wrote:

> hi,
> Can anyone please post the syntax or some examples for deleting solr data
> and index older than 3 days based on datetime field in solr collection.
>
> I have field data_Start_time which stores the date value in and is of type
> 'date' in my solr collection. I want to delete the index/data older than 3
> days and then optimize the collection.
>
> Please let me know the syntax.
>
> Thanks,
> Ppeddi
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: delete solr data and index older than 3 days

[Alexandre Rafalovitch <ar...@gmail.com>]

DocExpirationUpdateProcessorFactory may be interesting for you:
http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/update/processor/DocExpirationUpdateProcessorFactory.html

Also, if this is a rolling-log with distinct indexing, you could
actually do collection aliasing and start the new collection every
day. Then, you just point the 'current' alias at the last 3
collections.

You may also want to watch: https://issues.apache.org/jira/browse/SOLR-11653

Regards,
    Alex.

On 28 December 2017 at 13:53, ppeddi <ph...@gmail.com> wrote:
> hi,
> Can anyone please post the syntax or some examples for deleting solr data
> and index older than 3 days based on datetime field in solr collection.
>
> I have field data_Start_time which stores the date value in and is of type
> 'date' in my solr collection. I want to delete the index/data older than 3
> days and then optimize the collection.
>
> Please let me know the syntax.
>
> Thanks,
> Ppeddi
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: delete solr data and index older than 3 days

[Erick Erickson <er...@gmail.com>]

First off, please don't optimize unless you're willing to do it every
time, there's a long discussion of why here:
https://issues.apache.org/jira/browse/LUCENE-7976.

It's almost always a bad idea to optimize unless you're willing to
optimize every time you update your index.

But second, this is a straight-forward delete-by-query. Something like
solr_url/collection/update?stream.body=<delete><query>date_field:[* TO
NOW/DAY-3DAYS]</query></delete>

You have to have enabled stream.body in recent Solrs, or curl it in or
send it from SolrJ, you haven't said how you are doing your indexing
yet. The "date math" bit (NOW/DAY-3DAYS) means "everything prior to
midnight three days ago.

Best,
Erick

On Thu, Dec 28, 2017 at 10:53 AM, ppeddi <ph...@gmail.com> wrote:
> hi,
> Can anyone please post the syntax or some examples for deleting solr data
> and index older than 3 days based on datetime field in solr collection.
>
> I have field data_Start_time which stores the date value in and is of type
> 'date' in my solr collection. I want to delete the index/data older than 3
> days and then optimize the collection.
>
> Please let me know the syntax.
>
> Thanks,
> Ppeddi
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html