You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ashish Kumar Srivastava <as...@gmail.com> on 2009/08/04 15:26:51 UTC
Delete solr data from disk space
I am facing a problem in deleting solr data form disk space.
I had 80Gb of of solr data. I deleted 30% of these data by using query in
solr-php client and committed.
Now deleted data is not visible from the solr UI but used disk space is
still 80Gb for solr data.
Please reply if you have any solution to free the disk space after deleting
some solr data.
Thanks in advance.
--
View this message in context: http://www.nabble.com/Delete-solr-data-from-disk-space-tp24808676p24808676.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Delete solr data from disk space
Posted by Ashish Kumar Srivastava <as...@gmail.com>.
Sorry!! But this solution will not work because I deleted data by certain
query.
Then how can i know which files should be deleted. I cant delete whole data.
--
View this message in context: http://www.nabble.com/Delete-solr-data-from-disk-space-tp24808676p24808868.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Delete solr data from disk space
Posted by Ashish Kumar Srivastava <as...@gmail.com>.
Hi Toby,
Thanks but i have tried this solution earlier but the problem with this
solution is that
it is taking too much disk space for optimization(more than two times of
originally index data size)
Do you have any better solution or any other option by which we can use
optimize without using too much space.
Thanks
Ashish
Toby Cole-2 wrote:
>
> Hi Anish,
> Have you optimized your index?
> When you delete documents in lucene they are simply marked as
> 'deleted', they aren't physically removed from the disk.
> To get the disk space back you must run an optimize, which re-writes
> the index out to disk without the deleted documents, then deletes the
> original.
>
> Toby
>
> On 4 Aug 2009, at 14:41, Ashish Kumar Srivastava wrote:
>
>>
>> Hi ,
>>
>>
>> Sorry!! But this solution will not work because I deleted data by
>> certain
>> query.
>> Then how can i know which files should be deleted. I cant delete
>> whole data.
>>
>>
>>
>> Markus Jelsma - Buyways B.V. wrote:
>>>
>>> Hello,
>>>
>>>
>>> A rigorous but quite effective method is manually deleting the
>>> files in
>>> your SOLR_HOME/data directory and reindex the documents you want.
>>> This
>>> will surely free some diskspace.
>>>
>>>
>>> Cheers,
>>>
>>> -
>>> Markus Jelsma Buyways B.V. Tel.
>>> 050-3118123
>>> Technisch Architect Friesestraatweg 215c Fax.
>>> 050-3118124
>>> http://www.buyways.nl 9743 AD Groningen KvK 01074105
>>>
>>>
>>> On Tue, 2009-08-04 at 06:26 -0700, Ashish Kumar Srivastava wrote:
>>>
>>>> I am facing a problem in deleting solr data form disk space.
>>>> I had 80Gb of of solr data. I deleted 30% of these data by using
>>>> query in
>>>> solr-php client and committed.
>>>> Now deleted data is not visible from the solr UI but used disk
>>>> space is
>>>> still 80Gb for solr data.
>>>> Please reply if you have any solution to free the disk space after
>>>> deleting
>>>> some solr data.
>>>>
>>>> Thanks in advance.
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Delete-solr-data-from-disk-space-tp24808676p24808883.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>
>
> --
> Toby Cole
> Software Engineer, Semantico Limited
> Registered in England and Wales no. 03841410, VAT no. GB-744614334.
> Registered office Lees House, 21-23 Dyke Road, Brighton BN1 3FE, UK.
>
> Check out all our latest news and thinking on the Discovery blog
> http://blogs.semantico.com/discovery-blog/
>
>
>
--
View this message in context: http://www.nabble.com/Delete-solr-data-from-disk-space-tp24808676p24821241.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Delete solr data from disk space
Posted by Ashish Kumar Srivastava <as...@gmail.com>.
Hi Toby,
Thanks for the reply, But i have tried this solution earlier but the problem
with this solution is that
it is taking too much disk space for optimization(more than two times of
originally index data size)
Do you have any better solution or any other option by which we can use
optimize without using too much space.
Thanks
Ashish
Toby Cole-2 wrote:
>
> Hi Anish,
> Have you optimized your index?
> When you delete documents in lucene they are simply marked as
> 'deleted', they aren't physically removed from the disk.
> To get the disk space back you must run an optimize, which re-writes
> the index out to disk without the deleted documents, then deletes the
> original.
>
> Toby
>
> On 4 Aug 2009, at 14:41, Ashish Kumar Srivastava wrote:
>
>>
>> Hi ,
>>
>>
>> Sorry!! But this solution will not work because I deleted data by
>> certain
>> query.
>> Then how can i know which files should be deleted. I cant delete
>> whole data.
>>
>>
>>
>> Markus Jelsma - Buyways B.V. wrote:
>>>
>>> Hello,
>>>
>>>
>>> A rigorous but quite effective method is manually deleting the
>>> files in
>>> your SOLR_HOME/data directory and reindex the documents you want.
>>> This
>>> will surely free some diskspace.
>>>
>>>
>>> Cheers,
>>>
>>> -
>>> Markus Jelsma Buyways B.V. Tel.
>>> 050-3118123
>>> Technisch Architect Friesestraatweg 215c Fax.
>>> 050-3118124
>>> http://www.buyways.nl 9743 AD Groningen KvK 01074105
>>>
>>>
>>> On Tue, 2009-08-04 at 06:26 -0700, Ashish Kumar Srivastava wrote:
>>>
>>>> I am facing a problem in deleting solr data form disk space.
>>>> I had 80Gb of of solr data. I deleted 30% of these data by using
>>>> query in
>>>> solr-php client and committed.
>>>> Now deleted data is not visible from the solr UI but used disk
>>>> space is
>>>> still 80Gb for solr data.
>>>> Please reply if you have any solution to free the disk space after
>>>> deleting
>>>> some solr data.
>>>>
>>>> Thanks in advance.
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Delete-solr-data-from-disk-space-tp24808676p24808883.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>
>
> --
> Toby Cole
> Software Engineer, Semantico Limited
> Registered in England and Wales no. 03841410, VAT no. GB-744614334.
> Registered office Lees House, 21-23 Dyke Road, Brighton BN1 3FE, UK.
>
> Check out all our latest news and thinking on the Discovery blog
> http://blogs.semantico.com/discovery-blog/
>
>
>
--
View this message in context: http://www.nabble.com/Delete-solr-data-from-disk-space-tp24808676p24821271.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Delete solr data from disk space
Posted by Toby Cole <to...@semantico.com>.
Hi Anish,
Have you optimized your index?
When you delete documents in lucene they are simply marked as
'deleted', they aren't physically removed from the disk.
To get the disk space back you must run an optimize, which re-writes
the index out to disk without the deleted documents, then deletes the
original.
Toby
On 4 Aug 2009, at 14:41, Ashish Kumar Srivastava wrote:
>
> Hi ,
>
>
> Sorry!! But this solution will not work because I deleted data by
> certain
> query.
> Then how can i know which files should be deleted. I cant delete
> whole data.
>
>
>
> Markus Jelsma - Buyways B.V. wrote:
>>
>> Hello,
>>
>>
>> A rigorous but quite effective method is manually deleting the
>> files in
>> your SOLR_HOME/data directory and reindex the documents you want.
>> This
>> will surely free some diskspace.
>>
>>
>> Cheers,
>>
>> -
>> Markus Jelsma Buyways B.V. Tel.
>> 050-3118123
>> Technisch Architect Friesestraatweg 215c Fax.
>> 050-3118124
>> http://www.buyways.nl 9743 AD Groningen KvK 01074105
>>
>>
>> On Tue, 2009-08-04 at 06:26 -0700, Ashish Kumar Srivastava wrote:
>>
>>> I am facing a problem in deleting solr data form disk space.
>>> I had 80Gb of of solr data. I deleted 30% of these data by using
>>> query in
>>> solr-php client and committed.
>>> Now deleted data is not visible from the solr UI but used disk
>>> space is
>>> still 80Gb for solr data.
>>> Please reply if you have any solution to free the disk space after
>>> deleting
>>> some solr data.
>>>
>>> Thanks in advance.
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Delete-solr-data-from-disk-space-tp24808676p24808883.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Toby Cole
Software Engineer, Semantico Limited
Registered in England and Wales no. 03841410, VAT no. GB-744614334.
Registered office Lees House, 21-23 Dyke Road, Brighton BN1 3FE, UK.
Check out all our latest news and thinking on the Discovery blog
http://blogs.semantico.com/discovery-blog/
Re: Delete solr data from disk space
Posted by Otis Gospodnetic <ot...@yahoo.com>.
You simply can't delete individual index files.
Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
----- Original Message ----
> From: Ashish Kumar Srivastava <as...@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Tuesday, August 4, 2009 9:41:09 AM
> Subject: Re: Delete solr data from disk space
>
>
> Hi ,
>
>
> Sorry!! But this solution will not work because I deleted data by certain
> query.
> Then how can i know which files should be deleted. I cant delete whole data.
>
>
>
> Markus Jelsma - Buyways B.V. wrote:
> >
> > Hello,
> >
> >
> > A rigorous but quite effective method is manually deleting the files in
> > your SOLR_HOME/data directory and reindex the documents you want. This
> > will surely free some diskspace.
> >
> >
> > Cheers,
> >
> > -
> > Markus Jelsma Buyways B.V. Tel. 050-3118123
> > Technisch Architect Friesestraatweg 215c Fax. 050-3118124
> > http://www.buyways.nl 9743 AD Groningen KvK 01074105
> >
> >
> > On Tue, 2009-08-04 at 06:26 -0700, Ashish Kumar Srivastava wrote:
> >
> >> I am facing a problem in deleting solr data form disk space.
> >> I had 80Gb of of solr data. I deleted 30% of these data by using query in
> >> solr-php client and committed.
> >> Now deleted data is not visible from the solr UI but used disk space is
> >> still 80Gb for solr data.
> >> Please reply if you have any solution to free the disk space after
> >> deleting
> >> some solr data.
> >>
> >> Thanks in advance.
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Delete-solr-data-from-disk-space-tp24808676p24808883.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Delete solr data from disk space
Posted by Ashish Kumar Srivastava <as...@gmail.com>.
Hi ,
Sorry!! But this solution will not work because I deleted data by certain
query.
Then how can i know which files should be deleted. I cant delete whole data.
Markus Jelsma - Buyways B.V. wrote:
>
> Hello,
>
>
> A rigorous but quite effective method is manually deleting the files in
> your SOLR_HOME/data directory and reindex the documents you want. This
> will surely free some diskspace.
>
>
> Cheers,
>
> -
> Markus Jelsma Buyways B.V. Tel. 050-3118123
> Technisch Architect Friesestraatweg 215c Fax. 050-3118124
> http://www.buyways.nl 9743 AD Groningen KvK 01074105
>
>
> On Tue, 2009-08-04 at 06:26 -0700, Ashish Kumar Srivastava wrote:
>
>> I am facing a problem in deleting solr data form disk space.
>> I had 80Gb of of solr data. I deleted 30% of these data by using query in
>> solr-php client and committed.
>> Now deleted data is not visible from the solr UI but used disk space is
>> still 80Gb for solr data.
>> Please reply if you have any solution to free the disk space after
>> deleting
>> some solr data.
>>
>> Thanks in advance.
>
>
--
View this message in context: http://www.nabble.com/Delete-solr-data-from-disk-space-tp24808676p24808883.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Delete solr data from disk space
Posted by "Markus Jelsma - Buyways B.V." <ma...@buyways.nl>.
Hello,
A rigorous but quite effective method is manually deleting the files in
your SOLR_HOME/data directory and reindex the documents you want. This
will surely free some diskspace.
Cheers,
-
Markus Jelsma Buyways B.V. Tel. 050-3118123
Technisch Architect Friesestraatweg 215c Fax. 050-3118124
http://www.buyways.nl 9743 AD Groningen KvK 01074105
On Tue, 2009-08-04 at 06:26 -0700, Ashish Kumar Srivastava wrote:
> I am facing a problem in deleting solr data form disk space.
> I had 80Gb of of solr data. I deleted 30% of these data by using query in
> solr-php client and committed.
> Now deleted data is not visible from the solr UI but used disk space is
> still 80Gb for solr data.
> Please reply if you have any solution to free the disk space after deleting
> some solr data.
>
> Thanks in advance.