You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Bogdan Vatkov <bo...@gmail.com> on 2010/01/24 01:05:49 UTC
Index gets deleted after commit?
After mass upload of docs in Solr I get some "REMOVING ALL DOCUMENTS FROM
INDEX" without any explanation.
I was running indexing w/ Solr for several weeks now and everything was ok -
I indexed 22K+ docs using the SimplePostTool
I was first launching
<delete><query>*:*</query></delete>
<commit waitFlush="true" waitSearcher="true"/>
then some 22K+ <Add>...
with a finishing
<commit waitFlush="true" waitSearcher="true"/>
But you can see from the log - right after the last commit I get this
strange REMOVING ALL...
I do not remember what I changed last but now I have this issue that after
the mass upload of docs the index gets completely deleted.
why is this happening?
log after the last commit:
INFO: start
commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy onCommit
INFO: SolrDeletionPolicy.onCommit: commits:num=2
commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/index,segFN=segments_fr,version=1260734716752,generation=567,filenames=[segments_fr]
commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/index,segFN=segments_fs,version=1260734716753,generation=568,filenames=[_gv.nrm,
segments_fs, _gv.fdx, _gw.nrm, _gv.tii, _gv.prx, _gv.tvf, _gv.tis, _gv.tvd,
_gv.fdt, _gw.fnm, _gw.tis, _gw.frq, _gv.fnm, _gw.prx, _gv.tvx, _gw.tii,
_gv.frq]
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1260734716753
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@de26e52 main
Jan 24, 2010 1:48:24 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@de26e52 main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@de26e52 main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@de26e52 main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@de26e52 main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener sending requests to Searcher@de26e52 main
Jan 24, 2010 1:48:24 AM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore registerSearcher
INFO: [] Registered new searcher Searcher@de26e52 main
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher close
INFO: Closing Searcher@4e8deb8a main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {commit=} 0 1043
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={} status=0 QTime=1043
Jan 24, 2010 1:48:24 AM org.apache.solr.update.DirectUpdateHandler2
deleteAll
INFO: [] REMOVING ALL DOCUMENTS FROM INDEX
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/index,segFN=segments_fs,version=1260734716753,generation=568,filenames=[_gv.nrm,
segments_fs, _gv.fdx, _gw.nrm, _gv.tii, _gv.prx, _gv.tvf, _gv.tis, _gv.tvd,
_gv.fdt, _gw.fnm, _gw.tis, _gw.frq, _gv.fnm, _gw.prx, _gv.tvx, _gw.tii,
_gv.frq]
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1260734716753
Jan 24, 2010 1:48:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {deleteByQuery=*:*} 0 5
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={} status=0 QTime=5
Jan 24, 2010 1:48:24 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy onCommit
INFO: SolrDeletionPolicy.onCommit: commits:num=2
Re: Index gets deleted after commit?
Posted by Sven Maurmann <sv...@kippdata.de>.
DIH is the DataImportHandler. Please consult the two URLs
http://wiki.apache.org/solr/DataImportHandler
and
http://wiki.apache.org/solr/DataImportHandlerFaq
for further information.
Cheers,
Sven
--On Monday, January 25, 2010 11:33:59 AM +0200 Bogdan Vatkov
<bo...@gmail.com> wrote:
> Hi Amit,
>
> What is DIH? (I am Solr newbie).
> In the mean time I resolved my issue - it was very stupid one - on
> of the files in my folder with XMLs (that I send to Solr with the
> SimplePostTool), and actually the latest created one (so it got
> executed last each time I run the folder), contaoned
> <delete>*:*.... :)
>
> Best regards,
> Bogdan
>
> On Sun, Jan 24, 2010 at 6:25 AM, Amit Nithian <an...@gmail.com>
> wrote:
>
>> Are you using the DIH? If so, did you try setting clean=false in
>> the URL line? That prevents wiping out the index on load.
>>
>> On Jan 23, 2010 4:06 PM, "Bogdan Vatkov" <bo...@gmail.com>
>> wrote:
>>
>> After mass upload of docs in Solr I get some "REMOVING ALL
>> DOCUMENTS FROM INDEX" without any explanation.
>>
>> I was running indexing w/ Solr for several weeks now and
>> everything was ok -
>> I indexed 22K+ docs using the SimplePostTool
>> I was first launching
>>
>> <delete><query>*:*</query></delete>
>> <commit waitFlush="true" waitSearcher="true"/>
>>
>> then some 22K+ <Add>...
>> with a finishing
>> <commit waitFlush="true" waitSearcher="true"/>
>>
>> But you can see from the log - right after the last commit I get
>> this strange REMOVING ALL...
>> I do not remember what I changed last but now I have this issue
>> that after the mass upload of docs the index gets completely
>> deleted.
>>
>> why is this happening?
>>
>>
>> log after the last commit:
>>
>> INFO: start
>>
>> commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDele
>> tes=false) Jan 24, 2010 1:48:24 AM
>> org.apache.solr.core.SolrDeletionPolicy onCommit INFO:
>> SolrDeletionPolicy.onCommit: commits:num=2
>>
>> commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/ind
>> ex,segFN=segments_fr,version=1260734716752,generation=567,filename
>> s=[segments_fr]
>>
>> commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/ind
>> ex,segFN=segments_fs,version=1260734716753,generation=568,filename
>> s=[_gv.nrm, segments_fs, _gv.fdx, _gw.nrm, _gv.tii, _gv.prx,
>> _gv.tvf, _gv.tis, _gv.tvd, _gv.fdt, _gw.fnm, _gw.tis, _gw.frq,
>> _gv.fnm, _gw.prx, _gv.tvx, _gw.tii, _gv.frq]
>> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy
>> updateCommits
>> INFO: newest commit = 1260734716753
>> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher
>> <init> INFO: Opening Searcher@de26e52 main
>> Jan 24, 2010 1:48:24 AM
>> org.apache.solr.update.DirectUpdateHandler2 commit INFO:
>> end_commit_flush
>> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher
>> warm INFO: autowarming Searcher@de26e52 main from
>> Searcher@4e8deb8a main
>>
>> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions
>> =0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumu
>> lative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher
>> warm INFO: autowarming result for Searcher@de26e52 main
>>
>> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions
>> =0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumu
>> lative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher
>> warm INFO: autowarming Searcher@de26e52 main from
>> Searcher@4e8deb8a main
>>
>> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,s
>> ize=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulati
>> ve_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} Jan
>> 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
>> INFO: autowarming result for Searcher@de26e52 main
>>
>> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,s
>> ize=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulati
>> ve_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0} Jan
>> 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
>> INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
>>
>> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,eviction
>> s=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cum
>> ulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher
>> warm INFO: autowarming result for Searcher@de26e52 main
>>
>> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,eviction
>> s=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cum
>> ulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher
>> warm INFO: autowarming Searcher@de26e52 main from
>> Searcher@4e8deb8a main
>>
>> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0
>> ,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumula
>> tive_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher
>> warm INFO: autowarming result for Searcher@de26e52 main
>>
>> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0
>> ,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumula
>> tive_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>> Jan 24, 2010 1:48:24 AM org.apache.solr.core.QuerySenderListener
>> newSearcher
>> INFO: QuerySenderListener sending requests to Searcher@de26e52 main
>> Jan 24, 2010 1:48:24 AM org.apache.solr.core.QuerySenderListener
>> newSearcher
>> INFO: QuerySenderListener done.
>> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore
>> registerSearcher INFO: [] Registered new searcher Searcher@de26e52
>> main
>> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher
>> close INFO: Closing Searcher@4e8deb8a main
>>
>> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions
>> =0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumu
>> lative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>>
>> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,s
>> ize=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulati
>> ve_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>>
>> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,eviction
>> s=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cum
>> ulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>>
>> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0
>> ,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumula
>> tive_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>> Jan 24, 2010 1:48:24 AM
>> org.apache.solr.update.processor.LogUpdateProcessor finish
>> INFO: {commit=} 0 1043
>> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore execute
>> INFO: [] webapp=/solr path=/update params={} status=0 QTime=1043
>> Jan 24, 2010 1:48:24 AM org.apache.solr.update.DirectUpdateHandler2
>> deleteAll
>> INFO: [] REMOVING ALL DOCUMENTS FROM INDEX
>> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy
>> onInit INFO: SolrDeletionPolicy.onInit: commits:num=1
>>
>> commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/ind
>> ex,segFN=segments_fs,version=1260734716753,generation=568,filename
>> s=[_gv.nrm, segments_fs, _gv.fdx, _gw.nrm, _gv.tii, _gv.prx,
>> _gv.tvf, _gv.tis, _gv.tvd, _gv.fdt, _gw.fnm, _gw.tis, _gw.frq,
>> _gv.fnm, _gw.prx, _gv.tvx, _gw.tii, _gv.frq]
>> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy
>> updateCommits
>> INFO: newest commit = 1260734716753
>> Jan 24, 2010 1:48:24 AM
>> org.apache.solr.update.processor.LogUpdateProcessor finish
>> INFO: {deleteByQuery=*:*} 0 5
>> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore execute
>> INFO: [] webapp=/solr path=/update params={} status=0 QTime=5
>> Jan 24, 2010 1:48:24 AM
>> org.apache.solr.update.DirectUpdateHandler2 commit INFO: start
>>
>> commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDele
>> tes=false) Jan 24, 2010 1:48:24 AM
>> org.apache.solr.core.SolrDeletionPolicy onCommit INFO:
>> SolrDeletionPolicy.onCommit: commits:num=2
>>
>
>
>
> --
> Best regards,
> Bogdan
Re: Index gets deleted after commit?
Posted by Bogdan Vatkov <bo...@gmail.com>.
Hi Amit,
What is DIH? (I am Solr newbie).
In the mean time I resolved my issue - it was very stupid one - on of the
files in my folder with XMLs (that I send to Solr with the SimplePostTool),
and actually the latest created one (so it got executed last each time I run
the folder), contaoned <delete>*:*.... :)
Best regards,
Bogdan
On Sun, Jan 24, 2010 at 6:25 AM, Amit Nithian <an...@gmail.com> wrote:
> Are you using the DIH? If so, did you try setting clean=false in the URL
> line? That prevents wiping out the index on load.
>
> On Jan 23, 2010 4:06 PM, "Bogdan Vatkov" <bo...@gmail.com> wrote:
>
> After mass upload of docs in Solr I get some "REMOVING ALL DOCUMENTS FROM
> INDEX" without any explanation.
>
> I was running indexing w/ Solr for several weeks now and everything was ok
> -
> I indexed 22K+ docs using the SimplePostTool
> I was first launching
>
> <delete><query>*:*</query></delete>
> <commit waitFlush="true" waitSearcher="true"/>
>
> then some 22K+ <Add>...
> with a finishing
> <commit waitFlush="true" waitSearcher="true"/>
>
> But you can see from the log - right after the last commit I get this
> strange REMOVING ALL...
> I do not remember what I changed last but now I have this issue that after
> the mass upload of docs the index gets completely deleted.
>
> why is this happening?
>
>
> log after the last commit:
>
> INFO: start
>
> commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy onCommit
> INFO: SolrDeletionPolicy.onCommit: commits:num=2
>
> commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/index,segFN=segments_fr,version=1260734716752,generation=567,filenames=[segments_fr]
>
> commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/index,segFN=segments_fs,version=1260734716753,generation=568,filenames=[_gv.nrm,
> segments_fs, _gv.fdx, _gw.nrm, _gv.tii, _gv.prx, _gv.tvf, _gv.tis, _gv.tvd,
> _gv.fdt, _gw.fnm, _gw.tis, _gw.frq, _gv.fnm, _gw.prx, _gv.tvx, _gw.tii,
> _gv.frq]
> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy
> updateCommits
> INFO: newest commit = 1260734716753
> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@de26e52 main
> Jan 24, 2010 1:48:24 AM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: end_commit_flush
> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@de26e52 main
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@de26e52 main
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
>
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@de26e52 main
>
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@de26e52 main
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Jan 24, 2010 1:48:24 AM org.apache.solr.core.QuerySenderListener
> newSearcher
> INFO: QuerySenderListener sending requests to Searcher@de26e52 main
> Jan 24, 2010 1:48:24 AM org.apache.solr.core.QuerySenderListener
> newSearcher
> INFO: QuerySenderListener done.
> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore registerSearcher
> INFO: [] Registered new searcher Searcher@de26e52 main
> Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher close
> INFO: Closing Searcher@4e8deb8a main
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Jan 24, 2010 1:48:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {commit=} 0 1043
> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={} status=0 QTime=1043
> Jan 24, 2010 1:48:24 AM org.apache.solr.update.DirectUpdateHandler2
> deleteAll
> INFO: [] REMOVING ALL DOCUMENTS FROM INDEX
> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy onInit
> INFO: SolrDeletionPolicy.onInit: commits:num=1
>
> commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/index,segFN=segments_fs,version=1260734716753,generation=568,filenames=[_gv.nrm,
> segments_fs, _gv.fdx, _gw.nrm, _gv.tii, _gv.prx, _gv.tvf, _gv.tis, _gv.tvd,
> _gv.fdt, _gw.fnm, _gw.tis, _gw.frq, _gv.fnm, _gw.prx, _gv.tvx, _gw.tii,
> _gv.frq]
> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy
> updateCommits
> INFO: newest commit = 1260734716753
> Jan 24, 2010 1:48:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {deleteByQuery=*:*} 0 5
> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={} status=0 QTime=5
> Jan 24, 2010 1:48:24 AM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: start
>
> commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
> Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy onCommit
> INFO: SolrDeletionPolicy.onCommit: commits:num=2
>
--
Best regards,
Bogdan
Re: Index gets deleted after commit?
Posted by Amit Nithian <an...@gmail.com>.
Are you using the DIH? If so, did you try setting clean=false in the URL
line? That prevents wiping out the index on load.
On Jan 23, 2010 4:06 PM, "Bogdan Vatkov" <bo...@gmail.com> wrote:
After mass upload of docs in Solr I get some "REMOVING ALL DOCUMENTS FROM
INDEX" without any explanation.
I was running indexing w/ Solr for several weeks now and everything was ok -
I indexed 22K+ docs using the SimplePostTool
I was first launching
<delete><query>*:*</query></delete>
<commit waitFlush="true" waitSearcher="true"/>
then some 22K+ <Add>...
with a finishing
<commit waitFlush="true" waitSearcher="true"/>
But you can see from the log - right after the last commit I get this
strange REMOVING ALL...
I do not remember what I changed last but now I have this issue that after
the mass upload of docs the index gets completely deleted.
why is this happening?
log after the last commit:
INFO: start
commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy onCommit
INFO: SolrDeletionPolicy.onCommit: commits:num=2
commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/index,segFN=segments_fr,version=1260734716752,generation=567,filenames=[segments_fr]
commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/index,segFN=segments_fs,version=1260734716753,generation=568,filenames=[_gv.nrm,
segments_fs, _gv.fdx, _gw.nrm, _gv.tii, _gv.prx, _gv.tvf, _gv.tis, _gv.tvd,
_gv.fdt, _gw.fnm, _gw.tis, _gw.frq, _gv.fnm, _gw.prx, _gv.tvx, _gw.tii,
_gv.frq]
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1260734716753
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@de26e52 main
Jan 24, 2010 1:48:24 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@de26e52 main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@de26e52 main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@de26e52 main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@de26e52 main from Searcher@4e8deb8a main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@de26e52 main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener sending requests to Searcher@de26e52 main
Jan 24, 2010 1:48:24 AM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore registerSearcher
INFO: [] Registered new searcher Searcher@de26e52 main
Jan 24, 2010 1:48:24 AM org.apache.solr.search.SolrIndexSearcher close
INFO: Closing Searcher@4e8deb8a main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Jan 24, 2010 1:48:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {commit=} 0 1043
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={} status=0 QTime=1043
Jan 24, 2010 1:48:24 AM org.apache.solr.update.DirectUpdateHandler2
deleteAll
INFO: [] REMOVING ALL DOCUMENTS FROM INDEX
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
commit{dir=/store/dev/inst/apache-solr-1.4.0/example/solr/data/index,segFN=segments_fs,version=1260734716753,generation=568,filenames=[_gv.nrm,
segments_fs, _gv.fdx, _gw.nrm, _gv.tii, _gv.prx, _gv.tvf, _gv.tis, _gv.tvd,
_gv.fdt, _gw.fnm, _gw.tis, _gw.frq, _gv.fnm, _gw.prx, _gv.tvx, _gw.tii,
_gv.frq]
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1260734716753
Jan 24, 2010 1:48:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {deleteByQuery=*:*} 0 5
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={} status=0 QTime=5
Jan 24, 2010 1:48:24 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
Jan 24, 2010 1:48:24 AM org.apache.solr.core.SolrDeletionPolicy onCommit
INFO: SolrDeletionPolicy.onCommit: commits:num=2