You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Kamal Palei <pa...@gmail.com> on 2013/11/26 12:15:11 UTC
Is there any limit how many documents can be indexed by apache solr
Dear All
I am using Apache solr 3.6.2 with Drupal 7.
Users keeps adding their profiles (resumes) and with cron task from Drupal,
documents get indexed.
Recently I observed, after indexing around 11,000 documents, further
documents are not getting indexed.
Is there any configuration for max documents those can be indexed.
Kindly help.
Thanks
kamal
Re: Is there any limit how many documents can be indexed by apache solr
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi Kamal;
You said that: "Recently I observed, after indexing around 11,000
documents, further documents are not getting indexed." How do you think
that documents are not indexed? There may occur an error and indexing
process may be break. Do you see any error at your log file? On the other
hand how your commit mechanism works. Some systems sends commit command at
the end of indexing and if any error occurs when indexing they can not
send a commit command. So user may think that documents are not received
from Solr. Also you can try that: when you think that Solr will not index
any more documents, start your indexing process and tail your log file at
same time. You can share your logs when you do that.
Thanks;
Furkan KAMACI
2013/11/26 Kamal Palei <pa...@gmail.com>
> Hi All
> I tried to get the log from terminal.
> The complete log I have put at the end of this email.
> One place in log I see, it is logged as
>
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=1
> Nov 26, 2013 5:46:25 AM
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
> buildSpellIndex
> INFO: Building spell index for spell checker: jarowinkler
> Nov 26, 2013 5:46:26 AM org.apache.solr.core.SolrCore registerSearcher
> INFO: [] Registered new searcher Searcher@119a0c4e main
> Nov 26, 2013 5:46:26 AM org.apache.solr.search.SolrIndexSearcher close
> INFO: Closing Searcher@35d22ddb main
>
> Though it is not error or something when indexing got stopped this is the
> log that was showing in my terminal.
>
> The complete log is as below. Can somebody help me how can I index more
> number of documents, currently stopping after indexing around 11,000
> documents.
>
> ov 26, 2013 5:46:16 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/node/5773, 8lftl7/node/5774, 8lftl7/node/5775,
> 8lftl7/node/5776, 8lftl7/node/5777, 8lftl7/node/5778, 8lftl7/node/5779,
> 8lftl7/node/5780, ... (20 adds)]} 0 3
> Nov 26, 2013 5:46:16 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=3
> Nov 26, 2013 5:46:16 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/node/5793, 8lftl7/node/5794, 8lftl7/node/5795,
> 8lftl7/node/5796, 8lftl7/node/5797, 8lftl7/node/5798, 8lftl7/node/5799,
> 8lftl7/node/5800, ... (10 adds)]} 0 2
> Nov 26, 2013 5:46:16 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=2
> Nov 26, 2013 5:46:16 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
> Nov 26, 2013 5:46:16 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> Nov 26, 2013 5:46:16 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/file/5750-node-6276, 8lftl7/file/5751-node-6277,
> 8lftl7/file/5752-node-6278, 8lftl7/file/5753-node-6279,
> 8lftl7/file/5754-node-6280, 8lftl7/file/5755-node-6281,
> 8lftl7/file/5756-node-6282, 8lftl7/file/5757-node-6283, ... (20 adds)]} 0
> 25
> Nov 26, 2013 5:46:16 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=25
> Nov 26, 2013 5:46:16 AM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: start
>
> commit(optimize=false,waitFlush=false,waitSearcher=true,expungeDeletes=false)
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrDeletionPolicy onCommit
> INFO: SolrDeletionPolicy.onCommit: commits:num=2
>
>
> commit{dir=/usr/local/src/apachesolr/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1o4,version=1362286025701,generation=2164,filenames=[_20f.prx,
> _20f.tii, _20h.tvx, _20f.fnm, _20g.tii, _20h.frq, _20g.nrm, _20h.fdt,
> _20f.tvd, _20g.tis, _20f.frq, _20f.tvf, _20h.fdx, _20h.fnm, _20g.tvx,
> _20h.tvf, _20h.tis, _20f.tis, segments_1o4, _20g.fdx, _20g.fnm, _20h.tii,
> _20g.tvf, _20g.fdt, _20f_1.del, _20g.tvd, _20f.fdt, _20f.fdx, _20g.frq,
> _20g.prx, _20f.tvx, _20f.nrm, _20h.tvd, _20h.prx, _20h.nrm]
>
>
> commit{dir=/usr/local/src/apachesolr/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1o5,version=1362286025705,generation=2165,filenames=[_20j.prx,
> _20j.tii, _20j.fdt, _20j.nrm, _20j.frq, _20j.fdx, segments_1o5, _20j.tvf,
> _20j.tvd, _20j.fnm, _20j.tvx, _20j.tis]
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrDeletionPolicy
> updateCommits
> INFO: newest commit = 1362286025705
> Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@119a0c4e main
> Nov 26, 2013 5:46:23 AM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: end_commit_flush
> Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@119a0c4e main from Searcher@35d22ddb main
>
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@119a0c4e main
>
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@119a0c4e main from Searcher@35d22ddb main
>
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@119a0c4e main
>
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@119a0c4e main from Searcher@35d22ddb main
>
>
> queryResultCache{lookups=155,hits=155,hitratio=1.00,inserts=6,evictions=0,size=6,warmupTime=11,cumulative_lookups=582,cumulative_hits=581,cumulative_hitratio=0.99,cumulative_inserts=1,cumulative_evictions=0}
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrDeletionPolicy onInit
> INFO: SolrDeletionPolicy.onInit: commits:num=1
>
>
> commit{dir=/usr/local/src/apachesolr/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1o5,version=1362286025705,generation=2165,filenames=[_20j.prx,
> _20j.tii, _20j.fdt, _20j.nrm, _20j.frq, _20j.fdx, segments_1o5, _20j.tvf,
> _20j.tvd, _20j.fnm, _20j.tvx, _20j.tis]
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrDeletionPolicy
> updateCommits
> INFO: newest commit = 1362286025705
> Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@119a0c4e main
>
>
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=6,evictions=0,size=6,warmupTime=8,cumulative_lookups=582,cumulative_hits=581,cumulative_hitratio=0.99,cumulative_inserts=1,cumulative_evictions=0}
> Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@119a0c4e main from Searcher@35d22ddb main
>
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=30,evictions=0,size=30,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@119a0c4e main
>
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.QuerySenderListener
> newSearcher
> INFO: QuerySenderListener sending requests to Searcher@119a0c4e main
> Nov 26, 2013 5:46:23 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/file/5770-node-6296, 8lftl7/file/5771-node-6297,
> 8lftl7/file/5772-node-6298, 8lftl7/file/5773-node-6299,
> 8lftl7/file/5774-node-6300, 8lftl7/file/5775-node-6301,
> 8lftl7/file/5776-node-6302, 8lftl7/file/5777-node-6303, ... (20 adds)]} 0
> 6692
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=6692
> Nov 26, 2013 5:46:23 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/file/5790-node-6316, 8lftl7/file/5791-node-6317,
> 8lftl7/file/5792-node-6318, 8lftl7/file/5793-node-6319,
> 8lftl7/file/5794-node-6320, 8lftl7/file/5795-node-6321,
> 8lftl7/file/5796-node-6322, 8lftl7/file/5797-node-6323, ... (10 adds)]} 0
> 13
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=13
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=null path=null
> params={start=0&event=newSearcher&q=solr&rows=10} hits=11 status=0 QTime=39
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=0
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=null path=null
> params={start=0&event=newSearcher&q=rocks&rows=10} hits=38 status=0
> QTime=26
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=null path=null
>
> params={event=newSearcher&q=static+newSearcher+warming+query+from+solrconfig.xml}
> hits=4958 status=0 QTime=27
> Nov 26, 2013 5:46:23 AM org.apache.solr.core.QuerySenderListener
> newSearcher
> INFO: QuerySenderListener done.
> Nov 26, 2013 5:46:23 AM
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
> buildSpellIndex
> INFO: Building spell index for spell checker: default
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/node/5803, 8lftl7/node/5804, 8lftl7/node/5805,
> 8lftl7/node/5806, 8lftl7/node/5807, 8lftl7/node/5808, 8lftl7/node/5809,
> 8lftl7/node/5810, ... (20 adds)]} 0 3
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=3
> Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/node/5823, 8lftl7/node/5824, 8lftl7/node/5825,
> 8lftl7/node/5826, 8lftl7/node/5827, 8lftl7/node/5828, 8lftl7/node/5829,
> 8lftl7/node/5830, ... (20 adds)]} 0 3
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=3
> Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/node/5843, 8lftl7/node/5844, 8lftl7/node/5845,
> 8lftl7/node/5846, 8lftl7/node/5847, 8lftl7/node/5848, 8lftl7/node/5849,
> 8lftl7/node/5850, ... (10 adds)]} 0 1
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=1
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/file/5800-node-6326, 8lftl7/file/5801-node-6327,
> 8lftl7/file/5802-node-6328, 8lftl7/file/5803-node-6329,
> 8lftl7/file/5804-node-6330, 8lftl7/file/5805-node-6331,
> 8lftl7/file/5806-node-6332, 8lftl7/file/5807-node-6333, ... (20 adds)]} 0
> 20
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=20
> Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/file/5820-node-6346, 8lftl7/file/5821-node-6347,
> 8lftl7/file/5822-node-6348, 8lftl7/file/5823-node-6349,
> 8lftl7/file/5824-node-6350, 8lftl7/file/5825-node-6351,
> 8lftl7/file/5826-node-6352, 8lftl7/file/5827-node-6353, ... (20 adds)]} 0
> 22
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=22
> Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/file/5840-node-6366, 8lftl7/file/5841-node-6367,
> 8lftl7/file/5842-node-6368, 8lftl7/file/5843-node-6369,
> 8lftl7/file/5844-node-6370, 8lftl7/file/5845-node-6371,
> 8lftl7/file/5846-node-6372, 8lftl7/file/5847-node-6373, ... (10 adds)]} 0
> 13
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=13
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=0
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=0
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=0
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=0
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=0
> Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/node/5853, 8lftl7/node/5854, 8lftl7/node/5855,
> 8lftl7/node/5856, 8lftl7/node/5857, 8lftl7/node/5858, 8lftl7/node/5859,
> 8lftl7/node/5860, ... (20 adds)]} 0 3
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=3
> Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/node/5873, 8lftl7/node/5874, 8lftl7/node/5875,
> 8lftl7/node/5876, 8lftl7/node/5877, 8lftl7/node/5878, 8lftl7/node/5879,
> 8lftl7/node/5880, ... (20 adds)]} 0 2
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=2
> Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {add=[8lftl7/node/5893, 8lftl7/node/5894, 8lftl7/node/5895,
> 8lftl7/node/5896, 8lftl7/node/5897, 8lftl7/node/5898, 8lftl7/node/5899,
> 8lftl7/node/5900, ... (10 adds)]} 0 1
> Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=1
> Nov 26, 2013 5:46:25 AM
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
> buildSpellIndex
> INFO: Building spell index for spell checker: jarowinkler
> Nov 26, 2013 5:46:26 AM org.apache.solr.core.SolrCore registerSearcher
> INFO: [] Registered new searcher Searcher@119a0c4e main
> Nov 26, 2013 5:46:26 AM org.apache.solr.search.SolrIndexSearcher close
> INFO: Closing Searcher@35d22ddb main
>
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>
>
> queryResultCache{lookups=164,hits=164,hitratio=1.00,inserts=6,evictions=0,size=6,warmupTime=11,cumulative_lookups=591,cumulative_hits=590,cumulative_hitratio=0.99,cumulative_inserts=1,cumulative_evictions=0}
>
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=30,evictions=0,size=30,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:53 AM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: start
>
> commit(optimize=false,waitFlush=false,waitSearcher=true,expungeDeletes=false)
> Nov 26, 2013 5:46:53 AM org.apache.solr.core.SolrDeletionPolicy onCommit
> INFO: SolrDeletionPolicy.onCommit: commits:num=2
>
>
> commit{dir=/usr/local/src/apachesolr/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1o5,version=1362286025705,generation=2165,filenames=[_20j.prx,
> _20j.tii, _20j.fdt, _20j.nrm, _20j.frq, _20j.fdx, segments_1o5, _20j.tvf,
> _20j.tvd, _20j.fnm, _20j.tvx, _20j.tis]
>
>
> commit{dir=/usr/local/src/apachesolr/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1o6,version=1362286025707,generation=2166,filenames=[_20k.tvx,
> _20k.fdx, _20k.tii, _20j.frq, _20k.prx, _20k.fnm, _20k.frq, _20k.tis,
> _20k.fdt, _20j.tvx, _20j.fnm, _20j.prx, _20j.tii, _20k.tvf, _20k.tvd,
> _20j.fdt, _20j.nrm, _20j.fdx, segments_1o6, _20j.tvf, _20j.tvd, _20k.nrm,
> _20j.tis]
> Nov 26, 2013 5:46:53 AM org.apache.solr.core.SolrDeletionPolicy
> updateCommits
> INFO: newest commit = 1362286025707
> Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@1872c950 main
> Nov 26, 2013 5:46:53 AM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: end_commit_flush
> Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@1872c950 main from Searcher@119a0c4e main
>
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@1872c950 main
>
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@1872c950 main from Searcher@119a0c4e main
>
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@1872c950 main
>
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@1872c950 main from Searcher@119a0c4e main
>
>
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=6,evictions=0,size=6,warmupTime=8,cumulative_lookups=591,cumulative_hits=590,cumulative_hitratio=0.99,cumulative_inserts=1,cumulative_evictions=0}
> Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@1872c950 main
>
>
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=6,evictions=0,size=6,warmupTime=6,cumulative_lookups=591,cumulative_hits=590,cumulative_hitratio=0.99,cumulative_inserts=1,cumulative_evictions=0}
> Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@1872c950 main from Searcher@119a0c4e main
>
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=30,evictions=0,size=30,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@1872c950 main
>
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> Nov 26, 2013 5:46:53 AM org.apache.solr.core.QuerySenderListener
> newSearcher
> INFO: QuerySenderListener sending requests to Searcher@1872c950 main
> Nov 26, 2013 5:46:53 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=null path=null
> params={start=0&event=newSearcher&q=solr&rows=10} hits=11 status=0 QTime=25
> Nov 26, 2013 5:46:53 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=null path=null
> params={start=0&event=newSearcher&q=rocks&rows=10} hits=38 status=0
> QTime=15
>
>
> Thanks
> kamal
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Nov 26, 2013 at 6:19 PM, Kamal Palei <pa...@gmail.com>
> wrote:
>
> > Thanks Alejandro and Luis.
> > If I need to see logs, how can I see it. Is it stored in any default log
> > files.
> >
> > I am using below command to start apache solr.
> >
> > java -Xms64m -Xmx6g -jar start.jar &
> >
> > I am using it along with Drupal 7.1.5 , I am trying to find out if it is
> a
> > Drupal issue or Apache solr issue.
> >
> > Not getting clue where to start ....
> >
> > In terminal where I have started apache solr, I get below logs when I
> > attempt to index for remaining documents.
> >
> >
> > [root@example]#
> > [root@example]#
> > [root@example]# Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore
> > execute
> > INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
> > Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore execute
> > INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> > Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore execute
> > INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
> > Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore execute
> > INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> >
> > [root@example]#
> >
> >
> > In addition to this, is there any other log folder...
> >
> > Kindly bear with me,as I am very novice in apache solr.
> >
> > Regards
> > kamal
> >
> >
> >
> > On Tue, Nov 26, 2013 at 5:19 PM, Luis Cappa Banda <luiscappa@gmail.com
> >wrote:
> >
> >> Hello!
> >>
> >> Checkout also your application server logs. Maybe you're trying to index
> >> Documents with any syntax error and they are skipped.
> >>
> >> Regards,
> >>
> >> - Luis Cappa
> >>
> >>
> >> 2013/11/26 Alejandro Marqués Rodríguez <
> amarques@paradigmatecnologico.com
> >> >
> >>
> >> > Hi,
> >> >
> >> > In lucene you are supossed to be able to index up to 274 billion
> >> documents
> >> > ( http://lucene.apache.org/core/3_0_3/fileformats.html#Limitations ),
> >> so
> >> > in
> >> > Solr should be something like that. Anyway the maximum number is quite
> >> > bigger than those 11.000 ;)
> >> >
> >> > Could it be that you are reusing IDs so the new documents overwrite
> the
> >> old
> >> > ones?
> >> >
> >> >
> >> > 2013/11/26 Kamal Palei <pa...@gmail.com>
> >> >
> >> > > Dear All
> >> > > I am using Apache solr 3.6.2 with Drupal 7.
> >> > > Users keeps adding their profiles (resumes) and with cron task from
> >> > Drupal,
> >> > > documents get indexed.
> >> > >
> >> > > Recently I observed, after indexing around 11,000 documents, further
> >> > > documents are not getting indexed.
> >> > >
> >> > > Is there any configuration for max documents those can be indexed.
> >> > >
> >> > > Kindly help.
> >> > >
> >> > > Thanks
> >> > > kamal
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Alejandro Marqués Rodríguez
> >> >
> >> > Paradigma Tecnológico
> >> > http://www.paradigmatecnologico.com
> >> > Avenida de Europa, 26. Ática 5. 3ª Planta
> >> > 28224 Pozuelo de Alarcón
> >> > Tel.: 91 352 59 42
> >> >
> >>
> >>
> >>
> >> --
> >> - Luis Cappa
> >>
> >
> >
>
Re: Is there any limit how many documents can be indexed by apache solr
Posted by Kamal Palei <pa...@gmail.com>.
Hi All
I tried to get the log from terminal.
The complete log I have put at the end of this email.
One place in log I see, it is logged as
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=1
Nov 26, 2013 5:46:25 AM
org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
buildSpellIndex
INFO: Building spell index for spell checker: jarowinkler
Nov 26, 2013 5:46:26 AM org.apache.solr.core.SolrCore registerSearcher
INFO: [] Registered new searcher Searcher@119a0c4e main
Nov 26, 2013 5:46:26 AM org.apache.solr.search.SolrIndexSearcher close
INFO: Closing Searcher@35d22ddb main
Though it is not error or something when indexing got stopped this is the
log that was showing in my terminal.
The complete log is as below. Can somebody help me how can I index more
number of documents, currently stopping after indexing around 11,000
documents.
ov 26, 2013 5:46:16 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/node/5773, 8lftl7/node/5774, 8lftl7/node/5775,
8lftl7/node/5776, 8lftl7/node/5777, 8lftl7/node/5778, 8lftl7/node/5779,
8lftl7/node/5780, ... (20 adds)]} 0 3
Nov 26, 2013 5:46:16 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=3
Nov 26, 2013 5:46:16 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/node/5793, 8lftl7/node/5794, 8lftl7/node/5795,
8lftl7/node/5796, 8lftl7/node/5797, 8lftl7/node/5798, 8lftl7/node/5799,
8lftl7/node/5800, ... (10 adds)]} 0 2
Nov 26, 2013 5:46:16 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=2
Nov 26, 2013 5:46:16 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
Nov 26, 2013 5:46:16 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
Nov 26, 2013 5:46:16 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/file/5750-node-6276, 8lftl7/file/5751-node-6277,
8lftl7/file/5752-node-6278, 8lftl7/file/5753-node-6279,
8lftl7/file/5754-node-6280, 8lftl7/file/5755-node-6281,
8lftl7/file/5756-node-6282, 8lftl7/file/5757-node-6283, ... (20 adds)]} 0 25
Nov 26, 2013 5:46:16 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=25
Nov 26, 2013 5:46:16 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit(optimize=false,waitFlush=false,waitSearcher=true,expungeDeletes=false)
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrDeletionPolicy onCommit
INFO: SolrDeletionPolicy.onCommit: commits:num=2
commit{dir=/usr/local/src/apachesolr/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1o4,version=1362286025701,generation=2164,filenames=[_20f.prx,
_20f.tii, _20h.tvx, _20f.fnm, _20g.tii, _20h.frq, _20g.nrm, _20h.fdt,
_20f.tvd, _20g.tis, _20f.frq, _20f.tvf, _20h.fdx, _20h.fnm, _20g.tvx,
_20h.tvf, _20h.tis, _20f.tis, segments_1o4, _20g.fdx, _20g.fnm, _20h.tii,
_20g.tvf, _20g.fdt, _20f_1.del, _20g.tvd, _20f.fdt, _20f.fdx, _20g.frq,
_20g.prx, _20f.tvx, _20f.nrm, _20h.tvd, _20h.prx, _20h.nrm]
commit{dir=/usr/local/src/apachesolr/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1o5,version=1362286025705,generation=2165,filenames=[_20j.prx,
_20j.tii, _20j.fdt, _20j.nrm, _20j.frq, _20j.fdx, segments_1o5, _20j.tvf,
_20j.tvd, _20j.fnm, _20j.tvx, _20j.tis]
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1362286025705
Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@119a0c4e main
Nov 26, 2013 5:46:23 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@119a0c4e main from Searcher@35d22ddb main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@119a0c4e main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@119a0c4e main from Searcher@35d22ddb main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@119a0c4e main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@119a0c4e main from Searcher@35d22ddb main
queryResultCache{lookups=155,hits=155,hitratio=1.00,inserts=6,evictions=0,size=6,warmupTime=11,cumulative_lookups=582,cumulative_hits=581,cumulative_hitratio=0.99,cumulative_inserts=1,cumulative_evictions=0}
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
commit{dir=/usr/local/src/apachesolr/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1o5,version=1362286025705,generation=2165,filenames=[_20j.prx,
_20j.tii, _20j.fdt, _20j.nrm, _20j.frq, _20j.fdx, segments_1o5, _20j.tvf,
_20j.tvd, _20j.fnm, _20j.tvx, _20j.tis]
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1362286025705
Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@119a0c4e main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=6,evictions=0,size=6,warmupTime=8,cumulative_lookups=582,cumulative_hits=581,cumulative_hitratio=0.99,cumulative_inserts=1,cumulative_evictions=0}
Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@119a0c4e main from Searcher@35d22ddb main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=30,evictions=0,size=30,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:23 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@119a0c4e main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:23 AM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener sending requests to Searcher@119a0c4e main
Nov 26, 2013 5:46:23 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/file/5770-node-6296, 8lftl7/file/5771-node-6297,
8lftl7/file/5772-node-6298, 8lftl7/file/5773-node-6299,
8lftl7/file/5774-node-6300, 8lftl7/file/5775-node-6301,
8lftl7/file/5776-node-6302, 8lftl7/file/5777-node-6303, ... (20 adds)]} 0
6692
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=6692
Nov 26, 2013 5:46:23 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/file/5790-node-6316, 8lftl7/file/5791-node-6317,
8lftl7/file/5792-node-6318, 8lftl7/file/5793-node-6319,
8lftl7/file/5794-node-6320, 8lftl7/file/5795-node-6321,
8lftl7/file/5796-node-6322, 8lftl7/file/5797-node-6323, ... (10 adds)]} 0 13
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=13
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null
params={start=0&event=newSearcher&q=solr&rows=10} hits=11 status=0 QTime=39
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=0
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null
params={start=0&event=newSearcher&q=rocks&rows=10} hits=38 status=0 QTime=26
Nov 26, 2013 5:46:23 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null
params={event=newSearcher&q=static+newSearcher+warming+query+from+solrconfig.xml}
hits=4958 status=0 QTime=27
Nov 26, 2013 5:46:23 AM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Nov 26, 2013 5:46:23 AM
org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
buildSpellIndex
INFO: Building spell index for spell checker: default
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/node/5803, 8lftl7/node/5804, 8lftl7/node/5805,
8lftl7/node/5806, 8lftl7/node/5807, 8lftl7/node/5808, 8lftl7/node/5809,
8lftl7/node/5810, ... (20 adds)]} 0 3
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=3
Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/node/5823, 8lftl7/node/5824, 8lftl7/node/5825,
8lftl7/node/5826, 8lftl7/node/5827, 8lftl7/node/5828, 8lftl7/node/5829,
8lftl7/node/5830, ... (20 adds)]} 0 3
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=3
Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/node/5843, 8lftl7/node/5844, 8lftl7/node/5845,
8lftl7/node/5846, 8lftl7/node/5847, 8lftl7/node/5848, 8lftl7/node/5849,
8lftl7/node/5850, ... (10 adds)]} 0 1
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=1
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/file/5800-node-6326, 8lftl7/file/5801-node-6327,
8lftl7/file/5802-node-6328, 8lftl7/file/5803-node-6329,
8lftl7/file/5804-node-6330, 8lftl7/file/5805-node-6331,
8lftl7/file/5806-node-6332, 8lftl7/file/5807-node-6333, ... (20 adds)]} 0 20
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=20
Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/file/5820-node-6346, 8lftl7/file/5821-node-6347,
8lftl7/file/5822-node-6348, 8lftl7/file/5823-node-6349,
8lftl7/file/5824-node-6350, 8lftl7/file/5825-node-6351,
8lftl7/file/5826-node-6352, 8lftl7/file/5827-node-6353, ... (20 adds)]} 0 22
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=22
Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/file/5840-node-6366, 8lftl7/file/5841-node-6367,
8lftl7/file/5842-node-6368, 8lftl7/file/5843-node-6369,
8lftl7/file/5844-node-6370, 8lftl7/file/5845-node-6371,
8lftl7/file/5846-node-6372, 8lftl7/file/5847-node-6373, ... (10 adds)]} 0 13
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=13
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=0
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=0
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=0
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=0
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=0
Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/node/5853, 8lftl7/node/5854, 8lftl7/node/5855,
8lftl7/node/5856, 8lftl7/node/5857, 8lftl7/node/5858, 8lftl7/node/5859,
8lftl7/node/5860, ... (20 adds)]} 0 3
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=3
Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/node/5873, 8lftl7/node/5874, 8lftl7/node/5875,
8lftl7/node/5876, 8lftl7/node/5877, 8lftl7/node/5878, 8lftl7/node/5879,
8lftl7/node/5880, ... (20 adds)]} 0 2
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=2
Nov 26, 2013 5:46:24 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[8lftl7/node/5893, 8lftl7/node/5894, 8lftl7/node/5895,
8lftl7/node/5896, 8lftl7/node/5897, 8lftl7/node/5898, 8lftl7/node/5899,
8lftl7/node/5900, ... (10 adds)]} 0 1
Nov 26, 2013 5:46:24 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={wt=json} status=0 QTime=1
Nov 26, 2013 5:46:25 AM
org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener
buildSpellIndex
INFO: Building spell index for spell checker: jarowinkler
Nov 26, 2013 5:46:26 AM org.apache.solr.core.SolrCore registerSearcher
INFO: [] Registered new searcher Searcher@119a0c4e main
Nov 26, 2013 5:46:26 AM org.apache.solr.search.SolrIndexSearcher close
INFO: Closing Searcher@35d22ddb main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
queryResultCache{lookups=164,hits=164,hitratio=1.00,inserts=6,evictions=0,size=6,warmupTime=11,cumulative_lookups=591,cumulative_hits=590,cumulative_hitratio=0.99,cumulative_inserts=1,cumulative_evictions=0}
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=30,evictions=0,size=30,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:53 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit(optimize=false,waitFlush=false,waitSearcher=true,expungeDeletes=false)
Nov 26, 2013 5:46:53 AM org.apache.solr.core.SolrDeletionPolicy onCommit
INFO: SolrDeletionPolicy.onCommit: commits:num=2
commit{dir=/usr/local/src/apachesolr/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1o5,version=1362286025705,generation=2165,filenames=[_20j.prx,
_20j.tii, _20j.fdt, _20j.nrm, _20j.frq, _20j.fdx, segments_1o5, _20j.tvf,
_20j.tvd, _20j.fnm, _20j.tvx, _20j.tis]
commit{dir=/usr/local/src/apachesolr/apache-solr-3.6.2/example/solr/data/index,segFN=segments_1o6,version=1362286025707,generation=2166,filenames=[_20k.tvx,
_20k.fdx, _20k.tii, _20j.frq, _20k.prx, _20k.fnm, _20k.frq, _20k.tis,
_20k.fdt, _20j.tvx, _20j.fnm, _20j.prx, _20j.tii, _20k.tvf, _20k.tvd,
_20j.fdt, _20j.nrm, _20j.fdx, segments_1o6, _20j.tvf, _20j.tvd, _20k.nrm,
_20j.tis]
Nov 26, 2013 5:46:53 AM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1362286025707
Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@1872c950 main
Nov 26, 2013 5:46:53 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1872c950 main from Searcher@119a0c4e main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1872c950 main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1872c950 main from Searcher@119a0c4e main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1872c950 main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1872c950 main from Searcher@119a0c4e main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=6,evictions=0,size=6,warmupTime=8,cumulative_lookups=591,cumulative_hits=590,cumulative_hitratio=0.99,cumulative_inserts=1,cumulative_evictions=0}
Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1872c950 main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=6,evictions=0,size=6,warmupTime=6,cumulative_lookups=591,cumulative_hits=590,cumulative_hitratio=0.99,cumulative_inserts=1,cumulative_evictions=0}
Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1872c950 main from Searcher@119a0c4e main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=30,evictions=0,size=30,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:53 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1872c950 main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 26, 2013 5:46:53 AM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener sending requests to Searcher@1872c950 main
Nov 26, 2013 5:46:53 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null
params={start=0&event=newSearcher&q=solr&rows=10} hits=11 status=0 QTime=25
Nov 26, 2013 5:46:53 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null
params={start=0&event=newSearcher&q=rocks&rows=10} hits=38 status=0 QTime=15
Thanks
kamal
On Tue, Nov 26, 2013 at 6:19 PM, Kamal Palei <pa...@gmail.com> wrote:
> Thanks Alejandro and Luis.
> If I need to see logs, how can I see it. Is it stored in any default log
> files.
>
> I am using below command to start apache solr.
>
> java -Xms64m -Xmx6g -jar start.jar &
>
> I am using it along with Drupal 7.1.5 , I am trying to find out if it is a
> Drupal issue or Apache solr issue.
>
> Not getting clue where to start ....
>
> In terminal where I have started apache solr, I get below logs when I
> attempt to index for remaining documents.
>
>
> [root@example]#
> [root@example]#
> [root@example]# Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore
> execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
> Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
> Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
> Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
>
> [root@example]#
>
>
> In addition to this, is there any other log folder...
>
> Kindly bear with me,as I am very novice in apache solr.
>
> Regards
> kamal
>
>
>
> On Tue, Nov 26, 2013 at 5:19 PM, Luis Cappa Banda <lu...@gmail.com>wrote:
>
>> Hello!
>>
>> Checkout also your application server logs. Maybe you're trying to index
>> Documents with any syntax error and they are skipped.
>>
>> Regards,
>>
>> - Luis Cappa
>>
>>
>> 2013/11/26 Alejandro Marqués Rodríguez <amarques@paradigmatecnologico.com
>> >
>>
>> > Hi,
>> >
>> > In lucene you are supossed to be able to index up to 274 billion
>> documents
>> > ( http://lucene.apache.org/core/3_0_3/fileformats.html#Limitations ),
>> so
>> > in
>> > Solr should be something like that. Anyway the maximum number is quite
>> > bigger than those 11.000 ;)
>> >
>> > Could it be that you are reusing IDs so the new documents overwrite the
>> old
>> > ones?
>> >
>> >
>> > 2013/11/26 Kamal Palei <pa...@gmail.com>
>> >
>> > > Dear All
>> > > I am using Apache solr 3.6.2 with Drupal 7.
>> > > Users keeps adding their profiles (resumes) and with cron task from
>> > Drupal,
>> > > documents get indexed.
>> > >
>> > > Recently I observed, after indexing around 11,000 documents, further
>> > > documents are not getting indexed.
>> > >
>> > > Is there any configuration for max documents those can be indexed.
>> > >
>> > > Kindly help.
>> > >
>> > > Thanks
>> > > kamal
>> > >
>> >
>> >
>> >
>> > --
>> > Alejandro Marqués Rodríguez
>> >
>> > Paradigma Tecnológico
>> > http://www.paradigmatecnologico.com
>> > Avenida de Europa, 26. Ática 5. 3ª Planta
>> > 28224 Pozuelo de Alarcón
>> > Tel.: 91 352 59 42
>> >
>>
>>
>>
>> --
>> - Luis Cappa
>>
>
>
Re: Is there any limit how many documents can be indexed by apache solr
Posted by Kamal Palei <pa...@gmail.com>.
Thanks Alejandro and Luis.
If I need to see logs, how can I see it. Is it stored in any default log
files.
I am using below command to start apache solr.
java -Xms64m -Xmx6g -jar start.jar &
I am using it along with Drupal 7.1.5 , I am trying to find out if it is a
Drupal issue or Apache solr issue.
Not getting clue where to start ....
In terminal where I have started apache solr, I get below logs when I
attempt to index for remaining documents.
[root@example]#
[root@example]#
[root@example]# Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore
execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1
Nov 26, 2013 5:28:56 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/admin/ping params={} status=0 QTime=1
[root@example]#
In addition to this, is there any other log folder...
Kindly bear with me,as I am very novice in apache solr.
Regards
kamal
On Tue, Nov 26, 2013 at 5:19 PM, Luis Cappa Banda <lu...@gmail.com>wrote:
> Hello!
>
> Checkout also your application server logs. Maybe you're trying to index
> Documents with any syntax error and they are skipped.
>
> Regards,
>
> - Luis Cappa
>
>
> 2013/11/26 Alejandro Marqués Rodríguez <am...@paradigmatecnologico.com>
>
> > Hi,
> >
> > In lucene you are supossed to be able to index up to 274 billion
> documents
> > ( http://lucene.apache.org/core/3_0_3/fileformats.html#Limitations ), so
> > in
> > Solr should be something like that. Anyway the maximum number is quite
> > bigger than those 11.000 ;)
> >
> > Could it be that you are reusing IDs so the new documents overwrite the
> old
> > ones?
> >
> >
> > 2013/11/26 Kamal Palei <pa...@gmail.com>
> >
> > > Dear All
> > > I am using Apache solr 3.6.2 with Drupal 7.
> > > Users keeps adding their profiles (resumes) and with cron task from
> > Drupal,
> > > documents get indexed.
> > >
> > > Recently I observed, after indexing around 11,000 documents, further
> > > documents are not getting indexed.
> > >
> > > Is there any configuration for max documents those can be indexed.
> > >
> > > Kindly help.
> > >
> > > Thanks
> > > kamal
> > >
> >
> >
> >
> > --
> > Alejandro Marqués Rodríguez
> >
> > Paradigma Tecnológico
> > http://www.paradigmatecnologico.com
> > Avenida de Europa, 26. Ática 5. 3ª Planta
> > 28224 Pozuelo de Alarcón
> > Tel.: 91 352 59 42
> >
>
>
>
> --
> - Luis Cappa
>
Re: Is there any limit how many documents can be indexed by apache solr
Posted by Luis Cappa Banda <lu...@gmail.com>.
Hello!
Checkout also your application server logs. Maybe you're trying to index
Documents with any syntax error and they are skipped.
Regards,
- Luis Cappa
2013/11/26 Alejandro Marqués Rodríguez <am...@paradigmatecnologico.com>
> Hi,
>
> In lucene you are supossed to be able to index up to 274 billion documents
> ( http://lucene.apache.org/core/3_0_3/fileformats.html#Limitations ), so
> in
> Solr should be something like that. Anyway the maximum number is quite
> bigger than those 11.000 ;)
>
> Could it be that you are reusing IDs so the new documents overwrite the old
> ones?
>
>
> 2013/11/26 Kamal Palei <pa...@gmail.com>
>
> > Dear All
> > I am using Apache solr 3.6.2 with Drupal 7.
> > Users keeps adding their profiles (resumes) and with cron task from
> Drupal,
> > documents get indexed.
> >
> > Recently I observed, after indexing around 11,000 documents, further
> > documents are not getting indexed.
> >
> > Is there any configuration for max documents those can be indexed.
> >
> > Kindly help.
> >
> > Thanks
> > kamal
> >
>
>
>
> --
> Alejandro Marqués Rodríguez
>
> Paradigma Tecnológico
> http://www.paradigmatecnologico.com
> Avenida de Europa, 26. Ática 5. 3ª Planta
> 28224 Pozuelo de Alarcón
> Tel.: 91 352 59 42
>
--
- Luis Cappa
Re: Is there any limit how many documents can be indexed by apache solr
Posted by Alejandro Marqués Rodríguez <am...@paradigmatecnologico.com>.
Hi,
In lucene you are supossed to be able to index up to 274 billion documents
( http://lucene.apache.org/core/3_0_3/fileformats.html#Limitations ), so in
Solr should be something like that. Anyway the maximum number is quite
bigger than those 11.000 ;)
Could it be that you are reusing IDs so the new documents overwrite the old
ones?
2013/11/26 Kamal Palei <pa...@gmail.com>
> Dear All
> I am using Apache solr 3.6.2 with Drupal 7.
> Users keeps adding their profiles (resumes) and with cron task from Drupal,
> documents get indexed.
>
> Recently I observed, after indexing around 11,000 documents, further
> documents are not getting indexed.
>
> Is there any configuration for max documents those can be indexed.
>
> Kindly help.
>
> Thanks
> kamal
>
--
Alejandro Marqués Rodríguez
Paradigma Tecnológico
http://www.paradigmatecnologico.com
Avenida de Europa, 26. Ática 5. 3ª Planta
28224 Pozuelo de Alarcón
Tel.: 91 352 59 42