You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Lucas F. A. Teixeira" <lu...@gmail.com> on 2009/08/17 16:47:15 UTC
DIH opening searchers for every doc.
Hello all,
I'm trying Data Import Handler for the first time to generate my index based
on my db.
Looking the server's logs, I can see the index process is opening a new
searcher for every doc. Is this what we should expect? why? If not, how can
I avoid it? I think if this wasn't being done, could increase indexing
speed.
Thanks,
INFO: Opening Searcher@e07ac8 main
Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@b37a79 main
Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@f7e7e6 main
Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@37bd6a main
Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@ec9b6e main
Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@6892fd main
Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@967af7 main
Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@4e8de main
Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@aa8a6c main
Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@c27c36 main
Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
Lucas Frare Teixeira .·.
- lucastex@gmail.com
- blog.lucastex.com
- twitter.com/lucastex
Re: DIH opening searchers for every doc.
Posted by "Lucas F. A. Teixeira" <lu...@gmail.com>.
No I don't. It's commented.
This is giving me 40 docs/sec indexing, witch is very a poor rate.
(I know this rate depends in a lot of things, including that my database is
not in the same network and other stuff, but I think I can get more than
this).
Any clues on what is probably happening to open this searchers?
Thanks!
[]s,
Lucas Frare Teixeira .·.
- lucastex@gmail.com
- blog.lucastex.com
- twitter.com/lucastex
2009/8/17 Noble Paul നോബിള് नोब्ळ् <no...@corp.aol.com>
> DIH does not open searchers for each doc. Do you have any autocommit
> enabled?
>
> On Mon, Aug 17, 2009 at 8:17 PM, Lucas F. A. Teixeira<lu...@gmail.com>
> wrote:
> > Hello all,
> >
> > I'm trying Data Import Handler for the first time to generate my index
> based
> > on my db.
> > Looking the server's logs, I can see the index process is opening a new
> > searcher for every doc. Is this what we should expect? why? If not, how
> can
> > I avoid it? I think if this wasn't being done, could increase indexing
> > speed.
> >
> > Thanks,
> >
> > INFO: Opening Searcher@e07ac8 main
> > Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening Searcher@b37a79 main
> > Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening Searcher@f7e7e6 main
> > Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening Searcher@37bd6a main
> > Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening Searcher@ec9b6e main
> > Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening Searcher@6892fd main
> > Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening Searcher@967af7 main
> > Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening Searcher@4e8de main
> > Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening Searcher@aa8a6c main
> > Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening Searcher@c27c36 main
> > Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> >
> >
> >
> >
> > Lucas Frare Teixeira .·.
> > - lucastex@gmail.com
> > - blog.lucastex.com
> > - twitter.com/lucastex
> >
>
>
>
> --
> -----------------------------------------------------
> Noble Paul | Principal Engineer| AOL | http://aol.com
>
Re: DIH opening searchers for every doc.
Posted by Noble Paul നോബിള് नोब्ळ् <no...@corp.aol.com>.
DIH does not open searchers for each doc. Do you have any autocommit enabled?
On Mon, Aug 17, 2009 at 8:17 PM, Lucas F. A. Teixeira<lu...@gmail.com> wrote:
> Hello all,
>
> I'm trying Data Import Handler for the first time to generate my index based
> on my db.
> Looking the server's logs, I can see the index process is opening a new
> searcher for every doc. Is this what we should expect? why? If not, how can
> I avoid it? I think if this wasn't being done, could increase indexing
> speed.
>
> Thanks,
>
> INFO: Opening Searcher@e07ac8 main
> Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@b37a79 main
> Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@f7e7e6 main
> Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@37bd6a main
> Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@ec9b6e main
> Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@6892fd main
> Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@967af7 main
> Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@4e8de main
> Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@aa8a6c main
> Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@c27c36 main
> Aug 17, 2009 11:46:03 AM org.apache.solr.search.SolrIndexSearcher <init>
>
>
>
>
> Lucas Frare Teixeira .·.
> - lucastex@gmail.com
> - blog.lucastex.com
> - twitter.com/lucastex
>
--
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com