You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Aniket Bhoi <an...@gmail.com> on 2014/07/25 17:02:29 UTC
Solr Full Import frozen after indexing a fixed number of records
I have Apache Solr,hosted on my apache Tomcat Server with SQLServer Backend.
Details:
*Solr Version:*
Solr Specification Version: 3.4.0.2012.01.23.14.08.01
Solr Implementation Version: 3.4
Lucene Specification Version: 3.4
Lucene Implementation Version: 3.4
*Tomcat version:*
Apache Tomcat/6.0.18
*OS details:*
SUSE Linux Enterprise Server 11 (x86_64)
After I run a full import,Indexing proceeds sucessfully,but seems to freeze
everytime after fetching fixed number of records.What I mean is after it
fetches 10730 records it just freezes and doesnt process any more.
Excerpt from dataimport.xml:
<lst name="statusMessages">
<str name="Time Elapsed">0:15:31.959</str>
<str name="Total Requests made to DataSource">0</str>
<str name="Total Rows Fetched">*10730*</str>
<str name="Total Documents Processed">3579</str>
<str name="Total Documents Skipped">0</str>
<str name="Full Dump Started">2014-07-25 10:44:39</str>
This seems to happen everytime.
I checked the tomcatlog.Following is the excerpt when Solr freezes:
INFO: Generating record for Unique ID :null attachment Ref:null
parent ref :null executed by thread:25
Jul 25, 2014 10:53:31 AM
org.apache.solr.update.processor.LogUpdateProcessor processAdd
FINE: add AH_12345
Jul 25, 2014 10:53:31 AM
org.apache.solr.handler.dataimport.DocBuilder$EntityRunner runAThread
INFO: Generating record for Unique ID :null attachment Ref:null
parent ref :null executed by thread:26
Jul 25, 2014 10:53:31 AM
org.apache.solr.update.processor.LogUpdateProcessor processAdd
FINE: add AH_23451
Jul 25, 2014 10:53:34 AM org.apache.solr.core.SolrCore execute
INFO: [calls] webapp=/solr path=/dataimport params={} status=0 QTime=0
Jul 25, 2014 10:53:36 AM org.apache.solr.core.SolrCore execute
INFO: [calls] webapp=/solr path=/dataimport params={} status=0 QTime=0
Jul 25, 2014 10:53:38 AM org.apache.solr.core.SolrCore execute
INFO: [calls] webapp=/solr path=/dataimport params={} status=0 QTime=0
Help appreciated.
Regards,
Aniket
Re: Solr Full Import frozen after indexing a fixed number of records
Posted by Aniket Bhoi <an...@gmail.com>.
On Sun, Jul 27, 2014 at 12:28 PM, Gora Mohanty <go...@mimirtech.com> wrote:
> On 27 July 2014 12:13, Aniket Bhoi <an...@gmail.com> wrote:
>
> > On Fri, Jul 25, 2014 at 8:32 PM, Aniket Bhoi <an...@gmail.com>
> > wrote:
> >
> > > I have Apache Solr,hosted on my apache Tomcat Server with SQLServer
> > > Backend.
> >
> > [...]
>
> > > After I run a full import,Indexing proceeds sucessfully,but seems to
> > > freeze everytime after fetching fixed number of records.What I mean is
> > > after it fetches 10730 records it just freezes and doesnt process any
> > more.
> > >
> > > Excerpt from dataimport.xml:
> > >
> > > <lst name="statusMessages">
> > > <str name="Time Elapsed">0:15:31.959</str>
> > > <str name="Total Requests made to DataSource">0</str>
> > > <str name="Total Rows Fetched">*10730*</str>
> > > <str name="Total Documents Processed">3579</str>
> > > <str name="Total Documents Skipped">0</str>
> > > <str name="Full Dump Started">2014-07-25 10:44:39</str>
> > >
> > > This seems to happen everytime.
> > >
> > > I checked the tomcatlog.Following is the excerpt when Solr freezes:
> > >
> > > INFO: Generating record for Unique ID :null attachment Ref:null
> > > parent ref :null executed by thread:25
> >
>
> [...]
>
> Something is wrong with your DIH config file: You seem to be getting null
> for a document unique ID. Please share the file with us.
>
> Regards,
> Gora
>
Hi,
The thing is that I have 3 Solr instances deployed across Dev ,QA and
Production.They have the exact same configuration files but point to
different databases.The DIH config is the same across all 2 instances.It is
only in QA that this issue occurs though.Thoughts on this?
Regards,
Aniket
Re: Solr Full Import frozen after indexing a fixed number of records
Posted by Gora Mohanty <go...@mimirtech.com>.
On 27 July 2014 12:13, Aniket Bhoi <an...@gmail.com> wrote:
> On Fri, Jul 25, 2014 at 8:32 PM, Aniket Bhoi <an...@gmail.com>
> wrote:
>
> > I have Apache Solr,hosted on my apache Tomcat Server with SQLServer
> > Backend.
>
> [...]
> > After I run a full import,Indexing proceeds sucessfully,but seems to
> > freeze everytime after fetching fixed number of records.What I mean is
> > after it fetches 10730 records it just freezes and doesnt process any
> more.
> >
> > Excerpt from dataimport.xml:
> >
> > <lst name="statusMessages">
> > <str name="Time Elapsed">0:15:31.959</str>
> > <str name="Total Requests made to DataSource">0</str>
> > <str name="Total Rows Fetched">*10730*</str>
> > <str name="Total Documents Processed">3579</str>
> > <str name="Total Documents Skipped">0</str>
> > <str name="Full Dump Started">2014-07-25 10:44:39</str>
> >
> > This seems to happen everytime.
> >
> > I checked the tomcatlog.Following is the excerpt when Solr freezes:
> >
> > INFO: Generating record for Unique ID :null attachment Ref:null
> > parent ref :null executed by thread:25
>
[...]
Something is wrong with your DIH config file: You seem to be getting null
for a document unique ID. Please share the file with us.
Regards,
Gora
Re: Solr Full Import frozen after indexing a fixed number of records
Posted by Aniket Bhoi <an...@gmail.com>.
On Fri, Jul 25, 2014 at 8:32 PM, Aniket Bhoi <an...@gmail.com> wrote:
> I have Apache Solr,hosted on my apache Tomcat Server with SQLServer
> Backend.
>
>
> Details:
>
> *Solr Version:*
> Solr Specification Version: 3.4.0.2012.01.23.14.08.01
> Solr Implementation Version: 3.4
> Lucene Specification Version: 3.4
> Lucene Implementation Version: 3.4
>
> *Tomcat version:*
> Apache Tomcat/6.0.18
>
> *OS details:*
> SUSE Linux Enterprise Server 11 (x86_64)
>
> After I run a full import,Indexing proceeds sucessfully,but seems to
> freeze everytime after fetching fixed number of records.What I mean is
> after it fetches 10730 records it just freezes and doesnt process any more.
>
> Excerpt from dataimport.xml:
>
> <lst name="statusMessages">
> <str name="Time Elapsed">0:15:31.959</str>
> <str name="Total Requests made to DataSource">0</str>
> <str name="Total Rows Fetched">*10730*</str>
> <str name="Total Documents Processed">3579</str>
> <str name="Total Documents Skipped">0</str>
> <str name="Full Dump Started">2014-07-25 10:44:39</str>
>
> This seems to happen everytime.
>
> I checked the tomcatlog.Following is the excerpt when Solr freezes:
>
> INFO: Generating record for Unique ID :null attachment Ref:null
> parent ref :null executed by thread:25
> Jul 25, 2014 10:53:31 AM
> org.apache.solr.update.processor.LogUpdateProcessor processAdd
> FINE: add AH_12345
> Jul 25, 2014 10:53:31 AM
> org.apache.solr.handler.dataimport.DocBuilder$EntityRunner runAThread
> INFO: Generating record for Unique ID :null attachment Ref:null
> parent ref :null executed by thread:26
> Jul 25, 2014 10:53:31 AM
> org.apache.solr.update.processor.LogUpdateProcessor processAdd
> FINE: add AH_23451
> Jul 25, 2014 10:53:34 AM org.apache.solr.core.SolrCore execute
> INFO: [calls] webapp=/solr path=/dataimport params={} status=0 QTime=0
> Jul 25, 2014 10:53:36 AM org.apache.solr.core.SolrCore execute
> INFO: [calls] webapp=/solr path=/dataimport params={} status=0 QTime=0
> Jul 25, 2014 10:53:38 AM org.apache.solr.core.SolrCore execute
> INFO: [calls] webapp=/solr path=/dataimport params={} status=0 QTime=0
>
> Help appreciated.
>
> Regards,
>
> Aniket
>
>
>
Hi,
Anyone??..Help needed folks,
Regards,
Aniket