You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by srinalluri <na...@yahoo.com> on 2013/06/28 00:41:48 UTC
full-import failed after 5 hours with Exception: ORA-01555:
snapshot too old: rollback segment number with name "" too small
ORA-22924: snapshot too old
Hello,
I am using Solr 4.3.2 and Oracle DB. The sub entity is using
CachedSqlEntityProcessor. The dataSource is having batchSize="500". The
full-import is failed with 'ORA-01555: snapshot too old: rollback segment
number with name "" too small ORA-22924: snapshot too old' Exception after
5 hours.
We already increased the undo space 4 times at the database end. Number of
records in the jan_story table is 800,000 only. Tomcat is with 4GB JVM
memory.
Following is the entity (there are other sub-entities, I didn't mention them
here. As the import failed with article_details entity. article_details is
the first sub-entity)
<entity name="par8-article-testingprod" dataSource="par8_prod" pk="VCMID"
preImportDeleteQuery="content_type:article AND
repository:par8qatestingprod"
query="select ID as VCMID from jan_story">
<entity name="article_details" dataSource="par8_prod"
transformer="TemplateTransformer,ClobTransformer,RegexTransformer"
query="select bb.recordid, aa.ID as DID,aa.STORY_TITLE,
aa.STORY_HEADLINE, aa.SOURCE, aa.DECK, regexp_replace(aa.body,
'\<p\>\[(pullquote|summary)\]\</p\>|\[video [0-9]+?\]|\[youtube
.+?\]', '') as BODY, aa.PUBLISHED_DATE, aa.MODIFIED_DATE, aa.DATELINE,
aa.REPORTER_NAME, aa.TICKER_CODES,aa.ADVERTORIAL_CONTENT from jan_story
aa,mapp bb where aa.id=bb.keystring1" cacheKey="DID"
cacheLookup="par8-article-testingprod.VCMID"
processor="CachedSqlEntityProcessor" >
<field column="content_type" template="article" />
<field column="RECORDID" name="native_id" />
<field column="repository" template="par8qatestingprod" />
<field column="STORY_TITLE" name="title" />
<field column="DECK" name="description" clob="true" />
<field column="PUBLISHED_DATE" name="date" />
<field column="MODIFIED_DATE" name="last_modified_date" />
<field column="BODY" name="body" clob="true" />
<field column="SOURCE" name="source" />
<field column="DATELINE" name="dateline" />
<field column="STORY_HEADLINE" name="export_headline" />
</entity>
</entity>
The full-import without CachedSqlEntityProcessor is taking 7 days. That is
why I am doing all this.
--
View this message in context: http://lucene.472066.n3.nabble.com/full-import-failed-after-5-hours-with-Exception-ORA-01555-snapshot-too-old-rollback-segment-number-wd-tp4073822.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: full-import failed after 5 hours with Exception: ORA-01555:
snapshot too old: rollback segment number with name "" too small ORA-22924:
snapshot too old
Posted by Michael Della Bitta <mi...@appinions.com>.
I would say definitely investigate the performance of the query, but also
since you're using CachedSqlEntityProcessor, you might want to back off on
the transaction isolation to READ_COMMITTED, which I think is the lowest
one that Oracle supports:
http://wiki.apache.org/solr/DataImportHandler#Configuring_JdbcDataSource
Michael Della Bitta
Applications Developer
o: +1 646 532 3062 | c: +1 917 477 7906
appinions inc.
“The Science of Influence Marketing”
18 East 41st Street
New York, NY 10017
t: @appinions <https://twitter.com/Appinions> | g+:
plus.google.com/appinions
w: appinions.com <http://www.appinions.com/>
On Fri, Jun 28, 2013 at 2:52 PM, Otis Gospodnetic <
otis.gospodnetic@gmail.com> wrote:
> Hi,
>
> I'd go talk to the DBA. How long does this query take if you run it
> directly against Oracle? How long if you run it locally vs. from a
> remove server (like Solr is in relation to your Oracle server(s)).
> What happens if you increase batchSize?
>
> Otis
> --
> Solr & ElasticSearch Support -- http://sematext.com/
> Performance Monitoring -- http://sematext.com/spm
>
>
>
> On Thu, Jun 27, 2013 at 6:41 PM, srinalluri <na...@yahoo.com>
> wrote:
> > Hello,
> >
> > I am using Solr 4.3.2 and Oracle DB. The sub entity is using
> > CachedSqlEntityProcessor. The dataSource is having batchSize="500". The
> > full-import is failed with 'ORA-01555: snapshot too old: rollback segment
> > number with name "" too small ORA-22924: snapshot too old' Exception
> after
> > 5 hours.
> >
> > We already increased the undo space 4 times at the database end. Number
> of
> > records in the jan_story table is 800,000 only. Tomcat is with 4GB JVM
> > memory.
> >
> > Following is the entity (there are other sub-entities, I didn't mention
> them
> > here. As the import failed with article_details entity. article_details
> is
> > the first sub-entity)
> >
> > <entity name="par8-article-testingprod" dataSource="par8_prod" pk="VCMID"
> > preImportDeleteQuery="content_type:article AND
> > repository:par8qatestingprod"
> > query="select ID as VCMID from jan_story">
> > <entity name="article_details" dataSource="par8_prod"
> > transformer="TemplateTransformer,ClobTransformer,RegexTransformer"
> > query="select bb.recordid, aa.ID as DID,aa.STORY_TITLE,
> > aa.STORY_HEADLINE, aa.SOURCE, aa.DECK, regexp_replace(aa.body,
> > '\<p\>\[(pullquote|summary)\]\</p\>|\[video [0-9]+?\]|\[youtube
> > .+?\]', '') as BODY, aa.PUBLISHED_DATE, aa.MODIFIED_DATE, aa.DATELINE,
> > aa.REPORTER_NAME, aa.TICKER_CODES,aa.ADVERTORIAL_CONTENT from jan_story
> > aa,mapp bb where aa.id=bb.keystring1" cacheKey="DID"
> > cacheLookup="par8-article-testingprod.VCMID"
> > processor="CachedSqlEntityProcessor" >
> > <field column="content_type" template="article" />
> > <field column="RECORDID" name="native_id" />
> > <field column="repository" template="par8qatestingprod" />
> > <field column="STORY_TITLE" name="title" />
> > <field column="DECK" name="description" clob="true" />
> > <field column="PUBLISHED_DATE" name="date" />
> > <field column="MODIFIED_DATE" name="last_modified_date" />
> > <field column="BODY" name="body" clob="true" />
> > <field column="SOURCE" name="source" />
> > <field column="DATELINE" name="dateline" />
> > <field column="STORY_HEADLINE" name="export_headline" />
> > </entity>
> > </entity>
> >
> >
> > The full-import without CachedSqlEntityProcessor is taking 7 days. That
> is
> > why I am doing all this.
> >
> >
> >
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/full-import-failed-after-5-hours-with-Exception-ORA-01555-snapshot-too-old-rollback-segment-number-wd-tp4073822.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: full-import failed after 5 hours with Exception: ORA-01555:
snapshot too old: rollback segment number with name "" too small ORA-22924:
snapshot too old
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
I'd go talk to the DBA. How long does this query take if you run it
directly against Oracle? How long if you run it locally vs. from a
remove server (like Solr is in relation to your Oracle server(s)).
What happens if you increase batchSize?
Otis
--
Solr & ElasticSearch Support -- http://sematext.com/
Performance Monitoring -- http://sematext.com/spm
On Thu, Jun 27, 2013 at 6:41 PM, srinalluri <na...@yahoo.com> wrote:
> Hello,
>
> I am using Solr 4.3.2 and Oracle DB. The sub entity is using
> CachedSqlEntityProcessor. The dataSource is having batchSize="500". The
> full-import is failed with 'ORA-01555: snapshot too old: rollback segment
> number with name "" too small ORA-22924: snapshot too old' Exception after
> 5 hours.
>
> We already increased the undo space 4 times at the database end. Number of
> records in the jan_story table is 800,000 only. Tomcat is with 4GB JVM
> memory.
>
> Following is the entity (there are other sub-entities, I didn't mention them
> here. As the import failed with article_details entity. article_details is
> the first sub-entity)
>
> <entity name="par8-article-testingprod" dataSource="par8_prod" pk="VCMID"
> preImportDeleteQuery="content_type:article AND
> repository:par8qatestingprod"
> query="select ID as VCMID from jan_story">
> <entity name="article_details" dataSource="par8_prod"
> transformer="TemplateTransformer,ClobTransformer,RegexTransformer"
> query="select bb.recordid, aa.ID as DID,aa.STORY_TITLE,
> aa.STORY_HEADLINE, aa.SOURCE, aa.DECK, regexp_replace(aa.body,
> '\<p\>\[(pullquote|summary)\]\</p\>|\[video [0-9]+?\]|\[youtube
> .+?\]', '') as BODY, aa.PUBLISHED_DATE, aa.MODIFIED_DATE, aa.DATELINE,
> aa.REPORTER_NAME, aa.TICKER_CODES,aa.ADVERTORIAL_CONTENT from jan_story
> aa,mapp bb where aa.id=bb.keystring1" cacheKey="DID"
> cacheLookup="par8-article-testingprod.VCMID"
> processor="CachedSqlEntityProcessor" >
> <field column="content_type" template="article" />
> <field column="RECORDID" name="native_id" />
> <field column="repository" template="par8qatestingprod" />
> <field column="STORY_TITLE" name="title" />
> <field column="DECK" name="description" clob="true" />
> <field column="PUBLISHED_DATE" name="date" />
> <field column="MODIFIED_DATE" name="last_modified_date" />
> <field column="BODY" name="body" clob="true" />
> <field column="SOURCE" name="source" />
> <field column="DATELINE" name="dateline" />
> <field column="STORY_HEADLINE" name="export_headline" />
> </entity>
> </entity>
>
>
> The full-import without CachedSqlEntityProcessor is taking 7 days. That is
> why I am doing all this.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/full-import-failed-after-5-hours-with-Exception-ORA-01555-snapshot-too-old-rollback-segment-number-wd-tp4073822.html
> Sent from the Solr - User mailing list archive at Nabble.com.