Documentation on the new updateLog transaction log feature?

["David Smiley (@MITRE.org)" <DS...@mitre.org>]

Is there any documentation on the updateLog transaction log feature in Solr
4?

I started a quick prototype using Solr 4 alpha with a fairly structured
schema; no big text.  I disabled auto-commit which came pre-enabled and
there's no soft-commit either.  With CURL I posted a 1.8GB CSV file.  AFter
some time, I find this huge ~2.6GB transaction log file that didn't want to
go away.  FWIW A small number of records had errors, and maybe half of the
records were duplicates of existing records in the file because of
duplicated IDs.  When I restarted Solr, Solr spent a long time reading from
the transaction log before it was ready.  But the file is still there; I
manually deleted it.  This isn't a great user experience for a feature I
have no intention of using (no Solr Cloud for this project, and no so-called
"realtime get" which has always struck me as an odd feature).

~ David



-----
 Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
--
View this message in context: http://lucene.472066.n3.nabble.com/Documentation-on-the-new-updateLog-transaction-log-feature-tp4000537.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Documentation on the new updateLog transaction log feature?

[Yonik Seeley <yo...@lucidworks.com>]

On Fri, Aug 10, 2012 at 2:31 PM, David Smiley (@MITRE.org)
<DS...@mitre.org> wrote:
> Is there any documentation on the updateLog transaction log feature in Solr
> 4?

Not much beyond what's in solrconfig.xml

> I started a quick prototype using Solr 4 alpha with a fairly structured
> schema; no big text.  I disabled auto-commit which came pre-enabled and
> there's no soft-commit either.  With CURL I posted a 1.8GB CSV file.  AFter
> some time, I find this huge ~2.6GB transaction log file that didn't want to
> go away.  FWIW A small number of records had errors, and maybe half of the
> records were duplicates of existing records in the file because of
> duplicated IDs.  When I restarted Solr, Solr spent a long time reading from
> the transaction log before it was ready.  But the file is still there; I
> manually deleted it.  This isn't a great user experience for a feature I
> have no intention of using


Simply comment out the following in solrconfig.xml

    <updateLog>
      <str name="dir">${solr.data.dir:}</str>
    </updateLog>

> (no Solr Cloud for this project, and no so-called
> "realtime get" which has always struck me as an odd feature).

It's often pretty important for anyone using Solr as a NoSQL store.

-Yonik
http://lucidworks.com