Lots of tlog files remained, why?

[Floyd Wu <fl...@gmail.com>]

After re-index 2xxxx xml files and done commit, optimization many times, I
still have many tlog files in data/tlof directory.

Why?

How to remove those files(delete them directly or just ignored them?)

What is the difference if tlog files exist or not?

Please kindly guide me.

Thanks

Floyd

Re: Lots of tlog files remained, why?

[Floyd Wu <fl...@gmail.com>]

Hi Eric,

Sorry for replay being late.
The tlog file stay there for one week and seems no decease. Most of them
are 3~5 MB and totally 40MB.

The article your point I've read many times but no working. Everytime I
reindex files solr generate many tlog of them and no matter how many hard
commit I di , tlog still there.

I'm using Solr 4.3.2 on Windown server 2003 32bit enviroment.
What else detail information should I provide, please le me know.

PS: Should I ugrade Solr to 4.5.1?

Floyd



2013/11/4 Erick Erickson <er...@gmail.com>

> What is your commit strategy? A hard commit
> (openSearcher=true or false doesn't matter)
> should close the current tlog file, open
> a new one and delete old ones. That said, there
> will be enough tlog files kept around to hold at
> least 100 documents. So if you're committing
> too often (say after every document or something),
> you can expect to have a bunch around. The
> real question is whether they stay around forever
> or not. If you index more documents, do old ones
> disappear?
>
> Here's a writerup:
>
> http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>
> If that doesn't help, what version of Solr? How
> big are you tlog files? Details matter.
>
> Best,
> Erick
>
>
> On Sun, Nov 3, 2013 at 10:03 AM, Floyd Wu <fl...@gmail.com> wrote:
>
> > After re-index 2xxxx xml files and done commit, optimization many times,
> I
> > still have many tlog files in data/tlof directory.
> >
> > Why?
> >
> > How to remove those files(delete them directly or just ignored them?)
> >
> > What is the difference if tlog files exist or not?
> >
> > Please kindly guide me.
> >
> > Thanks
> >
> > Floyd
> >
>

Re: Lots of tlog files remained, why?

[Erick Erickson <er...@gmail.com>]

What is your commit strategy? A hard commit
(openSearcher=true or false doesn't matter)
should close the current tlog file, open
a new one and delete old ones. That said, there
will be enough tlog files kept around to hold at
least 100 documents. So if you're committing
too often (say after every document or something),
you can expect to have a bunch around. The
real question is whether they stay around forever
or not. If you index more documents, do old ones
disappear?

Here's a writerup:
http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

If that doesn't help, what version of Solr? How
big are you tlog files? Details matter.

Best,
Erick


On Sun, Nov 3, 2013 at 10:03 AM, Floyd Wu <fl...@gmail.com> wrote:

> After re-index 2xxxx xml files and done commit, optimization many times, I
> still have many tlog files in data/tlof directory.
>
> Why?
>
> How to remove those files(delete them directly or just ignored them?)
>
> What is the difference if tlog files exist or not?
>
> Please kindly guide me.
>
> Thanks
>
> Floyd
>