You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@jena.apache.org by Bill Roberts <bi...@swirrl.com> on 2011/08/18 11:54:41 UTC

tdbloader and stats.opt

Is there an option for tdbloader to ask it not to create the stats.opt file?

I am still finding that I get the best overall performance from TDB with no optimisation (perhaps because I am not smart enough to create a well-tuned optimisation file, despite Andy's useful advice!).  But if I use tdbloader to add new data to a store, then tdbloader (0.8.10) creates a new stats.opt file, which I then have to remove again.

Thanks

Bill

Re: tdbloader and stats.opt

Posted by Bill Roberts <bi...@swirrl.com>.

thanks Andy - yes that's pretty much what I do at the moment and I have a small script to do that.  Just sometimes I use the raw command instead of my script, forget to remove stats.opt then wonder for a few minutes why the performance has just got worse, until I remember!

It is indeed creating stats.opt on an incremental load.  I'll make a JIRA issue.

I'll put together some example data and queries and get back to you.  

Thanks

Bill


 
On 18 Aug 2011, at 11:08, Andy Seaborne wrote:

> Bill,
> 
> There isn't an option but you can directly set it with
> 
> tdbloader -loc DB ...
> rm -f DB/stats.opt
> touch DB/none.opt
> 
> fixed.opt is also another optimizer option.  Just change to "touch DB/fixed.opt".
> 
> If it's creating one even on an incremental add of data, then that's a bug - could you file a JIRA about it please?  Such a stats.opt file is probably wrong, either useless of maybe even a cause of poor optimization.
> 
> The fact that optimizer is unoptimizing is worth investigating.  Can you send me the data and a query that illustrates the effect?
> 
> 	Andy
> 
> On 18/08/11 10:54, Bill Roberts wrote:
>> Is there an option for tdbloader to ask it not to create the stats.opt file?
>> 
>> I am still finding that I get the best overall performance from TDB with no optimisation (perhaps because I am not smart enough to create a well-tuned optimisation file, despite Andy's useful advice!).  But if I use tdbloader to add new data to a store, then tdbloader (0.8.10) creates a new stats.opt file, which I then have to remove again.
>> 
>> Thanks
>> 
>> Bill

Re: tdbloader and stats.opt

Posted by Andy Seaborne <an...@epimorphics.com>.

Bill,

There isn't an option but you can directly set it with

tdbloader -loc DB ...
rm -f DB/stats.opt
touch DB/none.opt

fixed.opt is also another optimizer option.  Just change to "touch 
DB/fixed.opt".

If it's creating one even on an incremental add of data, then that's a 
bug - could you file a JIRA about it please?  Such a stats.opt file is 
probably wrong, either useless of maybe even a cause of poor optimization.

The fact that optimizer is unoptimizing is worth investigating.  Can you 
send me the data and a query that illustrates the effect?

	Andy

On 18/08/11 10:54, Bill Roberts wrote:
> Is there an option for tdbloader to ask it not to create the stats.opt file?
>
> I am still finding that I get the best overall performance from TDB with no optimisation (perhaps because I am not smart enough to create a well-tuned optimisation file, despite Andy's useful advice!).  But if I use tdbloader to add new data to a store, then tdbloader (0.8.10) creates a new stats.opt file, which I then have to remove again.
>
> Thanks
>
> Bill