TDB sync performance on Windows 7 on laptop/desktop hardware

[Brian McBride <br...@epimorphics.com>]

In a recent thread initiated by Frank called "Bad Jena TDB performance" 
I reported that I was getting around a factor of 2 differences in 
performance between two Windows 7 machines with surprisingly a my laptop 
being around twice as fast as my desktop when TDB was doing a lot of 
syncing to disk.

I believe I can now account for this.

My theory is that on the laptop SYNC's are returning before changed 
buffers have reached the spinning magnetic surface of the disk.  I have 
no reason to believe this is TBD's fault.

I've been using Frank's test application from the previous thread, 
modified to load only 200 graphs.  It is using TDB in DIRECT mode and 
loads the graphs directly into a TDB model.

My profiler shows that this results in 2289 calls to 
...tdb.base.block.BlockManagerFile().force().  This method just calls 
force() on the file channel to force all writes to disk.

On my desktop machine these 2289 calls take around 77 seconds.  On my 
laptop machine they take around 6 seconds.  This difference accounts for 
the difference in performance of the application on these two machines.

If I turn of disk write caching in both machines, the same number of 
calls takes 117 seconds on my desktop and 130 seconds on my laptop.  So 
my laptop goes from 6 seconds to 130 seconds and the only thing I have 
changed is to turn off disk writing caching.  This suggests that if disk 
write caching is enabled then java/windows/diskdriver is not forcing the 
write through to disk.

I know nothing about hardware architectures these days, so what follows 
is speculation.  It may be arguable that on a laptop one can view the 
disk subsystem as including the disk, the disk controller, the battery 
and the machine's ram.  With that view, then once a write command is 
queued in the disk controller it is effectively in the disk subsystem, 
just as if it were a battery backed buffered disk subsystem on a server, 
and it is therefore legitimate to claim that the write is safely on 
'disk' as soon as the write command is accepted by the controller.  
SYNC's could then return as soon as the controller accepted the write 
request even if the data is has not reached the spinning media.

As my desktop has no battery, it can't make that claim and is therefore 
has to wait until the bits have reached the spinning media.

Brian

Re: TDB sync performance on Windows 7 on laptop/desktop hardware

[Brian McBride <br...@epimorphics.com>]

Hi Damian,

On 30/03/2011 13:28, Damian Steer wrote:

[...]
> This reminds me of a discussion about sync (roughly flush to kernel) and
> fsync (flush kernel buffers to disk) on linux, triggered by firefox [1]
> many moons ago. In that discussion I recall one of the reasons for
> avoiding fsync was to preserve laptop battery. [2]
Thanks for the pointers  - interesting discussion.  You are right that 
one motivation was to preserve laptop battery by not spinning up an idle 
disk.  Personally, I'm not sure that's a good reason to ignore Posix.

In my tests the disk was already spinning and the laptop on mains power.

Brian

> Damian
>
> [1]<http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/>
> [2]<http://thunk.org/tytso/blog/2009/03/15/dont-fear-the-fsync/>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk2TIdUACgkQAyLCB+mTtykhPwCfbiz3bJw+tie81gZcuXQQF19w
> F3MAoIjtFmstC2JimfBAHLozaPrH4w1z
> =yple
> -----END PGP SIGNATURE-----

Re: TDB sync performance on Windows 7 on laptop/desktop hardware

[Damian Steer <d....@bristol.ac.uk>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 30/03/11 12:28, Brian McBride wrote:
> In a recent thread initiated by Frank called "Bad Jena TDB performance"
> I reported that I was getting around a factor of 2 differences in
> performance between two Windows 7 machines with surprisingly a my laptop
> being around twice as fast as my desktop when TDB was doing a lot of
> syncing to disk.
> 
> I believe I can now account for this.
> 
> My theory is that on the laptop SYNC's are returning before changed
> buffers have reached the spinning magnetic surface of the disk.  I have
> no reason to believe this is TBD's fault.

Hi Brian,

This reminds me of a discussion about sync (roughly flush to kernel) and
fsync (flush kernel buffers to disk) on linux, triggered by firefox [1]
many moons ago. In that discussion I recall one of the reasons for
avoiding fsync was to preserve laptop battery. [2]

Damian

[1] <http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/>
[2] <http://thunk.org/tytso/blog/2009/03/15/dont-fear-the-fsync/>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2TIdUACgkQAyLCB+mTtykhPwCfbiz3bJw+tie81gZcuXQQF19w
F3MAoIjtFmstC2JimfBAHLozaPrH4w1z
=yple
-----END PGP SIGNATURE-----