Corrupt fragments in large pipeline output

[Corin Moss <Co...@tvnz.co.nz>]

Hi Guys,

I've got some fairly large documents being serialized into various
transformers (XSLT, Xinclude and Cinclude.)  Every now and then I notice
that the stored output is corrupt.  Certain elements are in the wrong
order etc (I'm using RDF, so for an example I've noticed Description
elements out of synch with li elements.)   This is happening before I
ever get to apply any XSLT.  I figure that this could come from the XSPs
I'm using (some of which have been converted to generators) or from the
X or C include transformations or perhaps a corruption in the store
itself (all output is cached very heavily.)

I've been playing with the output buffer settings on the various
pipeline types I'm using, and I *think* that this is helping.  I suspect
that this is going to require a certain amount of contextual
configuration - ie:  one request will be heavier than another, and thus
may require a different buffer size.

My documents are often around the 1 meg mark, so, my question is: does
anyone have tips for an appropriate buffer size?  I notice *very* odd
results if I set the buffer too low: errors show up in included sections
of the output etc.  If I set the buffer too high, I obviously lose the
benefits.

At the moment my buffer size is around 150kb. 

What would you suggest as a reasonable number for a 1meg XML document?

Thanks,

Corin

Corin Moss
Lead Developer
TVNZ Interactive

+64 9 916 7367
+64 21 403 054
corin.moss@tvnz.co.nz



================================================================
CAUTION: This e-mail and any attachment(s) contains information
that is intended to be read only by the named recipient(s). It
may contain information that is confidential, proprietary or the
subject of legal privilege. This information is not to be used by
any other person and/or organisation. If you are not the intended
recipient, please advise us immediately and delete this e-mail
from your system. Do not use any information contained in it.

================================================================
For more information on the Television New Zealand Group, visit
us online at http://www.tvnz.co.nz
================================================================