Infinately repeating table headers (RE: Concurrency problem? Som ething else?)

[Joey Carr <Jo...@CanopySystems.com>]

Hello all!

About a week ago we experienced a mysterious problem with FOP generating an
absurd number of pages and exhausting our memory supply, the whole 1.5 gig!
We weren't initally able to reproduce the effect, but, after hacking over
the PageSequece class in version 0.17.0 such that it threw an Error in case
too many pages were getting generated, we were able to find a test case and
trap the input FO which caused the problem.

After a bit of analysis it became clear that some combination of table
headers and keep conditions were the culprit.  I tried making a table with a
header too big to fit on a page and FOP remained reasonable: it ouput two
blank pages and then ouput the remaining header text on the last page.
Adding text to the body of this table simply multiplied the number of pages
produced, that is, two blank pages for every page of body text untill the
body text was exhausted.  Next I tried placing the
keep-together.within-page="always" property in the block contained within
the table header cell; this had no effect.  Finally what did it was when I
split the header to contain TWO blocks, the first of which had the attribute
keep-with-next.within-page="always".  This finally reproduced the unbounded
condition.

I've tested this in version 0.20.1, and it's a problem there too.

This is sort-of a documented bug.  Bug #1773 describes a similar problem
that seems to have since been resolved.  Does this merit a new bug report?

Is anyone planning on working this, or working on it already?  I might
tackle this myself, if no one else is planning on doing it--my employers
tell me they're willing to let me do this at work :)

Thanks all!

-Joey


-----Original Message-----
From: Joey Carr [mailto:Joey.Carr@CanopySystems.com]
Sent: Friday, August 31, 2001 12:15 PM
To: 'fop-dev@xml.apache.org'
Subject: Concurrency problem? Something else?


I'm using FOP 0.17 with a hacked over version of Cocoon 1.8.2 that solves
the Cocoon embedding problem involving the Writer/OutputStream change in the
Driver class (why Cocoon had been stuck on an old version of FOP forever).
The hack involves modifying the cocoon method
org.apache.cocoon.formatter.FO2PDFFormatter.format() such that the
OutputSteam passed to the Driver class is written to a StringWriter en masse
at the end of the method.  Not pretty, but we're doing small things so the
penalty is small.
So this morning, on my birthday no less, we discovered that our production
system was slowly leaking towards the dreaded OME!  And lil ole' me being
the one responsible for this stuff!  It appears that FOP was generating a
giant document and since we were storing the output PDF temporarily as a
string, is was sucking up our memory resources.
The cocoon format method takes a DOM Document, the output Writer to write
the formatted output to, and a Dictionary of parameters, which it callously
disregards.  The Document is transformed into a String of XML and re-parsed
by a SAXParser, so presumably there is no pipeline here, so it couldn't be
that the XML data source just keeps making more FO for FOP to render.  But
continue to render is does!
The first snip (at the end of the message) is out of our log of stdout.  It
appears that FOP is in the middle of rendering a 10 page document when
another user request comes in for another report.  You can see that the
messages from the Driver constructor interrupt the ongoing PDF generation.
It appeared to me at first that the concurrent request had introduced some
inconsistent state in FOP, exploiting some critical threading oversight...
but it's simply not realistic that we had already produced a 10 page report!
Our reports are never any longer that 4 pages.
The second snip shows my hack over of the cocoon method,
org.apache.cocoon.formatter.FO2PDFFormatter.format(), which shows how I'm
using the Driver class, and, I think, shows that the XML data source
couldn't be the source of the leak since it must be rendered as a String
before FOP even looks at it.  Is it possible that some freaky arrangement of
FOs can cause unbounded recursion in the FOP area formatting algorithm?
So, if any of you have any ideas about this, please, PLEASE!! send them
along! 
Thanks in advance. 
-Joey 


<snip> 
using renderer org.apache.fop.render.pdf.PDFRenderer 
using element mapping org.apache.fop.fo.StandardElementMapping 
using element mapping org.apache.fop.svg.SVGElementMapping 
using property list mapping org.apache.fop.fo.StandardPropertyListMapping 
using property list mapping org.apache.fop.svg.SVGPropertyListMapping 
building formatting object tree 
setting up fonts 
formatting FOs into areas 
 [1] [2] [3] [4] [5] [6] [7] [8] [9] [10using renderer
org.apache.fop.render.pdf.PDFRenderer 
using element mapping org.apache.fop.fo.StandardElementMapping 
using element mapping org.apache.fop.svg.SVGElementMapping 
using property list mapping org.apache.fop.fo.StandardPropertyListMapping 
using property list mapping org.apache.fop.svg.SVGPropertyListMapping 
building formatting object tree 
setting up fonts 
formatting FOs into areas 
 [1] [11]rendering areas to PDF 
writing out PDF 
] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26]
[27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41]
[42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56]
[57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71]
[72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86]
[87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101]
[102] [103] [104] [105] [106] [107] [108] [109] [110] [111] [112] [113]
[114] [115] [116] [117] [118] [119] [120] [121] [122] [123] [124] [125]
[126] [127] [128] [129] [130] [131] [132] [133] [134] [135] [136] [137]
[138] [139] [140] [141] [142] [143] [144] [145] [146] [147] [148] [149]
[150] [151] [152] [153] [154]
</snip> 
<snip> 
   public void format(Document document, Writer writer, Dictionary
parameters) 
      throws Exception 
   { 
      Driver driver = new Driver(); 
      driver.setRenderer("org.apache.fop.render.pdf.PDFRenderer",
FOP_VERSION); 
      driver.addElementMapping("org.apache.fop.fo.StandardElementMapping"); 
      driver.addElementMapping("org.apache.fop.svg.SVGElementMapping"); 
      ByteArrayOutputStream byteOutStream = new ByteArrayOutputStream(); 
      driver.setOutputStream(byteOutStream); 
 
driver.addPropertyList("org.apache.fop.fo.StandardPropertyListMapping"); 
      driver.addPropertyList("org.apache.fop.svg.SVGPropertyListMapping"); 
      // To workaround the problem that Xalan 1.x does not output 
      // DOM2-compliant namespaces, we use a major hack - output xml as a 
      // string and read it in again.  With a DOM2-compliant parser such as 
      // Xerces, this should work fine. 
      deferredInit (); 
      StringWriter tempWriter = new StringWriter (); 
      xmlFormatter.format (document, tempWriter, NO_PARAMETERS); 
      String tempXml = tempWriter.toString (); 
      // For now, we just use Xerces - it would be more complicated to
support 
      // other parsers here. 
      SAXParser parser = new SAXParser (); 
      parser.setFeature("http://xml.org/sax/features/namespaces", true); 
      driver.buildFOTree(parser, new InputSource (new StringReader
(tempXml))); 
      driver.format(); 
      driver.render(); 
      writer.write(byteOutStream.toString()); 
   } 
} 
</snip> 

Re: Infinately repeating table headers (RE: Concurrency problem? Som ething else?)

[Keiron Liddle <ke...@aftexsw.com>]

On Mon, 10 Sep 2001 23:25:55 Joey Carr wrote:
> Is anyone planning on working this, or working on it already?  I might
> tackle this myself, if no one else is planning on doing it--my employers
> tell me they're willing to let me do this at work :)

This is really a general problem of how the code handles the layout.
The re-design should handle this much better, so in a way there are people
working on this. You are welcome to help out.

For example you can help with how it should handle table headers, table
footers that are too large for a page, or if you want there are plenty more
general problems to solve in the redesign.
The more people the better and hopefully quicker.

---------------------------------------------------------------------
To unsubscribe, e-mail: fop-dev-unsubscribe@xml.apache.org
For additional commands, email: fop-dev-help@xml.apache.org