Concurrency problem? Something else?

[Joey Carr <Jo...@CanopySystems.com>]

I'm using FOP 0.17 with a hacked over version of Cocoon 1.8.2 that solves
the Cocoon embedding problem involving the Writer/OutputStream change in the
Driver class (why Cocoon had been stuck on an old version of FOP forever).
The hack involves modifying the cocoon method
org.apache.cocoon.formatter.FO2PDFFormatter.format() such that the
OutputSteam passed to the Driver class is written to a StringWriter en masse
at the end of the method.  Not pretty, but we're doing small things so the
penalty is small.

So this morning, on my birthday no less, we discovered that our production
system was slowly leaking towards the dreaded OME!  And lil ole' me being
the one responsible for this stuff!  It appears that FOP was generating a
giant document and since we were storing the output PDF temporarily as a
string, is was sucking up our memory resources.

The cocoon format method takes a DOM Document, the output Writer to write
the formatted output to, and a Dictionary of parameters, which it callously
disregards.  The Document is transformed into a String of XML and re-parsed
by a SAXParser, so presumably there is no pipeline here, so it couldn't be
that the XML data source just keeps making more FO for FOP to render.  But
continue to render is does!

The first snip (at the end of the message) is out of our log of stdout.  It
appears that FOP is in the middle of rendering a 10 page document when
another user request comes in for another report.  You can see that the
messages from the Driver constructor interrupt the ongoing PDF generation.
It appeared to me at first that the concurrent request had introduced some
inconsistent state in FOP, exploiting some critical threading oversight...
but it's simply not realistic that we had already produced a 10 page report!
Our reports are never any longer that 4 pages.

The second snip shows my hack over of the cocoon method,
org.apache.cocoon.formatter.FO2PDFFormatter.format(), which shows how I'm
using the Driver class, and, I think, shows that the XML data source
couldn't be the source of the leak since it must be rendered as a String
before FOP even looks at it.  Is it possible that some freaky arrangement of
FOs can cause unbounded recursion in the FOP area formatting algorithm?

So, if any of you have any ideas about this, please, PLEASE!! send them
along!

Thanks in advance.

-Joey


<snip>
using renderer org.apache.fop.render.pdf.PDFRenderer
using element mapping org.apache.fop.fo.StandardElementMapping
using element mapping org.apache.fop.svg.SVGElementMapping
using property list mapping org.apache.fop.fo.StandardPropertyListMapping
using property list mapping org.apache.fop.svg.SVGPropertyListMapping
building formatting object tree
setting up fonts
formatting FOs into areas
 [1] [2] [3] [4] [5] [6] [7] [8] [9] [10using renderer
org.apache.fop.render.pdf.PDFRenderer
using element mapping org.apache.fop.fo.StandardElementMapping
using element mapping org.apache.fop.svg.SVGElementMapping
using property list mapping org.apache.fop.fo.StandardPropertyListMapping
using property list mapping org.apache.fop.svg.SVGPropertyListMapping
building formatting object tree
setting up fonts
formatting FOs into areas
 [1] [11]rendering areas to PDF
writing out PDF
] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26]
[27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41]
[42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56]
[57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71]
[72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86]
[87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101]
[102] [103] [104] [105] [106] [107] [108] [109] [110] [111] [112] [113]
[114] [115] [116] [117] [118] [119] [120] [121] [122] [123] [124] [125]
[126] [127] [128] [129] [130] [131] [132] [133] [134] [135] [136] [137]
[138] [139] [140] [141] [142] [143] [144] [145] [146] [147] [148] [149]
[150] [151] [152] [153] [154]
</snip>

<snip>
   public void format(Document document, Writer writer, Dictionary
parameters)
      throws Exception
   {
      Driver driver = new Driver();
      driver.setRenderer("org.apache.fop.render.pdf.PDFRenderer",
FOP_VERSION);
      driver.addElementMapping("org.apache.fop.fo.StandardElementMapping");
      driver.addElementMapping("org.apache.fop.svg.SVGElementMapping");

      ByteArrayOutputStream byteOutStream = new ByteArrayOutputStream();
      driver.setOutputStream(byteOutStream);

 
driver.addPropertyList("org.apache.fop.fo.StandardPropertyListMapping");
      driver.addPropertyList("org.apache.fop.svg.SVGPropertyListMapping");

      // To workaround the problem that Xalan 1.x does not output
      // DOM2-compliant namespaces, we use a major hack - output xml as a
      // string and read it in again.  With a DOM2-compliant parser such as
      // Xerces, this should work fine.
      deferredInit ();
      StringWriter tempWriter = new StringWriter ();
      xmlFormatter.format (document, tempWriter, NO_PARAMETERS);
      String tempXml = tempWriter.toString ();

      // For now, we just use Xerces - it would be more complicated to
support
      // other parsers here.
      SAXParser parser = new SAXParser ();
      parser.setFeature("http://xml.org/sax/features/namespaces", true);
      driver.buildFOTree(parser, new InputSource (new StringReader
(tempXml)));

      driver.format();
      driver.render();
      writer.write(byteOutStream.toString());
   }
}
</snip>