You are viewing a plain text version of this content. The canonical link for it is here.
Posted to fop-users@xmlgraphics.apache.org by Jean-François El Fouly <je...@elfouly.fr> on 2008/05/05 17:22:43 UTC
Major issue with image loading in FOP 0.95beta
Sorry, I guess in the strictest sense this is xmlgraphics-common related
now, but it was discovered and investigated using FOP 0.95beta.
We upgraded from 0.94 to use the new bookmarks features and our software
became globally unstable, hanging randomly during PDF generation -- to
be precise, getting all kinds of random problems during image loading.
The images we use are stored in SVN and retrieved via their HTTP URLs
from an SVN server. Reading the SVN logs, we see a very different
behaviour between 0.94 (which worked fine and has been retested since)
and 0.95beta: the FOP requests litteraly "suffocate" the SVN server,
sending quite a lot of requests for images and taking way too long to
consume the responses. So, after a (random) while, the SVN server gives
up (timeout) and the FOP application on the other side crashes
immediately after. At least that is how we understand the problem after
testing and reading all sorts of server logs (with a sysadmin) for a
whole day.
This looks like a major regression and makes 0.95beta completely
unusable for us -- though we badly need the new features :-(
If anybody had an idea, I must say I'd be extremely grateful...
Jean-Francois El Fouly
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Major issue with image loading in FOP 0.95beta
Posted by Jean-François El Fouly <je...@elfouly.fr>.
I'll dive but just to answer the simplest questions:
Jeremias Maerki a écrit :
> Without being able to reproduce the behaviour it is difficult to help.
> Some further questions from my side:
> - How many different PNGs are being accessed?
>
84
> - Are they smaller or bigger files?
>
2 kb to 395 kb.
> - Are you running FOP in a multi-threaded fashion? Or from many
> different machines against the SVN repository?
>
No, the problem is fully reproductible with one instance of FOP (running
in JBoss which is a multithreaded server but FOP is given one thread
only, basically the document is generated as a response from a HTTP
request).
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Major issue with image loading in FOP 0.95beta
Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
Hmm, I guess it was a wise decision to make a beta release before the
final. Please note that the image loading framework is completely new.
The old one (while working fine for a specific set of image types) was a
paing to maintain and extend and it often ate much more memory than
necessary. Some hickups were to be expected and I was hoping I have been
able to fix most of them by now. But maybe not...
http://xmlgraphics.apache.org/fop/0.95/releaseNotes_0.95beta.html
I've done some experiments and put about 20 PNGs (some of which pretty
large) in my private SVN repository (SVN 1.4.5 over HTTPD 2.2). I ran a
multi-threaded test (Sun JVM 1.5.0_15, 4 threads, FOP 0.95beta with
XG Commons 1.3 and FOP Trunk with XG Commons Trunk) against that setup.
My machine: WinXP SP2, Intel Core2Duo E6600
But I cannot see any errors or strange behaviour. Once all the images
have been loaded once they are not requested from the SVN server again
(at least as long there's enough memory in the JVM). The only thing I've
seen is that if I start a number of threads simultaneously and let them
render the same FO document, each document causes the same files to be
loaded as the framework currently doesn't synchronize concurrent initial
requests for the same URL. Once the PNG is in the cache there are no
more requests against the SVN server.
I dialed the available memory down so the image cache cannot hold all
the images in cache to force the image framework to reload the images
every time. Nothing bad happened. But I don't have a large hardware park
that would allow me to throw more at my SVN repository.
Another thing I've done is put an TCP proxy in between to find out
what's going on on that level. There was no obvious misbehaviour to be
seen there.
I fear you'll have to dive in yourself and find out what exactly is
going wrong. Just to give you an idea what's going on:
FOP always "preloads" an image before it fully loads it. That means that
it detects the image format and extracts some basic metadata so FOP
knows about the image's intrinsic size. That is used for layout and
allows us not to fully load the image into memory at layout time. Only
when rendering the output file the image is fully loaded. Whenever
possible the original connection that the preloading has been done with
is reused (because some people generate dynamic images using servlets).
The basic change related to your problem with the new image loading
framework: Instead of working with normal InputStreams, the new
framework works with ImageIO's ImageInputStream which provides
convenient methods for working with image files. For non-File-URLs, the
ImageInputStream is created using ImageIO.createImageInputStream(Object).
By default, ImageIO uses a "cached stream" where the stream data is
copied to a local file in a temp directory from which the actual data is
accessed. One thing you can try is to disable this caching to see if
anything changes: ImageIO.setUseCache(false) should do that (call it at
application startup or before you create the FopFactory). If that
changes the behaviour it may give us an idea where to look next.
You can also enable some logging in the image loading framework to get
some information what's going on inside. That would be the
"org.apache.xmlgraphics.image.loader" category.
Without being able to reproduce the behaviour it is difficult to help.
Some further questions from my side:
- How many different PNGs are being accessed?
- Are they smaller or bigger files?
- Are you running FOP in a multi-threaded fashion? Or from many
different machines against the SVN repository?
On 05.05.2008 18:09:55 Jean-François El Fouly wrote:
> Andreas Delmelle a écrit :
> > On May 5, 2008, at 17:22, Jean-François El Fouly wrote:
> >
> > Hi
> >
> >> Sorry, I guess in the strictest sense this is xmlgraphics-common
> >> related now, but it was discovered and investigated using FOP 0.95beta.
> >>
> >> We upgraded from 0.94 to use the new bookmarks features and our
> >> software became globally unstable, hanging randomly during PDF
> >> generation -- to be precise, getting all kinds of random problems
> >> during image loading.
> >> The images we use are stored in SVN and retrieved via their HTTP URLs
> >> from an SVN server. Reading the SVN logs, we see a very different
> >> behaviour between 0.94 (which worked fine and has been retested
> >> since) and 0.95beta: the FOP requests litteraly "suffocate" the SVN
> >> server, sending quite a lot of requests for images and taking way too
> >> long to consume the responses. So, after a (random) while, the SVN
> >> server gives up (timeout) and the FOP application on the other side
> >> crashes immediately after. At least that is how we understand the
> >> problem after testing and reading all sorts of server logs (with a
> >> sysadmin) for a whole day.
> >> This looks like a major regression and makes 0.95beta completely
> >> unusable for us -- though we badly need the new features :-(
> >>
> >> If anybody had an idea, I must say I'd be extremely grateful...
> >
> > Two questions, for the moment:
> > Which image format(s) are you using?
> PNG only (but lots of them).
>
> > Does the JAI/ImageIO implementation on the box where FOP runs, have a
> > native codec to read that format?
> >
> Will ask / investigate (it's a production server to which I don't have
> direct access). Debian 4.0 on JDK 1.5.0_11 and JBoss 4.2.1.GA.
> Thanks !
Jeremias Maerki
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: charts not rendered on Sun Solaris
Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
I have never used JFreeChart. I assume the chart:multiBarChart function
generates SVG since you embed it in an instream-foreign-object. Even for
FOP 0.20.5, the note about the graphical environment needed by Batik
applies equally: http://xmlgraphics.apache.org/fop/0.94/graphics.html#batik
So my initial suspicion is probably correct. Please make sure that your
Java application has a proper graphical environment to work with.
On 07.05.2008 11:20:03 Harshini Madurapperuma wrote:
> Hi
>
> Sorry about it.
>
> I'm using a JfreeChart a third party library to generate charts. Then the outcome to a XSL will look like this:
>
> <fo:table-cell border-style="none" white-space-collapse="false" vldt-object="FoTableCell" element-id="old test_18">
> <fo:block vldt-object="MultiBarChart">
> <fo:instream-foreign-object>
> <xsl:copy-of select="chart:multiBarChart(null, tns:OS_USER, tns:SESSION_ID, '', '', 'Arial', '12' ,0,-12566464, '', 'Arial', '18' ,1,-12566464, 'BOTTOM', 'Arial','12','0', 0, 0, -43691, -11184641, -11141291, -171, -43521, -11141121, -20561, -8355712, -4194304, -16777024, 5.820832666666667,17.0)"/>
> </fo:instream-foreign-object>
> </fo:block>
> </fo:table-cell>
>
> And my FOP version is quite old it's 0.20.5. And difficult to upgrade it to a latest version for the moment.
>
> Regards
> Many thanks in Advance
> Harshini
>
>
>
>
>
>
> -----Original Message-----
> From: Jeremias Maerki [mailto:dev@jeremias-maerki.ch]
> Sent: Wednesday, May 07, 2008 12:22 PM
> To: fop-users@xmlgraphics.apache.org
> Subject: Re: charts not rendered on Sun Solaris
>
> As a rule of thumb: Questions which don't require to ask back because there is enough information to work with are more likely to get answered quickly.
>
> Unfortunately, that's the case here. You didn't:
> - say which FOP version you're using (Don't assume everyone remembers from earlier posts which FOP version you use, or looks it up each time).
> - say what kind of "charts" these are. Are they PNGs? TIFFs? SVGs?
> Something else?
> All that can make a difference. You can save yourself and us time if you put more information in your questions.
>
> If I had to guess, I'd say you don't have the graphical environment set up properly on the Solaris machine. Batiks needs that. See:
> http://xmlgraphics.apache.org/fop/0.94/graphics.html#batik
>
> On 07.05.2008 05:44:49 Harshini Madurapperuma wrote:
> >
> > Hi
> >
> > Can anybody give a suggestion to my problem please :(
> >
> > /Harshini
> >
> > -----Original Message-----
> > From: Harshini Madurapperuma [mailto:harshini.madurapperuma@ifs.lk]
> > Sent: Tuesday, May 06, 2008 2:24 PM
> > To: fop-users@xmlgraphics.apache.org
> > Subject: Fo:charts not rendered on Sun Solaris
> >
> >
> > Hi all
> >
> > I tried to preview a report(with charts) in a Sun Solaris 10
> > environment, but the report is not displayed for Preview. But the same
> > report is previewed correctly in this same environment after the chart
> > is removed from the report.
> >
> > The same report is previewed (with charts) without problems when it is
> > running on a Windows machine.
> >
> > Is there a problem to render a chart when run on Sun Solaris 10. Or
> > should I set some parameters in a special way?
> >
> >
> > Your support is greatly appreciated
> > Many thanks in Advance
> > Harshini
>
>
> Jeremias Maerki
Jeremias Maerki
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
RE: charts not rendered on Sun Solaris
Posted by Harshini Madurapperuma <ha...@ifs.lk>.
Hi
Sorry about it.
I'm using a JfreeChart a third party library to generate charts. Then the outcome to a XSL will look like this:
<fo:table-cell border-style="none" white-space-collapse="false" vldt-object="FoTableCell" element-id="old test_18">
<fo:block vldt-object="MultiBarChart">
<fo:instream-foreign-object>
<xsl:copy-of select="chart:multiBarChart(null, tns:OS_USER, tns:SESSION_ID, '', '', 'Arial', '12' ,0,-12566464, '', 'Arial', '18' ,1,-12566464, 'BOTTOM', 'Arial','12','0', 0, 0, -43691, -11184641, -11141291, -171, -43521, -11141121, -20561, -8355712, -4194304, -16777024, 5.820832666666667,17.0)"/>
</fo:instream-foreign-object>
</fo:block>
</fo:table-cell>
And my FOP version is quite old it's 0.20.5. And difficult to upgrade it to a latest version for the moment.
Regards
Many thanks in Advance
Harshini
-----Original Message-----
From: Jeremias Maerki [mailto:dev@jeremias-maerki.ch]
Sent: Wednesday, May 07, 2008 12:22 PM
To: fop-users@xmlgraphics.apache.org
Subject: Re: charts not rendered on Sun Solaris
As a rule of thumb: Questions which don't require to ask back because there is enough information to work with are more likely to get answered quickly.
Unfortunately, that's the case here. You didn't:
- say which FOP version you're using (Don't assume everyone remembers from earlier posts which FOP version you use, or looks it up each time).
- say what kind of "charts" these are. Are they PNGs? TIFFs? SVGs?
Something else?
All that can make a difference. You can save yourself and us time if you put more information in your questions.
If I had to guess, I'd say you don't have the graphical environment set up properly on the Solaris machine. Batiks needs that. See:
http://xmlgraphics.apache.org/fop/0.94/graphics.html#batik
On 07.05.2008 05:44:49 Harshini Madurapperuma wrote:
>
> Hi
>
> Can anybody give a suggestion to my problem please :(
>
> /Harshini
>
> -----Original Message-----
> From: Harshini Madurapperuma [mailto:harshini.madurapperuma@ifs.lk]
> Sent: Tuesday, May 06, 2008 2:24 PM
> To: fop-users@xmlgraphics.apache.org
> Subject: Fo:charts not rendered on Sun Solaris
>
>
> Hi all
>
> I tried to preview a report(with charts) in a Sun Solaris 10
> environment, but the report is not displayed for Preview. But the same
> report is previewed correctly in this same environment after the chart
> is removed from the report.
>
> The same report is previewed (with charts) without problems when it is
> running on a Windows machine.
>
> Is there a problem to render a chart when run on Sun Solaris 10. Or
> should I set some parameters in a special way?
>
>
> Your support is greatly appreciated
> Many thanks in Advance
> Harshini
Jeremias Maerki
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
------------------------------------------------------------------------------
CONFIDENTIALITY AND DISCLAIMER NOTICE
This e-mail, including any attachments, is confidential and intended only for
the addressee. If you are not the intended recipient, please notify us
immediately and delete this e-mail from your system. Any use or disclosure of
the information contained herein is strictly prohibited.
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: charts not rendered on Sun Solaris
Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
As a rule of thumb: Questions which don't require to ask back because
there is enough information to work with are more likely to get answered
quickly.
Unfortunately, that's the case here. You didn't:
- say which FOP version you're using (Don't assume everyone remembers
from earlier posts which FOP version you use, or looks it up each time).
- say what kind of "charts" these are. Are they PNGs? TIFFs? SVGs?
Something else?
All that can make a difference. You can save yourself and us time if you
put more information in your questions.
If I had to guess, I'd say you don't have the graphical environment set
up properly on the Solaris machine. Batiks needs that. See:
http://xmlgraphics.apache.org/fop/0.94/graphics.html#batik
On 07.05.2008 05:44:49 Harshini Madurapperuma wrote:
>
> Hi
>
> Can anybody give a suggestion to my problem please :(
>
> /Harshini
>
> -----Original Message-----
> From: Harshini Madurapperuma [mailto:harshini.madurapperuma@ifs.lk]
> Sent: Tuesday, May 06, 2008 2:24 PM
> To: fop-users@xmlgraphics.apache.org
> Subject: Fo:charts not rendered on Sun Solaris
>
>
> Hi all
>
> I tried to preview a report(with charts) in a Sun Solaris 10
> environment, but the report is not displayed for Preview. But the same report is
> previewed correctly in this same environment after the chart is removed
> from the report.
>
> The same report is previewed (with charts) without problems when it is
> running on a Windows machine.
>
> Is there a problem to render a chart when run on Sun Solaris 10. Or
> should I set some parameters in a special way?
>
>
> Your support is greatly appreciated
> Many thanks in Advance
> Harshini
Jeremias Maerki
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
RE: charts not rendered on Sun Solaris
Posted by Harshini Madurapperuma <ha...@ifs.lk>.
Hi
Can anybody give a suggestion to my problem please :(
/Harshini
-----Original Message-----
From: Harshini Madurapperuma [mailto:harshini.madurapperuma@ifs.lk]
Sent: Tuesday, May 06, 2008 2:24 PM
To: fop-users@xmlgraphics.apache.org
Subject: Fo:charts not rendered on Sun Solaris
Hi all
I tried to preview a report(with charts) in a Sun Solaris 10 environment, but the report is not displayed for Preview. But the same report is previewed correctly in this same environment after the chart is removed from the report.
The same report is previewed (with charts) without problems when it is running on a Windows machine.
Is there a problem to render a chart when run on Sun Solaris 10. Or should I set some parameters in a special way?
Your support is greatly appreciated
Many thanks in Advance
Harshini
------------------------------------------------------------------------------
CONFIDENTIALITY AND DISCLAIMER NOTICE
This e-mail, including any attachments, is confidential and intended only for the addressee. If you are not the intended recipient, please notify us immediately and delete this e-mail from your system. Any use or disclosure of the information contained herein is strictly prohibited.
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
------------------------------------------------------------------------------
CONFIDENTIALITY AND DISCLAIMER NOTICE
This e-mail, including any attachments, is confidential and intended only for
the addressee. If you are not the intended recipient, please notify us
immediately and delete this e-mail from your system. Any use or disclosure of
the information contained herein is strictly prohibited.
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Fo:charts not rendered on Sun Solaris
Posted by Harshini Madurapperuma <ha...@ifs.lk>.
Hi all
I tried to preview a report(with charts) in a Sun Solaris 10 environment, but the report is not displayed for Preview. But the same report is previewed correctly in this same environment after the chart is removed from the report.
The same report is previewed (with charts) without problems when it is running on a Windows machine.
Is there a problem to render a chart when run on Sun Solaris 10. Or should I set some parameters in a special way?
Your support is greatly appreciated
Many thanks in Advance
Harshini
------------------------------------------------------------------------------
CONFIDENTIALITY AND DISCLAIMER NOTICE
This e-mail, including any attachments, is confidential and intended only for
the addressee. If you are not the intended recipient, please notify us
immediately and delete this e-mail from your system. Any use or disclosure of
the information contained herein is strictly prohibited.
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Major issue with image loading in FOP 0.95beta
Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
On 06.05.2008 10:29:52 Jean-François El Fouly wrote:
> Jeremias Maerki a écrit :
> > I've just re-read that and it suddenly made me think: could it be that
> > you produce really large FO documents (not many smaller ones) with many
> > images and the effect here is simply the timeout of the HTTP connection because
> > it is kept open after preloading the image? I'll add a system property
> > to the image loader framework so you can disable the stream reusage.
> > That could work around the problem. For a longer-term solution I see two
> > possibilities:
> > 1. Add a hint to fo:external-graphic that a URL is dynamic and that the
> > stream should be reused. Otherwise, it is closed immediately and
> > reopened when the full image is read. That idea is not new but never got
> > realized.
> > 2. Add some intelligence to the stream cache so it closes the stream if
> > it is not rerequested within a given time frame to avoid timeouts.
> >
> In fact we produce a very large document but it used to make FOP explode
> (OutOfMemoryError) just to generate one section. So we generate each
> chapter separately and "bind" them using iText. Basically that is why we
> need 0.95beta to use the named destinations, otherwise we can't add
> bookmarks at the end.
> So the problem occurs generating just one chapter.
Hehe, in that case I'm pretty sure that FOP 0.95beta should be able to
generate the full document in one run once we've sorted out the timout
problem. Provided you generate one page-sequence for each chapter.
> Here are the figures:
> - That chapter has 174 images, 84 unique PNG's for a total of 14 Mb. The
> smallest image is 2 kb, the largest is 395 kb.
> The whole chapter generated with FOP 0.94 is 69 pages, for a PDF that
> weighs 14.8 Mb.
>
> The whole document is about 900 pages and currently weighs about 150 Mb.
Ok, that gives me a good confidence that my temporary fix should address
your problem.
Jeremias Maerki
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Memory issue (was: Re: Major issue with image loading in FOP 0.95beta)
Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
I've done extensive tests about memory allocation with FOP when
implementing the new image loading framework and in my case the memory
was always released. So, some results from a profiler would be helpful.
Anyway, what I meant with my hint was that the iText step might not be
necessary anymore and that you should be able to safely reduce -Xmx on
the JVM (provided your document doesn't contain to much non-image
content). But if you release FopFactory and the memory is still not
reclaimed something's wrong somewhere and I have a somewhat hard time
believing that FOP itself somehow still holds on to it. Please make sure
you don't hold on to the FOUserAgent and other FOP-related objects
because they might have a reference to the FopFactory.
On 08.05.2008 08:40:55 Jean-François El Fouly wrote:
> Jeremias Maerki a écrit :
> >> And my next problem is to find a way to force memory recycling after
> >> this long and hefty FOP processing, but until further investigated this
> >> is OT ;-)
> >>
> >
> > You probably didn't get my hint earlier but with the new image loading
> > framework you should actually get away with lower memory settings. In my
> > tests I have been able to produce PDF with little text and many images
> > with 40MB of VM memory which wasn't possible with 0.94 and earlier.
> >
>
> Well, I got the hint, but it seems in contradiction with what I read.
> So to take the picture from a bit higher:
> - all XSL-FO transformation + FOP generation now work OK.
> - this generates 20-30 documents (chapters) for a grand total of about
> 150 Mb, to be bound together by iText.
> - source XML is 1.5 Mb
> - 1011 PNG images for a total of 151 Mb, the largest image is 715 kb.
>
> Now the figures:
> - XML -> XSL-FO transformation + FOP generation take 15 minutes on a
> pretty decent DELL Server (running Debian 4.0) having all the physical
> RAM possible (staging server for several customers)
> - JVM has 2000 Mb (which is BTW the grand max on this
> processor/server/OS/JVM architecture)
> - only one instance of FOP launched (one document generation)
> - the second next step in the publication process (opening the 150 Mb
> with iText to add the bookmarks) fails immediately (at file open) saying
> it cannot allocate memory
>
> If I try to investigate memory usage using
> Runtime.getRuntime().getFreeMemory() and logging the figures with log4j,
> these are the figures I get:
> - before XSLT + FOP: 1900 Mb free/2000 Mb
> - end of XSLT + FOP: 241 Mb free
> - set FopFactory instance to null as a desperate hint to the GC that FOP
> objects could be/should be recycled
> - I force garbage collection using System.gc() [OK, in an application
> server this is a brute force approach, but could not see a more clever
> maneuver ATM]
> - 350 Mb free/2000 Mb total
> - Bind all chapters with iText
> - 250 Mb free
> - Force another GC
> - 350 Mb free again (so the binding operation has no effect on the
> available memory).
> - the next iText step still fails.
>
> Now I don't take runtime.getXXXMemory() for bible word but at least it
> "looks like" the Xalan + FOP subsystem hogs 1500 Mb of RAM which I
> cannot recover.
> So I hired the team member who's competent in profiler usage next week
> but I must say at the moment I'm still stuck :-(
>
> Of course I've made my homework and read the f...riendly manual before
> daring to ask.
> Did I miss any important indication ?
>
Jeremias Maerki
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Memory issue
Posted by Jean-François El Fouly <je...@elfouly.fr>.
Jean-François El Fouly a écrit :
>
> So I hired the team member who's competent in profiler usage next week
> but I must say at the moment I'm still stuck :-(
>
The sysadmins made a tarball from the staging server and copied
everything to a similar server that has full profiling instrumentation
(with JProfiler).
And obviously there the application works and memory usage is quite
normal. Completely different behaviour. Very, very strange.
I'll try to understand what's happening (well, I badly need to) but it's
probably not going to be easy :-(
Re: Memory issue
Posted by Andreas Delmelle <an...@telenet.be>.
On May 8, 2008, at 12:03, Andreas Delmelle wrote:
Hi Jean-François,
> On May 8, 2008, at 12:57, Jean-François El Fouly wrote:
>>
>> Andreas Delmelle a écrit :
>>> OK. Just curious: Any chance you could test it on another build
>>> or maybe even Java 6?
>>>
>> Probably, if required or useful. Our sys admins are very
>> cooperative ;-)
>>>
For the moment, that would be more a nice-to-know. Chances are that,
if it's not JVM-related, this won't help a thing, so no need to go
out of your way to do that
<snip />
> Yes. That is exactly what happened to the stylesheet we use. I've
> reduced it drastically.
> One issue with stylesheets generated by StyleVision is that you
> must be careful when you tweak them to avoid certain [fo-block
> inside fo:inline] combinations that make FOP crash with a stack
> trace and no really useful information about what's happening or
> where. This bug is mentioned in the FOP bug tracker, though in a
> rather raw, loose manner. I removed all such constructs and that
> made the XSLT much simpler and cleaner.
>>
OK, so we can exclude that as well.
<snip />
>> AFAIU, this gives little opportunity for the XSLT processor to
>> clean up anything. Java 1.5 uses Xalan XSLTC by default, which
>> converts templates into Java objects. One giant template would
>> then mean one very long-living object that may reference numerous
>> others for the whole duration of the processing run. If you look
>> at the chain, when using XML+XSLT input, FOP is always the first
>> one to finish, then the XSLT processor, then the XML parser.
>> If the XSLT processor cannot reclaim anything, this will give FOP
>> less room to work with, so it ultimately runs slower. As the heap
>> increases to reach the maximum, the points where the JVM will
>> launch the GC by itself, will also increase. Since it cannot
>> expand the heap anymore, it will try to clean up more frequently.
> Yep, that is why I've tried to be cautious not to accuse FOP
> publicly ;-)
... which is also why /we/ are so cooperative/responsive. ;-)
BTW: If all users would have the time and motivation to be as
thorough as yourself, the traffic on this list would probably drop
significantly.
> The problem is in the (Xalan + FOP) subsystem and the profiling
> could well show that the issue is Xalan-related.
Or maybe even Xerces...? Xerces is a very feature-complete parser,
but reports in the past have shown that all those nice features come
with a price-tag. For FOP this holds as well, of course, and to be
honest, FOP can be a pretty memory-hungry beast if you're not careful
(but you definitely seem to be).
A relatively easy way to find out whether it's XSLT-related, would be
to try out Saxon instead. I don't know if you have any experience
with plugging in a different XSLT processor, but this is pretty
straightforward (but might require re-starting the JBoss service,
depending on how you go about it; for testing purposes, you could
ultimately also change the app-code to reference Saxon directly
instead of letting the JVM choose the
javax.xml.transform.TransformerFactory implementation, and then
redeploy).
Cheers
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Memory issue
Posted by Jean-François El Fouly <je...@elfouly.fr>.
Andreas Delmelle a écrit :
> OK. Just curious: Any chance you could test it on another build or
> maybe even Java 6?
>
Probably, if required or useful. Our sys admins are very cooperative ;-)
>
> In my personal experience, optimizing the stylesheet code usually does
> not offer much improvement in terms of global memory usage, but it
> could have a noticeable impact on the processing time. One of the
> things I've learned about generated XSL-FO stylesheets by Altova is
> that they add a lot of fo:inlines to specify, for example,
> font-properties on the lowest levels in the generated FO while, when
> comparing to the font-properties of the fo:inlines' parents nothing
> really changes, except for the size, style or weight. From FOP's point
> of view, that's somewhat of a waste. Much better to specify a global
> font-size on the page-sequence, and override on the lower levels only
> what is really necessary. After adapting the stylesheet manually, and
> removing the redundant fo:inlines, the stylesheet and the generated FO
> were reduced to not even half the original size.
>
Yes. That is exactly what happened to the stylesheet we use. I've
reduced it drastically.
One issue with stylesheets generated by StyleVision is that you must be
careful when you tweak them to avoid certain [fo-block inside fo:inline]
combinations that make FOP crash with a stack trace and no really useful
information about what's happening or where. This bug is mentioned in
the FOP bug tracker, though in a rather raw, loose manner. I removed all
such constructs and that made the XSLT much simpler and cleaner.
> Something else that bothered me, but I don't know if that was also
> generated by Altova, is that in one of the stylesheets I saw, the
> entire transformation was contained in one giant template...
With the last version, or our XSLT ? this was no longer the case.
> AFAIU, this gives little opportunity for the XSLT processor to clean
> up anything. Java 1.5 uses Xalan XSLTC by default, which converts
> templates into Java objects. One giant template would then mean one
> very long-living object that may reference numerous others for the
> whole duration of the processing run. If you look at the chain, when
> using XML+XSLT input, FOP is always the first one to finish, then the
> XSLT processor, then the XML parser.
> If the XSLT processor cannot reclaim anything, this will give FOP less
> room to work with, so it ultimately runs slower. As the heap increases
> to reach the maximum, the points where the JVM will launch the GC by
> itself, will also increase. Since it cannot expand the heap anymore,
> it will try to clean up more frequently.
Yep, that is why I've tried to be cautious not to accuse FOP publicly ;-)
The problem is in the (Xalan + FOP) subsystem and the profiling could
well show that the issue is Xalan-related.
BTW, we've made the Xalan-FOP coupling a parameter so that we can use
tight coupling (with Sax events) or loose coupling (writing the
intermediate FO files on disk). We usually use the second option, since
the possibility to read the FO intermediate code is helpful when you
debug. And I guess without being really sure that not to have Xalan and
FOP working at the same time should use less memory. This separation
probably accounts for the long execution time, but that is not an issue
since document generation does not occur often in the target system (you
can generate chapters for proofreading but you generate the whole
document once-twice a day).
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Memory issue
Posted by Andreas Delmelle <an...@telenet.be>.
On May 8, 2008, at 11:38, Jean-François El Fouly wrote:
> Andreas Delmelle a écrit :
>>
>> Which Java VM are you using? Practically every time someone tells
>> us about memory/GC issues, it appears they are using an
>> implementation other than Sun (IBM, GNU...)
>> Up to now, we still have to find out why precisely non-Sun VMs
>> have difficulties with FOP...
>>
> Nope. I'll double check but I'm pretty sure it's a genuine Sun JVM
> 1.5.0_11, or maybe the very minor build after.
OK. Just curious: Any chance you could test it on another build or
maybe even Java 6?
>> How large would the resulting FO-files be if you dump them to the
>> filesystem? The XML by itself says very little. From a 1.5MB XML,
>> you could get a FO of a few KB or one of 26MB, depending on the
>> stylesheet.
>>
> 5.08 Mb.
That's not what I would call a large FO, so this should be no problem.
>> Does the stylesheet adhere to XSLT best practices? Does it
>> generate a lot of redundant fo:blocks, fo:inlines?
>>
> I hope not. It has been a complicated thing generated by
> StyleVision in the very beginning but it has been simplified and
> tweaked a lot.
In my personal experience, optimizing the stylesheet code usually
does not offer much improvement in terms of global memory usage, but
it could have a noticeable impact on the processing time. One of the
things I've learned about generated XSL-FO stylesheets by Altova is
that they add a lot of fo:inlines to specify, for example, font-
properties on the lowest levels in the generated FO while, when
comparing to the font-properties of the fo:inlines' parents nothing
really changes, except for the size, style or weight. From FOP's
point of view, that's somewhat of a waste. Much better to specify a
global font-size on the page-sequence, and override on the lower
levels only what is really necessary. After adapting the stylesheet
manually, and removing the redundant fo:inlines, the stylesheet and
the generated FO were reduced to not even half the original size.
Something else that bothered me, but I don't know if that was also
generated by Altova, is that in one of the stylesheets I saw, the
entire transformation was contained in one giant template... AFAIU,
this gives little opportunity for the XSLT processor to clean up
anything. Java 1.5 uses Xalan XSLTC by default, which converts
templates into Java objects. One giant template would then mean one
very long-living object that may reference numerous others for the
whole duration of the processing run. If you look at the chain, when
using XML+XSLT input, FOP is always the first one to finish, then the
XSLT processor, then the XML parser.
If the XSLT processor cannot reclaim anything, this will give FOP
less room to work with, so it ultimately runs slower. As the heap
increases to reach the maximum, the points where the JVM will launch
the GC by itself, will also increase. Since it cannot expand the heap
anymore, it will try to clean up more frequently.
Cheers
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Memory issue
Posted by Jean-François El Fouly <je...@elfouly.fr>.
Andreas Delmelle a écrit :
>
> Which Java VM are you using? Practically every time someone tells us
> about memory/GC issues, it appears they are using an implementation
> other than Sun (IBM, GNU...)
> Up to now, we still have to find out why precisely non-Sun VMs have
> difficulties with FOP...
>
Nope. I'll double check but I'm pretty sure it's a genuine Sun JVM
1.5.0_11, or maybe the very minor build after.
> How large would the resulting FO-files be if you dump them to the
> filesystem? The XML by itself says very little. From a 1.5MB XML, you
> could get a FO of a few KB or one of 26MB, depending on the stylesheet.
>
5.08 Mb.
> Does the stylesheet adhere to XSLT best practices? Does it generate a
> lot of redundant fo:blocks, fo:inlines?
>
I hope not. It has been a complicated thing generated by StyleVision in
the very beginning but it has been simplified and tweaked a lot.
>
> A nit, for the record: There is no such thing as 'forcing garbage
> collection'. The most you can do with System.gc() is indicate to the
> VM that it should run the GC as soon as possible. Admitted, most
> implementations do run the algorithm virtually immediately upon
> execution of the statement, but the Java spec does not mandate such
> behavior. In theory, if the VM is too busy, it could still postpone
> the actual GC-run, until it acquires the necessary resources...
>
Indeed, but the log4j log has timestamps and they show that 20 seconds
are spent around System.gc() so my guess is that something really
happens at that time.
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Memory issue (was: Re: Major issue with image loading in FOP 0.95beta)
Posted by Andreas Delmelle <an...@telenet.be>.
On May 8, 2008, at 08:40, Jean-François El Fouly wrote:
Hi
> Jeremias Maerki a écrit :
>>
>>> And my next problem is to find a way to force memory recycling
>>> after this long and hefty FOP processing, but until further
>>> investigated this is OT ;-)
>> You probably didn't get my hint earlier but with the new image
>> loading framework you should actually get away with lower memory
>> settings. In my tests I have been able to produce PDF with little
>> text and many images with 40MB of VM memory which wasn't possible
>> with 0.94 and earlier.
>
> Well, I got the hint, but it seems in contradiction with what I read.
There are, of course, other factors to take into account than simply
document/image sizes.
Which Java VM are you using? Practically every time someone tells us
about memory/GC issues, it appears they are using an implementation
other than Sun (IBM, GNU...)
Up to now, we still have to find out why precisely non-Sun VMs have
difficulties with FOP...
What options does the Java VM offer to tweak the GC? What options
does it use by default?
> So to take the picture from a bit higher:
> - all XSL-FO transformation + FOP generation now work OK.
> - this generates 20-30 documents (chapters) for a grand total of
> about 150 Mb, to be bound together by iText.
> - source XML is 1.5 Mb
> - 1011 PNG images for a total of 151 Mb, the largest image is 715 kb.
>
> Now the figures:
> - XML -> XSL-FO transformation + FOP generation take 15 minutes on
> a pretty decent DELL Server (running Debian 4.0) having all the
> physical RAM possible (staging server for several customers)
How large would the resulting FO-files be if you dump them to the
filesystem? The XML by itself says very little. From a 1.5MB XML, you
could get a FO of a few KB or one of 26MB, depending on the stylesheet.
Does the stylesheet adhere to XSLT best practices? Does it generate a
lot of redundant fo:blocks, fo:inlines?
> - JVM has 2000 Mb (which is BTW the grand max on this processor/
> server/OS/JVM architecture)
<snip />
On my end, that has proven to be more than enough to generate one
page-sequence with a table of 15 columns, spanning 500+ pages. (Note:
only text-content, no images; more a test to check the memory usage
without doing anything special, just a whole lot of FOs)
> If I try to investigate memory usage using Runtime.getRuntime
> ().getFreeMemory() and logging the figures with log4j, these are
> the figures I get:
> - before XSLT + FOP: 1900 Mb free/2000 Mb
> - end of XSLT + FOP: 241 Mb free
Yikes! That looks troublesome indeed... :/
> - set FopFactory instance to null as a desperate hint to the GC
> that FOP objects could be/should be recycled
> - I force garbage collection using System.gc() [OK, in an
> application server this is a brute force approach, but could not
> see a more clever maneuver ATM]
A nit, for the record: There is no such thing as 'forcing garbage
collection'. The most you can do with System.gc() is indicate to the
VM that it should run the GC as soon as possible. Admitted, most
implementations do run the algorithm virtually immediately upon
execution of the statement, but the Java spec does not mandate such
behavior. In theory, if the VM is too busy, it could still postpone
the actual GC-run, until it acquires the necessary resources...
> <snip />
> Now I don't take runtime.getXXXMemory() for bible word but at least
> it "looks like" the Xalan + FOP subsystem hogs 1500 Mb of RAM which
> I cannot recover.
> So I hired the team member who's competent in profiler usage next
> week but I must say at the moment I'm still stuck :-(
If you're not on a Sun VM, then I have a very vague feeling that he's
going to discover the issue to be related to arrays, a special type
of object, but I could be wrong about this. Someone once reported
that the VM seemed to hold on to a lot of arrays. When profiling, he
discovered that the arrays were referenced nowhere, but still the GC
did not clean them up.
> Of course I've made my homework and read the f...riendly manual
> before daring to ask.
> Did I miss any important indication ?
>
I don't think so, but it seems we might do well by putting some of
the info concerning JVM/GC implementation we have gathered so far, on
the website or a Wiki.
Cheers
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Memory issue (was: Re: Major issue with image loading in FOP 0.95beta)
Posted by Jean-François El Fouly <je...@elfouly.fr>.
Jeremias Maerki a écrit :
>> And my next problem is to find a way to force memory recycling after
>> this long and hefty FOP processing, but until further investigated this
>> is OT ;-)
>>
>
> You probably didn't get my hint earlier but with the new image loading
> framework you should actually get away with lower memory settings. In my
> tests I have been able to produce PDF with little text and many images
> with 40MB of VM memory which wasn't possible with 0.94 and earlier.
>
Well, I got the hint, but it seems in contradiction with what I read.
So to take the picture from a bit higher:
- all XSL-FO transformation + FOP generation now work OK.
- this generates 20-30 documents (chapters) for a grand total of about
150 Mb, to be bound together by iText.
- source XML is 1.5 Mb
- 1011 PNG images for a total of 151 Mb, the largest image is 715 kb.
Now the figures:
- XML -> XSL-FO transformation + FOP generation take 15 minutes on a
pretty decent DELL Server (running Debian 4.0) having all the physical
RAM possible (staging server for several customers)
- JVM has 2000 Mb (which is BTW the grand max on this
processor/server/OS/JVM architecture)
- only one instance of FOP launched (one document generation)
- the second next step in the publication process (opening the 150 Mb
with iText to add the bookmarks) fails immediately (at file open) saying
it cannot allocate memory
If I try to investigate memory usage using
Runtime.getRuntime().getFreeMemory() and logging the figures with log4j,
these are the figures I get:
- before XSLT + FOP: 1900 Mb free/2000 Mb
- end of XSLT + FOP: 241 Mb free
- set FopFactory instance to null as a desperate hint to the GC that FOP
objects could be/should be recycled
- I force garbage collection using System.gc() [OK, in an application
server this is a brute force approach, but could not see a more clever
maneuver ATM]
- 350 Mb free/2000 Mb total
- Bind all chapters with iText
- 250 Mb free
- Force another GC
- 350 Mb free again (so the binding operation has no effect on the
available memory).
- the next iText step still fails.
Now I don't take runtime.getXXXMemory() for bible word but at least it
"looks like" the Xalan + FOP subsystem hogs 1500 Mb of RAM which I
cannot recover.
So I hired the team member who's competent in profiler usage next week
but I must say at the moment I'm still stuck :-(
Of course I've made my homework and read the f...riendly manual before
daring to ask.
Did I miss any important indication ?
Re: [SOLVED] Re: Major issue with image loading in FOP 0.95beta
Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
On 06.05.2008 15:25:20 Jean-François El Fouly wrote:
> Jeremias Maerki a écrit :
> > I've done that now: http://svn.apache.org/viewvc?rev=653704&view=rev
> > Jean-Fraçois, please download XG Commons Trunk, build it and switch to
> > it. Then set
> > -Dorg.apache.xmlgraphics.image.loader.impl.AbstractImageSessionContext.no-source-reuse=true
> > (system property). Hopefully, this will change something. Please let me
> > know if it does.
> >
> Yes, your patch does the trick !
> I've built my chapter successfully, then all the chapters of the
> document and it's OK.
> Congratulations and thanks very much !
Great, now we only have to figure out what the best variant is for a
long-term fix.
> Now I guess you all have to take a decision whether this patch belong to
> 0.95 "release" because it is probably needed in a number of situations...
If possible, this will go into 0.95.
> And my next problem is to find a way to force memory recycling after
> this long and hefty FOP processing, but until further investigated this
> is OT ;-)
You probably didn't get my hint earlier but with the new image loading
framework you should actually get away with lower memory settings. In my
tests I have been able to produce PDF with little text and many images
with 40MB of VM memory which wasn't possible with 0.94 and earlier.
Jeremias Maerki
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
[SOLVED] Re: Major issue with image loading in FOP 0.95beta
Posted by Jean-François El Fouly <je...@elfouly.fr>.
Jeremias Maerki a écrit :
> I've done that now: http://svn.apache.org/viewvc?rev=653704&view=rev
> Jean-Fraçois, please download XG Commons Trunk, build it and switch to
> it. Then set
> -Dorg.apache.xmlgraphics.image.loader.impl.AbstractImageSessionContext.no-source-reuse=true
> (system property). Hopefully, this will change something. Please let me
> know if it does.
>
Yes, your patch does the trick !
I've built my chapter successfully, then all the chapters of the
document and it's OK.
Congratulations and thanks very much !
Now I guess you all have to take a decision whether this patch belong to
0.95 "release" because it is probably needed in a number of situations...
And my next problem is to find a way to force memory recycling after
this long and hefty FOP processing, but until further investigated this
is OT ;-)
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Major issue with image loading in FOP 0.95beta
Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
On 06.05.2008 10:14:03 Jeremias Maerki wrote:
> On 05.05.2008 23:39:15 Jean-François El Fouly wrote:
> > Andreas Delmelle a écrit :
> > > For now, you also spoke about the requests "suffocating" the server.
> > > Do you mean that there are also a lot /more/ requests, or only that
> > > they take longer to process on the FOP-side? If you also have an
> > > increase in the total number of requests, this could mean that the
> > > image-loading framework (unnecessarily?) tries to access the same
> > > images multiple times, which may already provide a pointer as to where
> > > to start looking in the code.
> > >
> > No, but the behaviour looks very different in the server traces.
> >
> > FOP 0.94: a sequence of : one request / one response / one request / one
> > response etc.
> > with constant (server) response time seen in the server logs.
> >
> > Same application with FOP 0.95beta: seems to launch a whole bunch of
> > requests at the same time, say 5..10.15.. requests for different images
> > seen at the same time in the logs. And more a few seconds later.
> > Now the way we interpret the SVN server logs is that the corresponding
> > responses are consumed slower and slower and the SVN server response
> > time traced in the logs is growing in a linear way until it reaches the
> > server timeout (300 s = 5 min. was the default). Then the SVN server
> > supposes nobody's listening to an answer and somehow closes the
> > connection. And then FOP on the other side crashes immediately.
> > Looks somehow like someone very hungry ordering 10 plates at the same
> > time in a restaurant and eating slowly. Until the waiter gives up. (If
> > you forgive me the comparison.)
>
> I've just re-read that and it suddenly made me think: could it be that
> you produce really large FO documents (not many smaller ones) with many
> images and the effect here is simply the timeout of the HTTP connection because
> it is kept open after preloading the image? I'll add a system property
> to the image loader framework so you can disable the stream reusage.
I've done that now: http://svn.apache.org/viewvc?rev=653704&view=rev
Jean-Fraçois, please download XG Commons Trunk, build it and switch to
it. Then set
-Dorg.apache.xmlgraphics.image.loader.impl.AbstractImageSessionContext.no-source-reuse=true
(system property). Hopefully, this will change something. Please let me
know if it does.
> That could work around the problem. For a longer-term solution I see two
> possibilities:
> 1. Add a hint to fo:external-graphic that a URL is dynamic and that the
> stream should be reused. Otherwise, it is closed immediately and
> reopened when the full image is read. That idea is not new but never got
> realized.
> 2. Add some intelligence to the stream cache so it closes the stream if
> it is not rerequested within a given time frame to avoid timeouts.
Jeremias Maerki
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Major issue with image loading in FOP 0.95beta
Posted by Jean-François El Fouly <je...@elfouly.fr>.
Jeremias Maerki a écrit :
> I've just re-read that and it suddenly made me think: could it be that
> you produce really large FO documents (not many smaller ones) with many
> images and the effect here is simply the timeout of the HTTP connection because
> it is kept open after preloading the image? I'll add a system property
> to the image loader framework so you can disable the stream reusage.
> That could work around the problem. For a longer-term solution I see two
> possibilities:
> 1. Add a hint to fo:external-graphic that a URL is dynamic and that the
> stream should be reused. Otherwise, it is closed immediately and
> reopened when the full image is read. That idea is not new but never got
> realized.
> 2. Add some intelligence to the stream cache so it closes the stream if
> it is not rerequested within a given time frame to avoid timeouts.
>
In fact we produce a very large document but it used to make FOP explode
(OutOfMemoryError) just to generate one section. So we generate each
chapter separately and "bind" them using iText. Basically that is why we
need 0.95beta to use the named destinations, otherwise we can't add
bookmarks at the end.
So the problem occurs generating just one chapter.
Here are the figures:
- That chapter has 174 images, 84 unique PNG's for a total of 14 Mb. The
smallest image is 2 kb, the largest is 395 kb.
The whole chapter generated with FOP 0.94 is 69 pages, for a PDF that
weighs 14.8 Mb.
The whole document is about 900 pages and currently weighs about 150 Mb.
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Major issue with image loading in FOP 0.95beta
Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
On 05.05.2008 23:39:15 Jean-François El Fouly wrote:
> Andreas Delmelle a écrit :
> > For now, you also spoke about the requests "suffocating" the server.
> > Do you mean that there are also a lot /more/ requests, or only that
> > they take longer to process on the FOP-side? If you also have an
> > increase in the total number of requests, this could mean that the
> > image-loading framework (unnecessarily?) tries to access the same
> > images multiple times, which may already provide a pointer as to where
> > to start looking in the code.
> >
> No, but the behaviour looks very different in the server traces.
>
> FOP 0.94: a sequence of : one request / one response / one request / one
> response etc.
> with constant (server) response time seen in the server logs.
>
> Same application with FOP 0.95beta: seems to launch a whole bunch of
> requests at the same time, say 5..10.15.. requests for different images
> seen at the same time in the logs. And more a few seconds later.
> Now the way we interpret the SVN server logs is that the corresponding
> responses are consumed slower and slower and the SVN server response
> time traced in the logs is growing in a linear way until it reaches the
> server timeout (300 s = 5 min. was the default). Then the SVN server
> supposes nobody's listening to an answer and somehow closes the
> connection. And then FOP on the other side crashes immediately.
> Looks somehow like someone very hungry ordering 10 plates at the same
> time in a restaurant and eating slowly. Until the waiter gives up. (If
> you forgive me the comparison.)
I've just re-read that and it suddenly made me think: could it be that
you produce really large FO documents (not many smaller ones) with many
images and the effect here is simply the timeout of the HTTP connection because
it is kept open after preloading the image? I'll add a system property
to the image loader framework so you can disable the stream reusage.
That could work around the problem. For a longer-term solution I see two
possibilities:
1. Add a hint to fo:external-graphic that a URL is dynamic and that the
stream should be reused. Otherwise, it is closed immediately and
reopened when the full image is read. That idea is not new but never got
realized.
2. Add some intelligence to the stream cache so it closes the stream if
it is not rerequested within a given time frame to avoid timeouts.
Jeremias Maerki
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Major issue with image loading in FOP 0.95beta
Posted by Jean-François El Fouly <je...@elfouly.fr>.
Andreas Delmelle a écrit :
> For now, you also spoke about the requests "suffocating" the server.
> Do you mean that there are also a lot /more/ requests, or only that
> they take longer to process on the FOP-side? If you also have an
> increase in the total number of requests, this could mean that the
> image-loading framework (unnecessarily?) tries to access the same
> images multiple times, which may already provide a pointer as to where
> to start looking in the code.
>
No, but the behaviour looks very different in the server traces.
FOP 0.94: a sequence of : one request / one response / one request / one
response etc.
with constant (server) response time seen in the server logs.
Same application with FOP 0.95beta: seems to launch a whole bunch of
requests at the same time, say 5..10.15.. requests for different images
seen at the same time in the logs. And more a few seconds later.
Now the way we interpret the SVN server logs is that the corresponding
responses are consumed slower and slower and the SVN server response
time traced in the logs is growing in a linear way until it reaches the
server timeout (300 s = 5 min. was the default). Then the SVN server
supposes nobody's listening to an answer and somehow closes the
connection. And then FOP on the other side crashes immediately.
Looks somehow like someone very hungry ordering 10 plates at the same
time in a restaurant and eating slowly. Until the waiter gives up. (If
you forgive me the comparison.)
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Major issue with image loading in FOP 0.95beta
Posted by Andreas Delmelle <an...@telenet.be>.
On May 5, 2008, at 18:09, Jean-François El Fouly wrote:
>>
>>
>> Two questions, for the moment:
>> Which image format(s) are you using?
> PNG only (but lots of them).
OK, we recently bumped into Mac OS not having a native TIFF codec.
With a bit of fiddling, one could make the Sun default pure Java
implementation available as a fallback, but that ran about 30 times
slower than FOP's own TIFF loader from 0.94.
Maybe a similar thing is happening with PNG, I don't know. I really
hope Jeremias chimes in later...
For now, you also spoke about the requests "suffocating" the server.
Do you mean that there are also a lot /more/ requests, or only that
they take longer to process on the FOP-side? If you also have an
increase in the total number of requests, this could mean that the
image-loading framework (unnecessarily?) tries to access the same
images multiple times, which may already provide a pointer as to
where to start looking in the code.
Cheers
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Major issue with image loading in FOP 0.95beta
Posted by Jean-François El Fouly <je...@elfouly.fr>.
Andreas Delmelle a écrit :
> On May 5, 2008, at 17:22, Jean-François El Fouly wrote:
>
> Hi
>
>> Sorry, I guess in the strictest sense this is xmlgraphics-common
>> related now, but it was discovered and investigated using FOP 0.95beta.
>>
>> We upgraded from 0.94 to use the new bookmarks features and our
>> software became globally unstable, hanging randomly during PDF
>> generation -- to be precise, getting all kinds of random problems
>> during image loading.
>> The images we use are stored in SVN and retrieved via their HTTP URLs
>> from an SVN server. Reading the SVN logs, we see a very different
>> behaviour between 0.94 (which worked fine and has been retested
>> since) and 0.95beta: the FOP requests litteraly "suffocate" the SVN
>> server, sending quite a lot of requests for images and taking way too
>> long to consume the responses. So, after a (random) while, the SVN
>> server gives up (timeout) and the FOP application on the other side
>> crashes immediately after. At least that is how we understand the
>> problem after testing and reading all sorts of server logs (with a
>> sysadmin) for a whole day.
>> This looks like a major regression and makes 0.95beta completely
>> unusable for us -- though we badly need the new features :-(
>>
>> If anybody had an idea, I must say I'd be extremely grateful...
>
> Two questions, for the moment:
> Which image format(s) are you using?
PNG only (but lots of them).
> Does the JAI/ImageIO implementation on the box where FOP runs, have a
> native codec to read that format?
>
Will ask / investigate (it's a production server to which I don't have
direct access). Debian 4.0 on JDK 1.5.0_11 and JBoss 4.2.1.GA.
Thanks !
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Major issue with image loading in FOP 0.95beta
Posted by Andreas Delmelle <an...@telenet.be>.
On May 5, 2008, at 17:22, Jean-François El Fouly wrote:
Hi
> Sorry, I guess in the strictest sense this is xmlgraphics-common
> related now, but it was discovered and investigated using FOP
> 0.95beta.
>
> We upgraded from 0.94 to use the new bookmarks features and our
> software became globally unstable, hanging randomly during PDF
> generation -- to be precise, getting all kinds of random problems
> during image loading.
> The images we use are stored in SVN and retrieved via their HTTP
> URLs from an SVN server. Reading the SVN logs, we see a very
> different behaviour between 0.94 (which worked fine and has been
> retested since) and 0.95beta: the FOP requests litteraly
> "suffocate" the SVN server, sending quite a lot of requests for
> images and taking way too long to consume the responses. So, after
> a (random) while, the SVN server gives up (timeout) and the FOP
> application on the other side crashes immediately after. At least
> that is how we understand the problem after testing and reading all
> sorts of server logs (with a sysadmin) for a whole day.
> This looks like a major regression and makes 0.95beta completely
> unusable for us -- though we badly need the new features :-(
>
> If anybody had an idea, I must say I'd be extremely grateful...
Two questions, for the moment:
Which image format(s) are you using?
Does the JAI/ImageIO implementation on the box where FOP runs, have a
native codec to read that format?
If one of the formats is TIFF, and the answer to the second question
is No, then a possible solution may be to try out a more recent
version of the xmlgraphics-commons jar.
Jeremias has recently re-activated the old FOP TIFF-loader to avoid
the fallback to Sun's default pure Java implementation, which is
significantly slower.
Other than that, I wouldn't have a clue, unfortunately.
HTH!
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org