You are viewing a plain text version of this content. The canonical link for it is here.
Posted to fop-users@xmlgraphics.apache.org by Matthieu Siggen <ma...@gmail.com> on 2008/07/28 15:36:56 UTC

Fop instance keeping handle on font file

Hello,

I have a problem where Fop sometimes keeps an open handle on the fonts given
to the FOUserAgent object. The fonts are defined through the
FOUserAgent.setFontBaseURL method.

My code looks like this:

OutputStream out = //this is some output stream
try{ 
    Source source= message.getInputAs(getConfig(), SUPPORTED_IN_TYPES);
    FOUserAgent foUserAgent = fopFactory.newFOUserAgent();

	//defines values on the FOUserAgent through the setters like setFontBaseURL
    configureFOUserAgent(context, foUserAgent);
	
    org.apache.fop.apps.Fop fop = fopFactory.newFop(mimeType, foUserAgent,
out);            
    Result result= new SAXResult(fop.getDefaultHandler());
    
    transformer= transformerPool.lease();
    transformer.transform(source, result);
} finally {
    if ( null != out ) 
        out.close();
}

While debugging I found out that the fopFactory.newFop call obtains the
handle. Unfortunately, the handle seems to be arbitrarly released, ie. a
garbage collection is sometimes necessary to free the handle.
Did anyone encounter the same problem? Is there a way my environment or the
implementation I work on would interfere with fop? (I'm running fop 0.94
with Java 6 on Vista)
Thank you in advance for your help :)

Regards,

Matthieu
-- 
View this message in context: http://www.nabble.com/Fop-instance-keeping-handle-on-font-file-tp18689197p18689197.html
Sent from the FOP - Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org


Re: Fop instance keeping handle on font file

Posted by Matthieu Siggen <ma...@gmail.com>.
Thanks for all your answers. 

Unfortunately, I won't be able to use the GC to free the open handle as:
- we will have loads of concurrent fop transformation running in services
deployed on a same app server instance,
- the open handle makes undeploying a service tricky and manually calling
the GC won't ensure immediate garbage collection.

I guess for now we can survive with Linux servers as it seems the open
handle could be safely ignored on unix systems.

Matthieu
-- 
View this message in context: http://www.nabble.com/Fop-instance-keeping-handle-on-font-file-tp18689197p18716243.html
Sent from the FOP - Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org


Re: Fop instance keeping handle on font file

Posted by Andreas Delmelle <an...@telenet.be>.
On Jul 30, 2008, at 00:19, Jeremias Maerki wrote:

> On 28.07.2008 21:03:09 Andreas Delmelle wrote:
>>
>> As Jeremias already noted, no obvious bugs present. All FOP does, is
>> open a standard InputStream and properly close it when it is no
>> longer needed. After that, it's up to the InputStream implementation
>> to release the native file handle. It could be that this last step
>> does not occur until the InputStream itself is cleared by the GC. So,
>> it's possible that 'arbitrarily' simply means 'upon the next implicit
>> or explicit garbage-collection'.
>
> I consider it highly unlikely (and unwise) that this is implemented  
> like
> that. Resources need to be releases ASAP. That's what  
> InputStream.close
> () is there for.

Well, I'm not passing judgment, but I'm not ruling it out either,  
since I have seen bug reports in those areas where people actually  
suggested to tie the clearing of native file-handles to the GC cycle  
(to release handles held due to deleteOnExit() where the file had  
already been deleted). Besides, as I suspected, and Mathieu seemed to  
confirm (?), on *nix-based platforms, this is not really a problem.  
I'm assuming this to be because 'concurrent access to resources' is  
something that is already present at the lowest levels in the kernel  
and filesystem. IIRC, on Unix it is possible to create a file, write  
to it, then delete it, without the file ever having existed  
physically on disk (if you're fast enough ;-)). You'll never notice,  
but on Windows, one does notice that writing to a file *always*  
implies disk I/O.

>> Anyway, unfortunately it seems like it's not something we can
>> directly help you with. The only workaround may even be to explicitly
>> ask for Runtime.gc() after every run. Then again, I'm wondering
>> whether this could be a showstopper when running multiple
>> simultaneous FOP-sessions in the same environment. :-S
>
> I don't think triggering garbage collection will help at all.
> Runtime.gc() is not even guaranteed to initiate a GC run.

In theory indeed, but in practice the next run will follow ASAP after  
the call has been made. /Especially/ so on Windows, where thread- 
priority sometimes appears to be completely absent, or at least the  
deeper levels of the kernel seem to consider it a joke. What does it  
matter if you assign a priority to processes if, at the lowest levels  
the code does not seem to take them into account? Why does this GUI  
'hang' every once in a while, no matter how much RAM or CPUs you have  
at your disposal? Why does the system suddenly become unresponsive  
when you transfer large files? Because at some point, assumptions are  
made about priorities that seem very wise --from a human point-of- 
view. Not if you think like a machine, though... ;-)


OK, 'nuff ranting done. Back to work.

Cheers

Andreas

---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org


Re: Fop instance keeping handle on font file

Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
On 28.07.2008 21:03:09 Andreas Delmelle wrote:
> On Jul 28, 2008, at 18:03, Matthieu Siggen wrote:
> 
> Hi
> 
> FWIW:
> 
> >>> While debugging I found out that the fopFactory.newFop call  
> >>> obtains the
> >>> handle. Unfortunately, the handle seems to be arbitrarly  
> >>> released, ie. a
> >>> garbage collection is sometimes necessary to free the handle.
> 
> OK, this seems to indicate that:
> a) the issue is not located on the FOP-side
> b) it's not really 'arbitrary'
> 
> If FOP itself would unnecessarily hold on to a TTF, then a GC will  
> not magically release it.
> 
> As Jeremias already noted, no obvious bugs present. All FOP does, is  
> open a standard InputStream and properly close it when it is no  
> longer needed. After that, it's up to the InputStream implementation  
> to release the native file handle. It could be that this last step  
> does not occur until the InputStream itself is cleared by the GC. So,  
> it's possible that 'arbitrarily' simply means 'upon the next implicit  
> or explicit garbage-collection'.

I consider it highly unlikely (and unwise) that this is implemented like
that. Resources need to be releases ASAP. That's what InputStream.close
() is there for.

> Anyway, unfortunately it seems like it's not something we can  
> directly help you with. The only workaround may even be to explicitly  
> ask for Runtime.gc() after every run. Then again, I'm wondering  
> whether this could be a showstopper when running multiple  
> simultaneous FOP-sessions in the same environment. :-S

I don't think triggering garbage collection will help at all.
Runtime.gc() is not even guaranteed to initiate a GC run.

At first I thought that Andreas was a bit quick to give FOP a free pass
here. I've enhanced my little InputStream observer framework I once
started for XML Graphics Commons so I can observe the close() behaviour.
It showed no unclosed streams for the 0.95 branch. Well, I haven't
intercepted all InputStreams being opened by FOP, only the font-related
ones until I could see the font metric files and the TrueType fonts
being loaded.

Matthieu, if you do have open file handles that don't seem to be closed,
I think you'll have to look towards the author of the JVM you're running
on. However, I find it highly unlikely that anyone would make such a
mistake in such a central place. Maybe trying another JVM or another JVM
version might be a good experiment. What we don't know is what tool you
use to observe the file handles and if that produces reliable results.
What triggered your coming here in the first place? Was Vista running
out of file handles?

If you're willing to invest some more time, I can send you a patch on
the 0.95 branch with my stream observer so you could run it in your
environment. Maybe there's still something wrong with FOP somewhere. I
still don't rule that out categorically. But at any rate, I can't
reproduce the problem.


Jeremias Maerki


---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org


Re: Fop instance keeping handle on font file

Posted by Andreas Delmelle <an...@telenet.be>.
On Jul 29, 2008, at 01:37, Daniel Noll wrote:

> Andreas Delmelle wrote:
>> If FOP itself would unnecessarily hold on to a TTF, then a GC will  
>> not magically release it.
>
> But if it had a FileInputStream (for example) open and forgot to  
> close it, then the FileInputStream finaliser would close it later,  
> resulting in the exact same symptoms reported here.

If... then yes. Just checked, and occurrences of FileInputStreams are  
all local variables, and properly closed, so it's hypothetical.


Andreas

---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org


Re: Fop instance keeping handle on font file

Posted by Daniel Noll <da...@nuix.com>.
Andreas Delmelle wrote:
> If FOP itself would unnecessarily hold on to a TTF, then a GC will not 
> magically release it.

But if it had a FileInputStream (for example) open and forgot to close 
it, then the FileInputStream finaliser would close it later, resulting 
in the exact same symptoms reported here.

Daniel


-- 
Daniel Noll

---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org


Re: Fop instance keeping handle on font file

Posted by Andreas Delmelle <an...@telenet.be>.
On Jul 28, 2008, at 18:03, Matthieu Siggen wrote:

Hi

FWIW:

>>> While debugging I found out that the fopFactory.newFop call  
>>> obtains the
>>> handle. Unfortunately, the handle seems to be arbitrarly  
>>> released, ie. a
>>> garbage collection is sometimes necessary to free the handle.

OK, this seems to indicate that:
a) the issue is not located on the FOP-side
b) it's not really 'arbitrary'

If FOP itself would unnecessarily hold on to a TTF, then a GC will  
not magically release it.

As Jeremias already noted, no obvious bugs present. All FOP does, is  
open a standard InputStream and properly close it when it is no  
longer needed. After that, it's up to the InputStream implementation  
to release the native file handle. It could be that this last step  
does not occur until the InputStream itself is cleared by the GC. So,  
it's possible that 'arbitrarily' simply means 'upon the next implicit  
or explicit garbage-collection'.

Anyway, unfortunately it seems like it's not something we can  
directly help you with. The only workaround may even be to explicitly  
ask for Runtime.gc() after every run. Then again, I'm wondering  
whether this could be a showstopper when running multiple  
simultaneous FOP-sessions in the same environment. :-S



Cheers

Andreas

---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org


Re: Fop instance keeping handle on font file

Posted by Matthieu Siggen <ma...@gmail.com>.
My fonts are TTF. I tested with Acens.ttf and msgothic.ttf.
Last Friday, I tried the binary installation of 0.95beta without any
improvement. The same issue occured (sorry for not mentionning this
earlier).


Jeremias Maerki-2 wrote:
> 
> Can you tell on what kinds of files the handles don't get properly
> released? PFM, PFB, TTF or something else? Have you tried 0.95beta (or
> the https://svn.apache.org/repos/asf/xmlgraphics/fop/branches/fop-0_95
> branch) to see if anything changes? We've had quite a few changes in the
> font area for 0.95beta.
> 
> I've looked briefly through the font-related source code that deals with
> files and I don't see any obvious bug, except maybe for the loading of
> XML font metric files in LazyFont.
> 
> On 28.07.2008 15:36:56 Matthieu Siggen wrote:
>> 
>> Hello,
>> 
>> I have a problem where Fop sometimes keeps an open handle on the fonts
>> given
>> to the FOUserAgent object. The fonts are defined through the
>> FOUserAgent.setFontBaseURL method.
>> 
>> My code looks like this:
>> 
>> OutputStream out = //this is some output stream
>> try{ 
>>     Source source= message.getInputAs(getConfig(), SUPPORTED_IN_TYPES);
>>     FOUserAgent foUserAgent = fopFactory.newFOUserAgent();
>> 
>> 	//defines values on the FOUserAgent through the setters like
>> setFontBaseURL
>>     configureFOUserAgent(context, foUserAgent);
>> 	
>>     org.apache.fop.apps.Fop fop = fopFactory.newFop(mimeType,
>> foUserAgent,
>> out);            
>>     Result result= new SAXResult(fop.getDefaultHandler());
>>     
>>     transformer= transformerPool.lease();
>>     transformer.transform(source, result);
>> } finally {
>>     if ( null != out ) 
>>         out.close();
>> }
>> 
>> While debugging I found out that the fopFactory.newFop call obtains the
>> handle. Unfortunately, the handle seems to be arbitrarly released, ie. a
>> garbage collection is sometimes necessary to free the handle.
>> Did anyone encounter the same problem? Is there a way my environment or
>> the
>> implementation I work on would interfere with fop? (I'm running fop 0.94
>> with Java 6 on Vista)
>> Thank you in advance for your help :)
>> 
>> Regards,
>> 
>> Matthieu
>> -- 
>> View this message in context:
>> http://www.nabble.com/Fop-instance-keeping-handle-on-font-file-tp18689197p18689197.html
>> Sent from the FOP - Users mailing list archive at Nabble.com.
>> 
> 
> 
> Jeremias Maerki
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
> For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Fop-instance-keeping-handle-on-font-file-tp18689197p18692691.html
Sent from the FOP - Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org


Re: Fop instance keeping handle on font file

Posted by Jeremias Maerki <de...@jeremias-maerki.ch>.
Can you tell on what kinds of files the handles don't get properly
released? PFM, PFB, TTF or something else? Have you tried 0.95beta (or
the https://svn.apache.org/repos/asf/xmlgraphics/fop/branches/fop-0_95
branch) to see if anything changes? We've had quite a few changes in the
font area for 0.95beta.

I've looked briefly through the font-related source code that deals with
files and I don't see any obvious bug, except maybe for the loading of
XML font metric files in LazyFont.

On 28.07.2008 15:36:56 Matthieu Siggen wrote:
> 
> Hello,
> 
> I have a problem where Fop sometimes keeps an open handle on the fonts given
> to the FOUserAgent object. The fonts are defined through the
> FOUserAgent.setFontBaseURL method.
> 
> My code looks like this:
> 
> OutputStream out = //this is some output stream
> try{ 
>     Source source= message.getInputAs(getConfig(), SUPPORTED_IN_TYPES);
>     FOUserAgent foUserAgent = fopFactory.newFOUserAgent();
> 
> 	//defines values on the FOUserAgent through the setters like setFontBaseURL
>     configureFOUserAgent(context, foUserAgent);
> 	
>     org.apache.fop.apps.Fop fop = fopFactory.newFop(mimeType, foUserAgent,
> out);            
>     Result result= new SAXResult(fop.getDefaultHandler());
>     
>     transformer= transformerPool.lease();
>     transformer.transform(source, result);
> } finally {
>     if ( null != out ) 
>         out.close();
> }
> 
> While debugging I found out that the fopFactory.newFop call obtains the
> handle. Unfortunately, the handle seems to be arbitrarly released, ie. a
> garbage collection is sometimes necessary to free the handle.
> Did anyone encounter the same problem? Is there a way my environment or the
> implementation I work on would interfere with fop? (I'm running fop 0.94
> with Java 6 on Vista)
> Thank you in advance for your help :)
> 
> Regards,
> 
> Matthieu
> -- 
> View this message in context: http://www.nabble.com/Fop-instance-keeping-handle-on-font-file-tp18689197p18689197.html
> Sent from the FOP - Users mailing list archive at Nabble.com.
> 


Jeremias Maerki


---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org