[jira] Commented: (PIG-1519) Stop relying on finalize() to delete files, close filehandles in bag implementations

["Thejas M Nair (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892551#action_12892551 ] 

Thejas M Nair commented on PIG-1519:
------------------------------------

As part of these changes, we should consider keeping a (weak?) reference in the bags to all the iterators that have been created and call clear() (a new method in iterator impl class) that closes the DataInputStreams and invalidates the iterators.


> Stop relying on finalize() to delete files, close filehandles in bag implementations
> ------------------------------------------------------------------------------------
>
>                 Key: PIG-1519
>                 URL: https://issues.apache.org/jira/browse/PIG-1519
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.8.0
>            Reporter: Thejas M Nair
>            Priority: Minor
>
> In DefaultAbstractBag and its subclasses, the files used for spilling to disk are deleted using finalize() . 
> The iterators associated with these bags use DataInputStreams but don't call close on them, and the underlying FileInputStream.close() is called only through FileInputStream.finalize().
> The use of finalize has performance implications and also makes it hard to predict when the resources will get freed. 
> WeakReferences can be used to avoid the use of finalize().  See http://java.sun.com/developer/technicalArticles/javase/finalization/ (look for "An Alternative to Finalization") .
> I have marked the priority has minor because the allocation of these resources objects that have finalize happens only for large bags that spill to disk (see related jira - PIG-1516), so the performance  impact of the use of finalize is not likely to be significant. Also, I haven't come across any case where we have run out of these resources because finalize() thread has not freed them yet.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.