You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Bill Graham (JIRA)" <ji...@apache.org> on 2010/03/22 17:59:27 UTC
[jira] Created: (PIG-1313) PigServer leaks memory over time
PigServer leaks memory over time
--------------------------------
Key: PIG-1313
URL: https://issues.apache.org/jira/browse/PIG-1313
Project: Pig
Issue Type: Bug
Reporter: Bill Graham
When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12849453#action_12849453 ]
Alan Gates commented on PIG-1313:
---------------------------------
I'm not sure I understand all the pros and cons of Daniel's suggestion of moving the variables to PigServer versus Bill's suggestion of making them ThreadLocal. The advantages I can see of moving the values to PigServer are:
# It's clearer to other developers what's going on, since they can see that these values are associated with an instance of PigServer. Otherwise we're constructing a hidden dependency between the lifetime of PigServer and the thread it's running in.
# If at some future point Pig's frontend is multi-threaded this will still work (granted this is unlikely or at least far in the future)
The advantage I see with Bill's proposal is it's less change.
Are there other things I'm missing here?
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Attachments: Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Status: Patch Available (was: Open)
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852060#action_12852060 ]
Daniel Dai commented on PIG-1313:
---------------------------------
I will review it.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12849811#action_12849811 ]
Daniel Dai commented on PIG-1313:
---------------------------------
I think ThreadLocal could be a simpler alternative solution for now. In the long term, we should move static FileLocalizer variable into PigServer, since user may not launch different thread for different PigServer, ThreadLocal can not solve all situation. But moving static variables require more code changes. So I am fine to use ThreadLocal for now if it solves Bill's problem. Hi, Bill, can you give a patch on this?
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Attachments: Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851717#action_12851717 ]
Hadoop QA commented on PIG-1313:
--------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12440276/PIG-1313-3.patch
against trunk revision 929236.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/272/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/272/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/272/console
This message is automatically generated.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Status: Patch Available (was: Open)
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Attachment: PIG-1313-0.4.0-4.patch
Here's the same patch for 0.4.0 if anyone wants it.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.7.0
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.8.0
>
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-0.4.0-4.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, PIG-1313-4.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848968#action_12848968 ]
Bill Graham commented on PIG-1313:
----------------------------------
Looking more closely at FileLocalizer it does seem that by just making toDelete, deleteOnFail and relativeRoot into ThreadLocals, this issue around multiple threads and temp files could be solved. If there's agreement on this approach, I can take on this issue and contribute a patch. Thoughts?
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Attachments: Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851087#action_12851087 ]
Hadoop QA commented on PIG-1313:
--------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12440102/PIG-1313-0.4.0-1.patch
against trunk revision 928384.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/255/console
This message is automatically generated.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Attachment: PIG-1313-0.4.0-1.patch
PIG-1313-1.patch
Attaching two patches, one built from the trunk and one built from the 0.4.0 branch (since 0.4.0 is what we're using). Not sure if patches to older branches are still being applied or not though. Let me know if I should provide other patches for other branches.
The patch includes the following changes to FileLocalizer:
- toDelete, deleteOnFail and relativeRoot are ThreadLocals.
- The initialized attribute and the init method has been removed, since they're no longer needed.
In PigServer I've done the following:
- FileLocalizer.deleteTempFiles() is called from the shutdown method.
I've run my server with this patch applied for a few days now and the memory consumption has stabilized. I also no longer see exceptions in the logs from when a PigServers temp files would go missing.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Status: Patch Available (was: Open)
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848391#action_12848391 ]
Daniel Dai commented on PIG-1313:
---------------------------------
Yes, your observation is valid. When we design PigServer, we do not put long running and multi-thread in mind. We already have [PIG-240|https://issues.apache.org/jira/browse/PIG-240] in an attempt to make PigServer multi-thread. In your case, it seems all you need is to intermittently instantiate PigServer and run script within one java VM. So we need to ensure when the first PigServer is done, we reclaim all the resources local to that particular PigServer instance. And PigServer.shutdown sounds like a reasonable place to do that.
FileLocalizer.toDelete and FileLocalizer.deleteOnFail are such resources local to PigServer instance. Seems that we should move them into PigServer so that we can clear them safely during shutdown.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Status: Open (was: Patch Available)
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Attachment: PIG-1313-2.patch
Previous patch caused issues with dependencies in test code, which I've resolved in PIG-1313-2.patch. Also, the 0.4.0 patch was getting picked up and was causing failures. Holding off on reattaching that patch until tests pass with the trunk patch.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1313:
----------------------------
Component/s: impl
Affects Version/s: 0.7.0
Fix Version/s: 0.8.0
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.7.0
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.8.0
>
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, PIG-1313-4.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12849846#action_12849846 ]
Daniel Dai commented on PIG-1313:
---------------------------------
In this case, we can skip the unit test and test it manually, just put a notes on the Jira. Thanks!
I don't have the permission to assign it to you, but you cannot see "assign to me" link on your Jira?
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Attachments: Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Status: Open (was: Patch Available)
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1313) PigServer leaks memory over time
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates reassigned PIG-1313:
-------------------------------
Assignee: Bill Graham
I've added you to the contributors list in JIRA, so you should now be able to assign issues to yourself.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852240#action_12852240 ]
Daniel Dai commented on PIG-1313:
---------------------------------
I read the patch again. Two things I notice:
1. PigServer.shutdown() should be explicitly called user in order to take advantage of it. We need to put some document on it.
2. deleteOnFail is now dangling, no one is calling registerDeleteOnFail/triggerDeleteOnFail. We will address it in a seperate Jira.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851102#action_12851102 ]
Hadoop QA commented on PIG-1313:
--------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12440107/PIG-1313-1.patch
against trunk revision 928384.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/256/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/256/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/256/console
This message is automatically generated.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Attachment: PIG-1313-4.patch
# Attaching PIG-1313-4.patch with additional Javadocs on PigServer and PigServer.shutdown().
# triggerDeleteOnFail is still called by TestMultQueryLocal.executePlan, but you are correct in that no one is (or was) calling registerDeleteOnFail, which is the only entry point to push something onto the deleteOnFail stack.
I will gladly remove deleteOnFail and all calls to it as part of this JIRA, or we can handle it in another one if that's cleaner w.r.t. issue tracking. Let me know.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, PIG-1313-4.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12849836#action_12849836 ]
Bill Graham commented on PIG-1313:
----------------------------------
Sure, I can work on a patch. Not sure how I'd create a unit test for this though, since it's a change to how private internals are managed. We should be able to assert the fix though with the attached test harness that reproduces the issue.
Can you please assign this JIRA to me, or give me permissions to assign?
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Attachments: Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1313:
----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
PIG-1313-4.patch committed to trunk. Will come with Pig 0.8 release. This issue is about memory leak and it is hard to write a unit test for it. Bill tested it manually and it works.
Thanks Bill for contributing!
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.7.0
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.8.0
>
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-0.4.0-4.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, PIG-1313-4.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Attachment: PIG-1313-3.patch
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852463#action_12852463 ]
Daniel Dai commented on PIG-1313:
---------------------------------
Thanks, Bill,
Let's leave triggerDeleteOnFail. This is the thing we want to fix. I've opened another Jira [PIG-1347|https://issues.apache.org/jira/browse/PIG-1347] for that. This patch is good to go and I will commit it shortly.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.7.0
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.8.0
>
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, PIG-1313-4.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12849501#action_12849501 ]
Bill Graham commented on PIG-1313:
----------------------------------
You summed it up well Alan. My ThreadLocal suggestion was really just because we could modify one class internally instead of doing a much larger refactor.
I'm unclear though on how we'd go about moving FileLocalizer.toDelete and FileLocalizer.deleteOnFail into PigServer? Currently, calls to the FileLocalizer methods that create these temp file objects happen all over the codebase in places where the calling code wouldn't have a handle to their PigServer instance AFAIK. Unless they could get the PigServer from the PigContext or something of the sort. Otherwise, it would need to be a static call to PigServer methods, and we've just moved the same problem to another class.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Attachments: Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852202#action_12852202 ]
Daniel Dai commented on PIG-1313:
---------------------------------
+1. This patch faithfully translate static variable into ThreadLocal variable. Will commit it shortly.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, PIG-1313-3.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848442#action_12848442 ]
Bill Graham commented on PIG-1313:
----------------------------------
Thanks for the pointer to PIG-240. The patch included make it look like that bug is focusing specifically on the LogicalPlan refactor.
In my case I do need a multi-threaded PigServer instance. My VM is running a scheduler which wakes up and invokes scripts periodically and it is possible that one script will be kicked off while another is already running. As a result, calling FileLocalizer.deleteTempFiles() can cause problem. This is because the toDelete Stack and others are static.
One possible fix would be to bind a toDelete Stack to each thread local instance, or to make FileLocalizer non-static. The later seems cleaner but would require modifying a lot more code.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Attachment: PIG-1313-1.patch
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Attachment: Pig1313Reproducer.java
Attaching {{Pig1313Reproducer.java}}, which is a class that can be used to reproduce the issue. It re-runs a PigServer job in an infinite loop.
It's difficult to expose run time info about the toDelete Stack because it's private but you can run a script like this to repeatedly dump the size/count of the elements on the Stack:
{{while true; do jmap -histo <pid> | grep HFile; sleep 5; done}}
Over time the overall count of HFile objects will increase.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Attachments: Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1313) PigServer leaks memory over time
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851228#action_12851228 ]
Hadoop QA commented on PIG-1313:
--------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12440121/PIG-1313-2.patch
against trunk revision 928384.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/257/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/257/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/257/console
This message is automatically generated.
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Status: Open (was: Patch Available)
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, PIG-1313-1.patch, PIG-1313-2.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1313) PigServer leaks memory over time
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-1313:
-----------------------------
Release Note: PigServer shutdown() method cleans up after itself better.
Status: Patch Available (was: Open)
> PigServer leaks memory over time
> --------------------------------
>
> Key: PIG-1313
> URL: https://issues.apache.org/jira/browse/PIG-1313
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-1313-0.4.0-1.patch, PIG-1313-1.patch, Pig1313Reproducer.java
>
>
> When {{PigServer}} runs it creates temporary files using the {{FileLocalizer.getTemporaryPath(..)}}. This static method creates and returns a handle to a temporary file (as an instance of {{ElementDescriptor}}). The {{ElementDescriptors}} returned by this method are kept on a static {{Stack}} named {{toDelete}}. The items on {{toDelete}} get removed by the {{FileLocalizer.deleteTempFile()}} method.
> The only place in the code where I see {{FileLocalizer.deleteTempFile()}} called is in the Main class. {{PigServer}} does not call that method though, so a long-running VM that repeatedly uses instances of {{PigServer}} to run jobs will leak memory via {{toDelete}}.
> One suggested fix is to have {{PigServer.shutdown()}} call {{FileLocalizer.deleteTempFile()}}, but this would cause problems in a multi-threaded environment, since it seems {{ElementDescriptors}} are pushed onto the {{toDelete}} stack before they're used, not once they're done with. With this approach, running multiple instances of {{PigServer}} in separate threads could cause one completed job to clobber the other's still-in-use temp files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.