[jira] [Comment Edited] (PHOENIX-4910) Improvements to spooled MappedByteBufferQueue files

["Josh Elser (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16626574#comment-16626574 ] 

Josh Elser edited comment on PHOENIX-4910 at 9/24/18 11:48 PM:
---------------------------------------------------------------

.002 Adds deleteOnExit to both SpillFile and ServerCacheClient (found the latter by doing a quick look around at {{Files.createTempFile}} usages)

I tried to write a test for this on the ORDER BY case. I was able to see in the debugger that the files were created in the correct directory, but the RS would clean them up before I could inspect the filesystem. I'm open to suggestions on how to test this if there are thoughts :)


was (Author: elserj):
.002 Adds deleteOnExit to both SpillFile and ServerCacheClient (found the latter by doing a quick look around at \{{Files.createTempFile}} usages)

> Improvements to spooled MappedByteBufferQueue files
> ---------------------------------------------------
>
>                 Key: PHOENIX-4910
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4910
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Major
>             Fix For: 4.15.0, 5.1.0
>
>         Attachments: PHOENIX-4910.001.patch, PHOENIX-4910.002.patch
>
>
> A user ran into a JVM bug which appears to have caused a RegionServer to crash while running a topN aggregate query. This left a large number of files in {{/tmp}} after the RS had gone away (due to a JVM SIGBUS crash). MappedByteBufferQueue will buffer results in memory up to 20MB by default (controlled by {{phoenix.query.spoolThresholdBytes}}) and then start appending them to a file. I'm seeing two things which could be improved:
>  * If the RS exits abnormally, there is no process to clean up files - would be nice to register the {{deleteOnExit()}} hook to try to clean these up.
>  * There is no ability to control where MappedByteBufferQueue writes its spool file - would be nice to use something other than /tmp (I think we have a property to control this already in our config..)
> FYI [~ankit@apache.org]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)