[jira] [Updated] (JENA-44) Support external sorting of bindings in ARQ

["Paolo Castagna (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/JENA-44?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Paolo Castagna updated JENA-44:
-------------------------------

    Attachment: JENA-44_ARQ_r1165123.patch

QueryIterSort now reads the threshold value from a Symbol in the ExecutionContext (default is Integer.MAX_VALUE == no spilling).

On requestCancel() we call cancel on the emdeddedIterator, close on the SortedDataBag and super.requestCancel.

I added a TestQueryIterSort to make sure we delete temporary files on all the possible cancellation scenarios.

@Andy, @Stephen, what do you think?

I'd like to commit this and close this issue to move onto JENA-45.

> Support external sorting of bindings in ARQ
> -------------------------------------------
>
>                 Key: JENA-44
>                 URL: https://issues.apache.org/jira/browse/JENA-44
>             Project: Jena
>          Issue Type: New Feature
>          Components: ARQ
>            Reporter: Sam Tunnicliffe
>            Assignee: Paolo Castagna
>            Priority: Minor
>         Attachments: JENA-44-0.patch, JENA-44-Depends-on-JENA-99-r1157891.patch, JENA-44_ARQ_r1156212.patch, JENA-44_ARQ_r1165123.patch, JENA-44_ARQ_r8531.patch, JENA-44_ARQ_r8724.patch
>
>
> In QueryIterSort, the sorting of the contents of an Iterator<Binding> is done in memory, using Arrays.sort. This can be problematic where the set to be sorted is large. A possible solution could be to use an external, disk-backed algorithm. A hybrid approach may be better, whereby we attempt the in-memory sort, but when the number of bindings encountered goes over a certain number, resort to the disk-backed variant.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira