You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by "Paolo Castagna (JIRA)" <ji...@apache.org> on 2011/06/13 12:23:51 UTC
[jira] [Issue Comment Edited] (JENA-44) Support external sorting of
bindings in ARQ
[ https://issues.apache.org/jira/browse/JENA-44?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043406#comment-13043406 ]
Paolo Castagna edited comment on JENA-44 at 6/13/11 10:21 AM:
--------------------------------------------------------------
@Andy
1) BindingSerializer uses NodeFmtLib.serialize(Node node) to serialize a Node into a byte[], but it uses a ByteBuffer internally (which needs to be expanded as necessary) and then makes a copy onto a byte[]. To deserialize a byte[] into a Node it uses a Node decode(byte[] bytes) method taken from TDB com.hp.hlp.jena.tdb.nodetable.NodecSSE.
2) JENA uses a symbol since that is the common way to configure internal parameters within ARQ and it allows to be changed by users. Isn't it?
3) We could have something adaptive, but I would have this as a further improvement. (*)
(*) We are currently using a patched version of ARQ with JENA-44 and this has solved some of the problems we were experiencing on our servers.
was (Author: castagna):
@Andy
1) BindingSerializer uses NodeFmtLib.serialize(Node node) to serialize a Node into a byte[], but it uses a ByteBuffer internally (which needs to be expanded as necessary) and then makes a copy onto a byte[]. To deserialize a byte[] into a Node it uses a Node decode(byte[] bytes) method taken from TDB com.hp.hlp.jena.tdb.nodetable.NodecSSE.
2) JENA uses a symbol since that is the common way to configure internal parameters within ARQ and it allows to be changed by users. Isn't it?
3) We could have something adaptive, but I would have though this as a further improvement. (*)
(*) We are currently using a patched version of ARQ with JENA-44 and this has solved some of the problems we were experiencing on our servers.
> Support external sorting of bindings in ARQ
> -------------------------------------------
>
> Key: JENA-44
> URL: https://issues.apache.org/jira/browse/JENA-44
> Project: Jena
> Issue Type: New Feature
> Components: ARQ
> Reporter: Sam Tunnicliffe
> Assignee: Paolo Castagna
> Priority: Minor
> Attachments: JENA-44-0.patch, JENA-44_ARQ_r8531.patch, JENA-44_ARQ_r8724.patch
>
>
> In QueryIterSort, the sorting of the contents of an Iterator<Binding> is done in memory, using Arrays.sort. This can be problematic where the set to be sorted is large. A possible solution could be to use an external, disk-backed algorithm. A hybrid approach may be better, whereby we attempt the in-memory sort, but when the number of bindings encountered goes over a certain number, resort to the disk-backed variant.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira