[jira] Commented: (CLEREZZA-286) Inserting lager literals takes too long

["Reto Bachmann-Gmür (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CLEREZZA-286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902340#action_12902340 ] 

Reto Bachmann-Gmür commented on CLEREZZA-286:
---------------------------------------------

The solution approach I'm planing to implement is the following:

- On write operations objects that are of type base64 are stored (possibly decoded, i.e. just there values) as files with a directory and file name based on a strong hash of the content. instead of the literals a uri (a urn:x-something scheme?) encoding the hash/filename is stored in the undelying graph
- On read operations if the object is of that particular uri-type it is replaced with the literal reconstructed from the file

issues:
- this prevents sparql fastlane for queries containing such a literal
- as the literal could be stored by its binary value it is akward transforming it to base64 on the storage layer and recreate the byte[] in the literal-factory
- this can quite easily be implemented in a single storage provider (i.e. in the TDB provider) but one could image this being a storage option available for any StorageProvider 

> Inserting lager literals takes too long
> ---------------------------------------
>
>                 Key: CLEREZZA-286
>                 URL: https://issues.apache.org/jira/browse/CLEREZZA-286
>             Project: Clerezza
>          Issue Type: Improvement
>            Reporter: Reto Bachmann-Gmür
>            Assignee: Reto Bachmann-Gmür
>
> As binary resources are stored as literal values in the graph inserting literals of many MB is not a seldom usage scenario. Inserting such literals however take very long and thus require a very long write-lock on the graph (e.g. on my laptop 9 seconds when uploading a 40MB file with TDB and 3 second with Sesame)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.