You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by "Andy Seaborne (Jira)" <ji...@apache.org> on 2021/10/07 11:02:00 UTC
[jira] [Commented] (JENA-2181) Support zstd compression for nq
files
[ https://issues.apache.org/jira/browse/JENA-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425484#comment-17425484 ]
Andy Seaborne commented on JENA-2181:
-------------------------------------
The code is in on class - {{org.apache.jena.atlas.io.IO}} - if you'd like to create a pull request.
There is support for zStandard in Apache Common Compress. Unlike lucene-core dependencies for Jena are fine and Apache Common Compress is already a dependency.
Jena needs a Java version - a JNI wrapper to C code is a significant additional complexity that in my opinion has a higher barrier for inclusion.{{ riot}} can read from the stdin so an external decompressor can be used already (and in parallel). It may be that the costs of parsing means there is not a significant improvement.
> Support zstd compression for nq files
> -------------------------------------
>
> Key: JENA-2181
> URL: https://issues.apache.org/jira/browse/JENA-2181
> Project: Apache Jena
> Issue Type: Improvement
> Reporter: Ioan Eugen Stan
> Priority: Major
>
> Hello,
> New compression algorightms appear and some bring in benefits.
> ZSTD seems to provide very good compression ratio and also supports fast compress/decompress.
> It would be great to support this additional format for NQ files alongside veteran gzip.
> Extension would be zst.
>
> Related work that hints at zstd benefits: https://issues.apache.org/jira/browse/LUCENE-8739
> Thanks,
> Eugen
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)