You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by "Matt Gilman (JIRA)" <ji...@apache.org> on 2014/12/05 21:40:12 UTC
[jira] [Created] (NIFI-71) Persistent Prov Repo should compress in
blocks
Matt Gilman created NIFI-71:
-------------------------------
Summary: Persistent Prov Repo should compress in blocks
Key: NIFI-71
URL: https://issues.apache.org/jira/browse/NIFI-71
Project: Apache NiFi
Issue Type: Improvement
Reporter: Matt Gilman
Priority: Minor
Currently we write a bunch of events to a file and then compress the file. We then index the file offset of the uncompressed version of the file.
We should instead compress in chunks of X number of events of X number of bytes. Then index the offset of the chunk in the compressed version. This way, we can use FileInputStream.skip to seek to the appropriate offset and then wrap the stream in GZIPInputStream. This allwos us to avoid reading a lot of compressed data to get to the desired offset.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)