You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by "Michael Kozakov (JIRA)" <ji...@apache.org> on 2014/07/10 20:56:04 UTC
[jira] [Updated] (JENA-744) Error importing from large gzip
[ https://issues.apache.org/jira/browse/JENA-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Kozakov updated JENA-744:
---------------------------------
Description:
gzip has a documented bug:
http://www.freebsd.org/cgi/man.cgi?query=gzip#end
"According to RFC 1952, the recorded file size is stored in a 32-bit inte-
ger, therefore, it can not represent files larger than 4GB. This limita-
tion also applies to -l option of gzip utility."
As a result, a 28gb compressed gz shows that the uncompressed size is 1.6gb. (screenshot attached)
It seems like tdbloader relies on this information to know when to stop importing, and as a result, the database is incomplete. In other words, I have to extract the archive before using tdbloader to import the database, otherwise it will be missing the majority of items.
was:
gzip has a documented bug:
http://www.freebsd.org/cgi/man.cgi?query=gzip#end
"According to RFC 1952, the recorded file size is stored in a 32-bit inte-
ger, therefore, it can not represent files larger than 4GB. This limita-
tion also applies to -l option of gzip utility."
As a result, a 26gb compressed gz shows that the uncompressed size is 1.6gb. (screenshot attached)
It seems like tdbloader relies on this information to know when to stop importing, and as a result, the database is incomplete. In other words, I have to extract the archive before using tdbloader to import the database, otherwise it will be missing the majority of items.
> Error importing from large gzip
> -------------------------------
>
> Key: JENA-744
> URL: https://issues.apache.org/jira/browse/JENA-744
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB
> Reporter: Michael Kozakov
> Attachments: gzip.png
>
>
> gzip has a documented bug:
> http://www.freebsd.org/cgi/man.cgi?query=gzip#end
> "According to RFC 1952, the recorded file size is stored in a 32-bit inte-
> ger, therefore, it can not represent files larger than 4GB. This limita-
> tion also applies to -l option of gzip utility."
> As a result, a 28gb compressed gz shows that the uncompressed size is 1.6gb. (screenshot attached)
> It seems like tdbloader relies on this information to know when to stop importing, and as a result, the database is incomplete. In other words, I have to extract the archive before using tdbloader to import the database, otherwise it will be missing the majority of items.
--
This message was sent by Atlassian JIRA
(v6.2#6252)