[1/50] git commit: Fix retrieval of headers larger than 4k

[ko...@apache.org]

Updated Branches:
  refs/heads/1319-large-headers-are-corrupted 22fae5003 -> c923e8757 (forced update)


Fix retrieval of headers larger than 4k

Our headers start with a <<1>> and then four bytes indicating the length
of the header and its checksum. When the header is larger than 4090
bytes it will be split across multiple blocks in the file and will need
to be reassembled on read. The reassembly consists of stripping out
<<0>> from the beginning of each subsequent block in the
remove_block_prefixes/2 function. The bug here is that we tell
remove_block_prefixes that we're starting 1 byte into the current block
instead of 5, so it ends up removing one good byte from the header and
injecting one or more random <<0>>s.

Headers larger than 4k are very rare and generally require a view group
with a huge number of indexes or indexes with fairly large reductions,
which explains why this bug has gone undetected until now.


Project: http://git-wip-us.apache.org/repos/asf/couchdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/couchdb/commit/c923e875
Tree: http://git-wip-us.apache.org/repos/asf/couchdb/tree/c923e875
Diff: http://git-wip-us.apache.org/repos/asf/couchdb/diff/c923e875

Branch: refs/heads/1319-large-headers-are-corrupted
Commit: c923e87577f2fbe928c965b758167b865a2b1d3e
Parents: 02e808a
Author: Adam Kocoloski <ko...@apache.org>
Authored: Wed Oct 26 14:04:54 2011 -0400
Committer: Adam Kocoloski <ko...@apache.org>
Committed: Wed Oct 26 14:06:56 2011 -0400

----------------------------------------------------------------------
 src/couchdb/couch_file.erl |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/couchdb/blob/c923e875/src/couchdb/couch_file.erl
----------------------------------------------------------------------
diff --git a/src/couchdb/couch_file.erl b/src/couchdb/couch_file.erl
index f1d6cc4..2c2f11a 100644
--- a/src/couchdb/couch_file.erl
+++ b/src/couchdb/couch_file.erl
@@ -433,7 +433,7 @@ load_header(Fd, Block) ->
         RawBin = <<RestBlock/binary, Missing/binary>>
     end,
     <<Md5Sig:16/binary, HeaderBin/binary>> =
-        iolist_to_binary(remove_block_prefixes(1, RawBin)),
+        iolist_to_binary(remove_block_prefixes(5, RawBin)),
     Md5Sig = couch_util:md5(HeaderBin),
     {ok, HeaderBin}.