You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Adam Kocoloski (Created) (JIRA)" <ji...@apache.org> on 2011/10/26 20:03:32 UTC
[jira] [Created] (COUCHDB-1319) Headers larger than 4k cannot be
retrieved
Headers larger than 4k cannot be retrieved
------------------------------------------
Key: COUCHDB-1319
URL: https://issues.apache.org/jira/browse/COUCHDB-1319
Project: CouchDB
Issue Type: Bug
Affects Versions: 1.1
Reporter: Adam Kocoloski
Assignee: Adam Kocoloski
Fix For: 1.2
Our headers start with a <<1>> and then four bytes indicating the length of the header and its checksum. When the header is larger than 4090 bytes it will be split across multiple blocks in the file and will need to be reassembled on read. The reassembly consists of stripping out <<0>> from the beginning of each subsequent block in the remove_block_prefixes/2 function. The bug here is that we tell remove_block_prefixes that we're starting 1 byte into the current block instead of 5, so it ends up removing one or more good bytes from the header and injecting one or more random <<0>>s.
Headers larger than 4k are very rare and generally require a view group with a huge number of indexes or indexes with fairly large reductions, which explains why this bug has gone undetected until now.
Patch forthcoming.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1319) Headers larger than 4k cannot be
retrieved
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139723#comment-13139723 ]
Paul Joseph Davis commented on COUCHDB-1319:
--------------------------------------------
It's a straightforward bug and has a test. I wouldn't have even bothered with a ticket.
> Headers larger than 4k cannot be retrieved
> ------------------------------------------
>
> Key: COUCHDB-1319
> URL: https://issues.apache.org/jira/browse/COUCHDB-1319
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 1.1, 1.1.1
> Reporter: Adam Kocoloski
> Assignee: Adam Kocoloski
> Fix For: 1.2, 1.1.2
>
>
> Our headers start with a <<1>> and then four bytes indicating the length of the header and its checksum. When the header is larger than 4090 bytes it will be split across multiple blocks in the file and will need to be reassembled on read. The reassembly consists of stripping out <<0>> from the beginning of each subsequent block in the remove_block_prefixes/2 function. The bug here is that we tell remove_block_prefixes that we're starting 1 byte into the current block instead of 5, so it ends up removing one or more good bytes from the header and injecting one or more random <<0>>s.
> Headers larger than 4k are very rare and generally require a view group with a huge number of indexes or indexes with fairly large reductions, which explains why this bug has gone undetected until now.
> Patch forthcoming.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1319) Headers larger than 4k cannot be
retrieved
Posted by "Jan Lehnardt (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139717#comment-13139717 ]
Jan Lehnardt commented on COUCHDB-1319:
---------------------------------------
Thanks Adam.
In the future, it wouldn't hurt to ask the side channels to comment here, so we can all see them :)
> Headers larger than 4k cannot be retrieved
> ------------------------------------------
>
> Key: COUCHDB-1319
> URL: https://issues.apache.org/jira/browse/COUCHDB-1319
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 1.1, 1.1.1
> Reporter: Adam Kocoloski
> Assignee: Adam Kocoloski
> Fix For: 1.2, 1.1.2
>
>
> Our headers start with a <<1>> and then four bytes indicating the length of the header and its checksum. When the header is larger than 4090 bytes it will be split across multiple blocks in the file and will need to be reassembled on read. The reassembly consists of stripping out <<0>> from the beginning of each subsequent block in the remove_block_prefixes/2 function. The bug here is that we tell remove_block_prefixes that we're starting 1 byte into the current block instead of 5, so it ends up removing one or more good bytes from the header and injecting one or more random <<0>>s.
> Headers larger than 4k are very rare and generally require a view group with a huge number of indexes or indexes with fairly large reductions, which explains why this bug has gone undetected until now.
> Patch forthcoming.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (COUCHDB-1319) Headers larger than 4k cannot be
retrieved
Posted by "Adam Kocoloski (Closed) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Kocoloski closed COUCHDB-1319.
-----------------------------------
> Headers larger than 4k cannot be retrieved
> ------------------------------------------
>
> Key: COUCHDB-1319
> URL: https://issues.apache.org/jira/browse/COUCHDB-1319
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 1.1, 1.1.1
> Reporter: Adam Kocoloski
> Assignee: Adam Kocoloski
> Fix For: 1.2, 1.1.2
>
>
> Our headers start with a <<1>> and then four bytes indicating the length of the header and its checksum. When the header is larger than 4090 bytes it will be split across multiple blocks in the file and will need to be reassembled on read. The reassembly consists of stripping out <<0>> from the beginning of each subsequent block in the remove_block_prefixes/2 function. The bug here is that we tell remove_block_prefixes that we're starting 1 byte into the current block instead of 5, so it ends up removing one or more good bytes from the header and injecting one or more random <<0>>s.
> Headers larger than 4k are very rare and generally require a view group with a huge number of indexes or indexes with fairly large reductions, which explains why this bug has gone undetected until now.
> Patch forthcoming.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1319) Headers larger than 4k cannot be
retrieved
Posted by "Adam Kocoloski (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Kocoloski resolved COUCHDB-1319.
-------------------------------------
Resolution: Fixed
Got some positive reviews of the patch in side channels so I've gone ahead and landed it on 1.1.x, 1.2.x and master
> Headers larger than 4k cannot be retrieved
> ------------------------------------------
>
> Key: COUCHDB-1319
> URL: https://issues.apache.org/jira/browse/COUCHDB-1319
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 1.1, 1.1.1
> Reporter: Adam Kocoloski
> Assignee: Adam Kocoloski
> Fix For: 1.2, 1.1.2
>
>
> Our headers start with a <<1>> and then four bytes indicating the length of the header and its checksum. When the header is larger than 4090 bytes it will be split across multiple blocks in the file and will need to be reassembled on read. The reassembly consists of stripping out <<0>> from the beginning of each subsequent block in the remove_block_prefixes/2 function. The bug here is that we tell remove_block_prefixes that we're starting 1 byte into the current block instead of 5, so it ends up removing one or more good bytes from the header and injecting one or more random <<0>>s.
> Headers larger than 4k are very rare and generally require a view group with a huge number of indexes or indexes with fairly large reductions, which explains why this bug has gone undetected until now.
> Patch forthcoming.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1319) Headers larger than 4k cannot be
retrieved
Posted by "Adam Kocoloski (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136209#comment-13136209 ]
Adam Kocoloski commented on COUCHDB-1319:
-----------------------------------------
Added an etap test to the branch.
> Headers larger than 4k cannot be retrieved
> ------------------------------------------
>
> Key: COUCHDB-1319
> URL: https://issues.apache.org/jira/browse/COUCHDB-1319
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 1.1, 1.1.1
> Reporter: Adam Kocoloski
> Assignee: Adam Kocoloski
> Fix For: 1.2, 1.1.2
>
>
> Our headers start with a <<1>> and then four bytes indicating the length of the header and its checksum. When the header is larger than 4090 bytes it will be split across multiple blocks in the file and will need to be reassembled on read. The reassembly consists of stripping out <<0>> from the beginning of each subsequent block in the remove_block_prefixes/2 function. The bug here is that we tell remove_block_prefixes that we're starting 1 byte into the current block instead of 5, so it ends up removing one or more good bytes from the header and injecting one or more random <<0>>s.
> Headers larger than 4k are very rare and generally require a view group with a huge number of indexes or indexes with fairly large reductions, which explains why this bug has gone undetected until now.
> Patch forthcoming.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1319) Headers larger than 4k cannot be
retrieved
Posted by "Adam Kocoloski (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136177#comment-13136177 ]
Adam Kocoloski commented on COUCHDB-1319:
-----------------------------------------
http://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=shortlog;h=refs/heads/1319-large-headers-are-corrupted
Will look into an etap test. I think we may also want to log when load_header crashes. That's not normal.
> Headers larger than 4k cannot be retrieved
> ------------------------------------------
>
> Key: COUCHDB-1319
> URL: https://issues.apache.org/jira/browse/COUCHDB-1319
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 1.1, 1.1.1
> Reporter: Adam Kocoloski
> Assignee: Adam Kocoloski
> Fix For: 1.2, 1.1.2
>
>
> Our headers start with a <<1>> and then four bytes indicating the length of the header and its checksum. When the header is larger than 4090 bytes it will be split across multiple blocks in the file and will need to be reassembled on read. The reassembly consists of stripping out <<0>> from the beginning of each subsequent block in the remove_block_prefixes/2 function. The bug here is that we tell remove_block_prefixes that we're starting 1 byte into the current block instead of 5, so it ends up removing one or more good bytes from the header and injecting one or more random <<0>>s.
> Headers larger than 4k are very rare and generally require a view group with a huge number of indexes or indexes with fairly large reductions, which explains why this bug has gone undetected until now.
> Patch forthcoming.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1319) Headers larger than 4k cannot be
retrieved
Posted by "Robert Newson (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Newson updated COUCHDB-1319:
-----------------------------------
Affects Version/s: 1.1.1
Fix Version/s: 1.1.2
Mark for fixing in next 1.1.x maintenance release.
> Headers larger than 4k cannot be retrieved
> ------------------------------------------
>
> Key: COUCHDB-1319
> URL: https://issues.apache.org/jira/browse/COUCHDB-1319
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 1.1, 1.1.1
> Reporter: Adam Kocoloski
> Assignee: Adam Kocoloski
> Fix For: 1.2, 1.1.2
>
>
> Our headers start with a <<1>> and then four bytes indicating the length of the header and its checksum. When the header is larger than 4090 bytes it will be split across multiple blocks in the file and will need to be reassembled on read. The reassembly consists of stripping out <<0>> from the beginning of each subsequent block in the remove_block_prefixes/2 function. The bug here is that we tell remove_block_prefixes that we're starting 1 byte into the current block instead of 5, so it ends up removing one or more good bytes from the header and injecting one or more random <<0>>s.
> Headers larger than 4k are very rare and generally require a view group with a huge number of indexes or indexes with fairly large reductions, which explains why this bug has gone undetected until now.
> Patch forthcoming.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1319) Headers larger than 4k cannot be
retrieved
Posted by "Adam Kocoloski (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139733#comment-13139733 ]
Adam Kocoloski commented on COUCHDB-1319:
-----------------------------------------
Yeah ... the important thing is it got reviewed. Pretty sure Bob, Bob, Paul and Filipe all gave it a once-over. And yes, it's a simple bug with a simple test, but I think changes to "core" pretty much always need to be reviewed at this point.
> Headers larger than 4k cannot be retrieved
> ------------------------------------------
>
> Key: COUCHDB-1319
> URL: https://issues.apache.org/jira/browse/COUCHDB-1319
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 1.1, 1.1.1
> Reporter: Adam Kocoloski
> Assignee: Adam Kocoloski
> Fix For: 1.2, 1.1.2
>
>
> Our headers start with a <<1>> and then four bytes indicating the length of the header and its checksum. When the header is larger than 4090 bytes it will be split across multiple blocks in the file and will need to be reassembled on read. The reassembly consists of stripping out <<0>> from the beginning of each subsequent block in the remove_block_prefixes/2 function. The bug here is that we tell remove_block_prefixes that we're starting 1 byte into the current block instead of 5, so it ends up removing one or more good bytes from the header and injecting one or more random <<0>>s.
> Headers larger than 4k are very rare and generally require a view group with a huge number of indexes or indexes with fairly large reductions, which explains why this bug has gone undetected until now.
> Patch forthcoming.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira