You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Damien Katz (JIRA)" <ji...@apache.org> on 2010/06/15 20:53:23 UTC
[jira] Created: (COUCHDB-800) Problem when writing larger than 4kb
file headers
Problem when writing larger than 4kb file headers
-------------------------------------------------
Key: COUCHDB-800
URL: https://issues.apache.org/jira/browse/COUCHDB-800
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 0.11
Reporter: Damien Katz
Assignee: Damien Katz
Fix For: 0.11.1, 0.12
>From Andrey Somov:
Hi,
while reading the CouchDB source I found a question in couch_file.erl,
I am not sure whether it is a bug or not.
Lines 297-311:
handle_call({write_header, Bin}, _From, #file{fd=Fd, eof=Pos}=File) ->
BinSize = size(Bin),
case Pos rem ?SIZE_BLOCK of
0 ->
Padding = <<>>;
BlockOffset ->
Padding = <<0:(8*(?SIZE_BLOCK-BlockOffset))>>
end,
FinalBin = [Padding, <<1, BinSize:32/integer>> | make_blocks(1, [Bin])],
case file:write(Fd, FinalBin) of
ok ->
{reply, ok, File#file{eof=Pos+iolist_size(FinalBin)}};
Error ->
{reply, Error, File}
end;
Because <<1, BinSize:32/integer>> occupies 5 bytes make_blocks() shall
use offset=5, but the offset is only 1.
(it should be make_blocks(5, [Bin]))
Since the header is smaller then 4k there is no difference and it
works (the tests succeed with both 1 and 5). But it makes it more
difficult to understand the code for those who study the source to
understand how it works.
-
Thank you,
Andrey
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-800) Problem when writing larger than
4kb file headers
Posted by "Andrey Somov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915356#action_12915356 ]
Andrey Somov commented on COUCHDB-800:
--------------------------------------
But the issue has been fixed by Damian 3 months ago. I think I can try to deliver patches once I learn more about CouchDB internals.
> Problem when writing larger than 4kb file headers
> -------------------------------------------------
>
> Key: COUCHDB-800
> URL: https://issues.apache.org/jira/browse/COUCHDB-800
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.11
> Reporter: Damien Katz
> Assignee: Damien Katz
> Fix For: 0.11.1, 0.12
>
>
> From Andrey Somov:
> Hi,
> while reading the CouchDB source I found a question in couch_file.erl,
> I am not sure whether it is a bug or not.
> Lines 297-311:
> handle_call({write_header, Bin}, _From, #file{fd=Fd, eof=Pos}=File) ->
> BinSize = size(Bin),
> case Pos rem ?SIZE_BLOCK of
> 0 ->
> Padding = <<>>;
> BlockOffset ->
> Padding = <<0:(8*(?SIZE_BLOCK-BlockOffset))>>
> end,
> FinalBin = [Padding, <<1, BinSize:32/integer>> | make_blocks(1, [Bin])],
> case file:write(Fd, FinalBin) of
> ok ->
> {reply, ok, File#file{eof=Pos+iolist_size(FinalBin)}};
> Error ->
> {reply, Error, File}
> end;
> Because <<1, BinSize:32/integer>> occupies 5 bytes make_blocks() shall
> use offset=5, but the offset is only 1.
> (it should be make_blocks(5, [Bin]))
> Since the header is smaller then 4k there is no difference and it
> works (the tests succeed with both 1 and 5). But it makes it more
> difficult to understand the code for those who study the source to
> understand how it works.
> -
> Thank you,
> Andrey
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-800) Problem when writing larger than
4kb file headers
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915431#action_12915431 ]
Paul Joseph Davis commented on COUCHDB-800:
-------------------------------------------
I'm an idiot. I managed to fix the bug by doing an `svn up`. Happened to be looking at an old copy that was out of date.
> Problem when writing larger than 4kb file headers
> -------------------------------------------------
>
> Key: COUCHDB-800
> URL: https://issues.apache.org/jira/browse/COUCHDB-800
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.11
> Reporter: Damien Katz
> Assignee: Damien Katz
> Fix For: 0.11.1, 0.12
>
>
> From Andrey Somov:
> Hi,
> while reading the CouchDB source I found a question in couch_file.erl,
> I am not sure whether it is a bug or not.
> Lines 297-311:
> handle_call({write_header, Bin}, _From, #file{fd=Fd, eof=Pos}=File) ->
> BinSize = size(Bin),
> case Pos rem ?SIZE_BLOCK of
> 0 ->
> Padding = <<>>;
> BlockOffset ->
> Padding = <<0:(8*(?SIZE_BLOCK-BlockOffset))>>
> end,
> FinalBin = [Padding, <<1, BinSize:32/integer>> | make_blocks(1, [Bin])],
> case file:write(Fd, FinalBin) of
> ok ->
> {reply, ok, File#file{eof=Pos+iolist_size(FinalBin)}};
> Error ->
> {reply, Error, File}
> end;
> Because <<1, BinSize:32/integer>> occupies 5 bytes make_blocks() shall
> use offset=5, but the offset is only 1.
> (it should be make_blocks(5, [Bin]))
> Since the header is smaller then 4k there is no difference and it
> works (the tests succeed with both 1 and 5). But it makes it more
> difficult to understand the code for those who study the source to
> understand how it works.
> -
> Thank you,
> Andrey
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (COUCHDB-800) Problem when writing larger than 4kb
file headers
Posted by "Damien Katz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Damien Katz closed COUCHDB-800.
-------------------------------
Resolution: Fixed
> Problem when writing larger than 4kb file headers
> -------------------------------------------------
>
> Key: COUCHDB-800
> URL: https://issues.apache.org/jira/browse/COUCHDB-800
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.11
> Reporter: Damien Katz
> Assignee: Damien Katz
> Fix For: 0.11.1, 0.12
>
>
> From Andrey Somov:
> Hi,
> while reading the CouchDB source I found a question in couch_file.erl,
> I am not sure whether it is a bug or not.
> Lines 297-311:
> handle_call({write_header, Bin}, _From, #file{fd=Fd, eof=Pos}=File) ->
> BinSize = size(Bin),
> case Pos rem ?SIZE_BLOCK of
> 0 ->
> Padding = <<>>;
> BlockOffset ->
> Padding = <<0:(8*(?SIZE_BLOCK-BlockOffset))>>
> end,
> FinalBin = [Padding, <<1, BinSize:32/integer>> | make_blocks(1, [Bin])],
> case file:write(Fd, FinalBin) of
> ok ->
> {reply, ok, File#file{eof=Pos+iolist_size(FinalBin)}};
> Error ->
> {reply, Error, File}
> end;
> Because <<1, BinSize:32/integer>> occupies 5 bytes make_blocks() shall
> use offset=5, but the offset is only 1.
> (it should be make_blocks(5, [Bin]))
> Since the header is smaller then 4k there is no difference and it
> works (the tests succeed with both 1 and 5). But it makes it more
> difficult to understand the code for those who study the source to
> understand how it works.
> -
> Thank you,
> Andrey
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-800) Problem when writing larger than
4kb file headers
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915401#action_12915401 ]
Paul Joseph Davis commented on COUCHDB-800:
-------------------------------------------
Odd. When I checked this morning it was still in couch_file as a 1 byte offset instead of 5. I didn't spend too much time reading though so maybe its a subtlety somewhere else. The test itself would be as easy as writing a header larger than 4K and checking the byte boundary position.
> Problem when writing larger than 4kb file headers
> -------------------------------------------------
>
> Key: COUCHDB-800
> URL: https://issues.apache.org/jira/browse/COUCHDB-800
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.11
> Reporter: Damien Katz
> Assignee: Damien Katz
> Fix For: 0.11.1, 0.12
>
>
> From Andrey Somov:
> Hi,
> while reading the CouchDB source I found a question in couch_file.erl,
> I am not sure whether it is a bug or not.
> Lines 297-311:
> handle_call({write_header, Bin}, _From, #file{fd=Fd, eof=Pos}=File) ->
> BinSize = size(Bin),
> case Pos rem ?SIZE_BLOCK of
> 0 ->
> Padding = <<>>;
> BlockOffset ->
> Padding = <<0:(8*(?SIZE_BLOCK-BlockOffset))>>
> end,
> FinalBin = [Padding, <<1, BinSize:32/integer>> | make_blocks(1, [Bin])],
> case file:write(Fd, FinalBin) of
> ok ->
> {reply, ok, File#file{eof=Pos+iolist_size(FinalBin)}};
> Error ->
> {reply, Error, File}
> end;
> Because <<1, BinSize:32/integer>> occupies 5 bytes make_blocks() shall
> use offset=5, but the offset is only 1.
> (it should be make_blocks(5, [Bin]))
> Since the header is smaller then 4k there is no difference and it
> works (the tests succeed with both 1 and 5). But it makes it more
> difficult to understand the code for those who study the source to
> understand how it works.
> -
> Thank you,
> Andrey
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-800) Problem when writing larger than
4kb file headers
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915336#action_12915336 ]
Paul Joseph Davis commented on COUCHDB-800:
-------------------------------------------
I do believe you're correct. Think you could make a failing test case and/or patch to show the behaviour?
> Problem when writing larger than 4kb file headers
> -------------------------------------------------
>
> Key: COUCHDB-800
> URL: https://issues.apache.org/jira/browse/COUCHDB-800
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.11
> Reporter: Damien Katz
> Assignee: Damien Katz
> Fix For: 0.11.1, 0.12
>
>
> From Andrey Somov:
> Hi,
> while reading the CouchDB source I found a question in couch_file.erl,
> I am not sure whether it is a bug or not.
> Lines 297-311:
> handle_call({write_header, Bin}, _From, #file{fd=Fd, eof=Pos}=File) ->
> BinSize = size(Bin),
> case Pos rem ?SIZE_BLOCK of
> 0 ->
> Padding = <<>>;
> BlockOffset ->
> Padding = <<0:(8*(?SIZE_BLOCK-BlockOffset))>>
> end,
> FinalBin = [Padding, <<1, BinSize:32/integer>> | make_blocks(1, [Bin])],
> case file:write(Fd, FinalBin) of
> ok ->
> {reply, ok, File#file{eof=Pos+iolist_size(FinalBin)}};
> Error ->
> {reply, Error, File}
> end;
> Because <<1, BinSize:32/integer>> occupies 5 bytes make_blocks() shall
> use offset=5, but the offset is only 1.
> (it should be make_blocks(5, [Bin]))
> Since the header is smaller then 4k there is no difference and it
> works (the tests succeed with both 1 and 5). But it makes it more
> difficult to understand the code for those who study the source to
> understand how it works.
> -
> Thank you,
> Andrey
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.