You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Randall Leeds (JIRA)" <ji...@apache.org> on 2010/06/05 01:06:54 UTC
[jira] Created: (COUCHDB-786) Make fewer gen_server calls to fsync
the db header
Make fewer gen_server calls to fsync the db header
--------------------------------------------------
Key: COUCHDB-786
URL: https://issues.apache.org/jira/browse/COUCHDB-786
Project: CouchDB
Issue Type: Improvement
Components: Database Core
Affects Versions: 0.11, 0.10.2
Reporter: Randall Leeds
Fix For: 0.12
Attachments: 0001-db-header-fsync-with-fewer-gen_server-calls.patch
Currently the couch_db_updater process is responsible for calling couch_file:sync based on the fsync options when the db header is written. Instead, this patch just passes the fsync options down to couch_file so that writing the header can always be performed with a single gen_server call.
Writes are serialized through couch_db_updater but reads can have high concurrency. This patch should boost write performance when the message queue on couch_file is long.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-786) Make fewer gen_server calls to fsync
the db header
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds updated COUCHDB-786:
----------------------------------
Attachment: (was: 0001-db-header-fsync-with-fewer-gen_server-calls.patch)
> Make fewer gen_server calls to fsync the db header
> --------------------------------------------------
>
> Key: COUCHDB-786
> URL: https://issues.apache.org/jira/browse/COUCHDB-786
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Affects Versions: 0.10.2, 0.11
> Reporter: Randall Leeds
> Fix For: 0.12
>
> Attachments: 0001-db-header-fsync-with-fewer-gen_server-calls.patch
>
>
> Currently the couch_db_updater process is responsible for calling couch_file:sync based on the fsync options when the db header is written. Instead, this patch just passes the fsync options down to couch_file so that writing the header can always be performed with a single gen_server call.
> Writes are serialized through couch_db_updater but reads can have high concurrency. This patch should boost write performance when the message queue on couch_file is long.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (COUCHDB-786) Make fewer gen_server calls to fsync
the db header
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Kocoloski closed COUCHDB-786.
----------------------------------
Assignee: Adam Kocoloski
Fix Version/s: 1.1
(was: 0.12)
Resolution: Fixed
COUCHDB-767 accomplishes the end goal -- only 1 call to the couch_file gen_server in order to commit the header.
> Make fewer gen_server calls to fsync the db header
> --------------------------------------------------
>
> Key: COUCHDB-786
> URL: https://issues.apache.org/jira/browse/COUCHDB-786
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Affects Versions: 0.10.2, 0.11
> Reporter: Randall Leeds
> Assignee: Adam Kocoloski
> Fix For: 1.1
>
> Attachments: 0001-db-header-fsync-with-fewer-gen_server-calls.patch
>
>
> Currently the couch_db_updater process is responsible for calling couch_file:sync based on the fsync options when the db header is written. Instead, this patch just passes the fsync options down to couch_file so that writing the header can always be performed with a single gen_server call.
> Writes are serialized through couch_db_updater but reads can have high concurrency. This patch should boost write performance when the message queue on couch_file is long.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-786) Make fewer gen_server calls to
fsync the db header
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12876305#action_12876305 ]
Adam Kocoloski commented on COUCHDB-786:
----------------------------------------
I'm worried about the interaction of this patch with COUCHDB-767. That issue contains a patch which causes couch_file to spawn a worker process to sync the file. The idea behind COUCHDB-767 is that reads can proceed even while the file is being synced.
It seems that this patch would have the opposite effect -- all readers would be blocked, not just while a file is synced, but while the header is being written (which typically involves 2 syncs).
Perhaps it would be a good idea to combine the strategies from both issues, and spawn a function to do the sync-write-sync dance for write_header? The challenge is that #file.eof needs to be updated when the header is written, which is not the case for a simple sync.
> Make fewer gen_server calls to fsync the db header
> --------------------------------------------------
>
> Key: COUCHDB-786
> URL: https://issues.apache.org/jira/browse/COUCHDB-786
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Affects Versions: 0.10.2, 0.11
> Reporter: Randall Leeds
> Fix For: 0.12
>
> Attachments: 0001-db-header-fsync-with-fewer-gen_server-calls.patch
>
>
> Currently the couch_db_updater process is responsible for calling couch_file:sync based on the fsync options when the db header is written. Instead, this patch just passes the fsync options down to couch_file so that writing the header can always be performed with a single gen_server call.
> Writes are serialized through couch_db_updater but reads can have high concurrency. This patch should boost write performance when the message queue on couch_file is long.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-786) Make fewer gen_server calls to fsync
the db header
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds updated COUCHDB-786:
----------------------------------
Attachment: 0001-db-header-fsync-with-fewer-gen_server-calls.patch
> Make fewer gen_server calls to fsync the db header
> --------------------------------------------------
>
> Key: COUCHDB-786
> URL: https://issues.apache.org/jira/browse/COUCHDB-786
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Affects Versions: 0.10.2, 0.11
> Reporter: Randall Leeds
> Fix For: 0.12
>
> Attachments: 0001-db-header-fsync-with-fewer-gen_server-calls.patch
>
>
> Currently the couch_db_updater process is responsible for calling couch_file:sync based on the fsync options when the db header is written. Instead, this patch just passes the fsync options down to couch_file so that writing the header can always be performed with a single gen_server call.
> Writes are serialized through couch_db_updater but reads can have high concurrency. This patch should boost write performance when the message queue on couch_file is long.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-786) Make fewer gen_server calls to fsync
the db header
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds updated COUCHDB-786:
----------------------------------
Attachment: 0001-db-header-fsync-with-fewer-gen_server-calls.patch
> Make fewer gen_server calls to fsync the db header
> --------------------------------------------------
>
> Key: COUCHDB-786
> URL: https://issues.apache.org/jira/browse/COUCHDB-786
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Affects Versions: 0.10.2, 0.11
> Reporter: Randall Leeds
> Fix For: 0.12
>
> Attachments: 0001-db-header-fsync-with-fewer-gen_server-calls.patch
>
>
> Currently the couch_db_updater process is responsible for calling couch_file:sync based on the fsync options when the db header is written. Instead, this patch just passes the fsync options down to couch_file so that writing the header can always be performed with a single gen_server call.
> Writes are serialized through couch_db_updater but reads can have high concurrency. This patch should boost write performance when the message queue on couch_file is long.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-786) Make fewer gen_server calls to fsync
the db header
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds updated COUCHDB-786:
----------------------------------
Attachment: (was: 0001-db-header-fsync-with-fewer-gen_server-calls.patch)
> Make fewer gen_server calls to fsync the db header
> --------------------------------------------------
>
> Key: COUCHDB-786
> URL: https://issues.apache.org/jira/browse/COUCHDB-786
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Affects Versions: 0.10.2, 0.11
> Reporter: Randall Leeds
> Fix For: 0.12
>
>
> Currently the couch_db_updater process is responsible for calling couch_file:sync based on the fsync options when the db header is written. Instead, this patch just passes the fsync options down to couch_file so that writing the header can always be performed with a single gen_server call.
> Writes are serialized through couch_db_updater but reads can have high concurrency. This patch should boost write performance when the message queue on couch_file is long.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-786) Make fewer gen_server calls to fsync
the db header
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds updated COUCHDB-786:
----------------------------------
Attachment: 0001-db-header-fsync-with-fewer-gen_server-calls.patch
doh. how many times will I forget to run git add.
> Make fewer gen_server calls to fsync the db header
> --------------------------------------------------
>
> Key: COUCHDB-786
> URL: https://issues.apache.org/jira/browse/COUCHDB-786
> Project: CouchDB
> Issue Type: Improvement
> Components: Database Core
> Affects Versions: 0.10.2, 0.11
> Reporter: Randall Leeds
> Fix For: 0.12
>
> Attachments: 0001-db-header-fsync-with-fewer-gen_server-calls.patch
>
>
> Currently the couch_db_updater process is responsible for calling couch_file:sync based on the fsync options when the db header is written. Instead, this patch just passes the fsync options down to couch_file so that writing the header can always be performed with a single gen_server call.
> Writes are serialized through couch_db_updater but reads can have high concurrency. This patch should boost write performance when the message queue on couch_file is long.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.