You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Graham (JIRA)" <ji...@apache.org> on 2012/11/21 19:57:58 UTC
[jira] [Commented] (ZOOKEEPER-866) Adding no disk persistence
option in zookeeper.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502200#comment-13502200 ]
Graham commented on ZOOKEEPER-866:
----------------------------------
We ran some benchmarks using zk-latencies.
We tried to ways to boost performance: Using a RAM disk (tmpfs) and using libeatmydata (Makes all file system sync operations no-ops).
libeatmydata benchmarks: http://pastebin.com/cNLjfhPG
Ramdisk using tmpfs benchmarks: http://pastebin.com/mfe92nXn (Note: Different box to the last one)
In summary: Synchronous calls are boosted by two orders of magnitude with either libeatmydata or ramdisk (In Standalone mode and also clustered mode). Asynchronous calls are boosted by a factor of 2 or 3.
For tests, simulations etc. a Zookeeper without snapshots or logs makes a lot of sense, but for production use, the ramdisk or eatmydata options both looks pretty good.
Another thing we found works well is to have a battery backed raid array; writes to go to raid cache and will sync to disk eventually.
> Adding no disk persistence option in zookeeper.
> -----------------------------------------------
>
> Key: ZOOKEEPER-866
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-866
> Project: ZooKeeper
> Issue Type: New Feature
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-nodisk.patch
>
>
> Its been seen that some folks would like to use zookeeper for very fine grained locking. Also, in there use case they are fine with loosing all old zookeeper state if they reboot zookeeper or zookeeper goes down. The use case is more of a runtime locking wherein forgetting the state of locks is acceptable in case of a zookeeper reboot. Not logging to disk allows high throughput on and low latency on the writes to zookeeper. This would be a configuration option to set (ofcourse the default would be logging to disk).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (ZOOKEEPER-866) Adding no disk persistence
option in zookeeper.
Posted by Ted Dunning <te...@gmail.com>.
+1
But of course the speedups are large enough that it is pretty clear that
this will make a significant difference almost regardless of size.
On Wed, Nov 21, 2012 at 12:26 PM, Dayal Dilli <da...@userevents.com>wrote:
> hi Graham,
>
> Could you please update with some more details like the size of znode
> created and some of the system parameters like size of the ramdisk, size of
> RAM etc.. I guess this would give a better idea to compare your results
> with ours and discuss on improvements.
>
> ________________________________________
> From: Graham (JIRA) [jira@apache.org]
> Sent: Wednesday, November 21, 2012 2:57 PM
> To: dev@zookeeper.apache.org
> Subject: [jira] [Commented] (ZOOKEEPER-866) Adding no disk persistence
> option in zookeeper.
>
> [
> https://issues.apache.org/jira/browse/ZOOKEEPER-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502200#comment-13502200]
>
> Graham commented on ZOOKEEPER-866:
> ----------------------------------
>
> We ran some benchmarks using zk-latencies.
>
> We tried to ways to boost performance: Using a RAM disk (tmpfs) and using
> libeatmydata (Makes all file system sync operations no-ops).
>
> libeatmydata benchmarks: http://pastebin.com/cNLjfhPG
>
> Ramdisk using tmpfs benchmarks: http://pastebin.com/mfe92nXn (Note:
> Different box to the last one)
>
> In summary: Synchronous calls are boosted by two orders of magnitude with
> either libeatmydata or ramdisk (In Standalone mode and also clustered
> mode). Asynchronous calls are boosted by a factor of 2 or 3.
>
> For tests, simulations etc. a Zookeeper without snapshots or logs makes a
> lot of sense, but for production use, the ramdisk or eatmydata options both
> looks pretty good.
>
> Another thing we found works well is to have a battery backed raid array;
> writes to go to raid cache and will sync to disk eventually.
>
>
>
> > Adding no disk persistence option in zookeeper.
> > -----------------------------------------------
> >
> > Key: ZOOKEEPER-866
> > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-866
> > Project: ZooKeeper
> > Issue Type: New Feature
> > Reporter: Mahadev konar
> > Assignee: Mahadev konar
> > Fix For: 3.5.0
> >
> > Attachments: ZOOKEEPER-nodisk.patch
> >
> >
> > Its been seen that some folks would like to use zookeeper for very fine
> grained locking. Also, in there use case they are fine with loosing all old
> zookeeper state if they reboot zookeeper or zookeeper goes down. The use
> case is more of a runtime locking wherein forgetting the state of locks is
> acceptable in case of a zookeeper reboot. Not logging to disk allows high
> throughput on and low latency on the writes to zookeeper. This would be a
> configuration option to set (ofcourse the default would be logging to disk).
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
RE: [jira] [Commented] (ZOOKEEPER-866) Adding no disk persistence
option in zookeeper.
Posted by Dayal Dilli <da...@userevents.com>.
hi Graham,
Could you please update with some more details like the size of znode created and some of the system parameters like size of the ramdisk, size of RAM etc.. I guess this would give a better idea to compare your results with ours and discuss on improvements.
________________________________________
From: Graham (JIRA) [jira@apache.org]
Sent: Wednesday, November 21, 2012 2:57 PM
To: dev@zookeeper.apache.org
Subject: [jira] [Commented] (ZOOKEEPER-866) Adding no disk persistence option in zookeeper.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502200#comment-13502200 ]
Graham commented on ZOOKEEPER-866:
----------------------------------
We ran some benchmarks using zk-latencies.
We tried to ways to boost performance: Using a RAM disk (tmpfs) and using libeatmydata (Makes all file system sync operations no-ops).
libeatmydata benchmarks: http://pastebin.com/cNLjfhPG
Ramdisk using tmpfs benchmarks: http://pastebin.com/mfe92nXn (Note: Different box to the last one)
In summary: Synchronous calls are boosted by two orders of magnitude with either libeatmydata or ramdisk (In Standalone mode and also clustered mode). Asynchronous calls are boosted by a factor of 2 or 3.
For tests, simulations etc. a Zookeeper without snapshots or logs makes a lot of sense, but for production use, the ramdisk or eatmydata options both looks pretty good.
Another thing we found works well is to have a battery backed raid array; writes to go to raid cache and will sync to disk eventually.
> Adding no disk persistence option in zookeeper.
> -----------------------------------------------
>
> Key: ZOOKEEPER-866
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-866
> Project: ZooKeeper
> Issue Type: New Feature
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-nodisk.patch
>
>
> Its been seen that some folks would like to use zookeeper for very fine grained locking. Also, in there use case they are fine with loosing all old zookeeper state if they reboot zookeeper or zookeeper goes down. The use case is more of a runtime locking wherein forgetting the state of locks is acceptable in case of a zookeeper reboot. Not logging to disk allows high throughput on and low latency on the writes to zookeeper. This would be a configuration option to set (ofcourse the default would be logging to disk).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira