Use a RAM database to store zookeeper data

["PADIOU Pierre-Marie (MORPHO)" <pi...@morpho.com>]

Hello,

Suppose I've got only < live > data in zookeeper, but I want high write throughput. Is there any issue with using /dev/shm to store zookeeper data? (Assuming datalogs are properly cleaned up)

Has anyone ever done that?

Thanks,

Pierre-Marie
#
" Ce courriel et les documents qui lui sont joints peuvent contenir des informations confidentielles ou ayant un caractère privé. S'ils ne vous sont pas destinés, nous vous signalons qu'il est strictement interdit de les divulguer, de les reproduire ou d'en utiliser de quelque manière que ce soit le contenu. Si ce message vous a été transmis par erreur, merci d'en informer l'expéditeur et de supprimer immédiatement de votre système informatique ce courriel ainsi que tous les documents qui y sont attachés."
******
" This e-mail and any attached documents may contain confidential or proprietary information. If you are not the intended recipient, you are notified that any dissemination, copying of this e-mail and any attachments thereto or use of their contents by any means whatsoever is strictly prohibited. If you have received this e-mail in error, please advise the sender immediately and delete this e-mail and all attached documents from your computer system."
#

Re: Use a RAM database to store zookeeper data

[Kapil Thangavleu <ka...@gmail.com>]

indeed my primary reason for doing so was preserving ssd writes (probably 
silly).. actual write throughput gains against the api are minimal as per the 
test suite total runtime comparative benefit.

cheers,

kapil

Excerpts from Ted Dunning's message of Fri Sep 09 09:04:05 -0700 2011:
> Also keep in mind that it probably won't change throughput all that much.
> 
> You should run tests before counting your chickens.
> 
> On Fri, Sep 9, 2011 at 7:37 AM, Kapil Thangavleu <ka...@gmail.com>wrote:
> 
> > Excerpts from PADIOU Pierre-Marie (MORPHO)'s message of Fri Sep 09 04:50:19
> > -0700 2011:
> > > Hello,
> > >
> > > Suppose I've got only < live > data in zookeeper, but I want high write
> > throughput. Is there any issue with using /dev/shm to store zookeeper data?
> > (Assuming datalogs are properly cleaned up)
> > >
> > > Has anyone ever done that?
> > >
> > > Thanks,
> > >
> > > Pierre-Marie
> >
> > I occasionally do something similiar for running large test suites. a tmpfs
> > mount for the data dirs, and /dev/null for log4j. Keep in mind zk is
> > already
> > keeping data in memory, so ram usage is at least 2x data set size with
> > this.
> >
> > cheers,
> >
> > Kapil
> >

Re: Use a RAM database to store zookeeper data

[Ted Dunning <te...@gmail.com>]

Also keep in mind that it probably won't change throughput all that much.

You should run tests before counting your chickens.

On Fri, Sep 9, 2011 at 7:37 AM, Kapil Thangavleu <ka...@gmail.com>wrote:

> Excerpts from PADIOU Pierre-Marie (MORPHO)'s message of Fri Sep 09 04:50:19
> -0700 2011:
> > Hello,
> >
> > Suppose I've got only < live > data in zookeeper, but I want high write
> throughput. Is there any issue with using /dev/shm to store zookeeper data?
> (Assuming datalogs are properly cleaned up)
> >
> > Has anyone ever done that?
> >
> > Thanks,
> >
> > Pierre-Marie
>
> I occasionally do something similiar for running large test suites. a tmpfs
> mount for the data dirs, and /dev/null for log4j. Keep in mind zk is
> already
> keeping data in memory, so ram usage is at least 2x data set size with
> this.
>
> cheers,
>
> Kapil
>

Re: Use a RAM database to store zookeeper data

[Kapil Thangavleu <ka...@gmail.com>]

Excerpts from PADIOU Pierre-Marie (MORPHO)'s message of Fri Sep 09 04:50:19 -0700 2011:
> Hello,
> 
> Suppose I've got only < live > data in zookeeper, but I want high write throughput. Is there any issue with using /dev/shm to store zookeeper data? (Assuming datalogs are properly cleaned up)
> 
> Has anyone ever done that?
> 
> Thanks,
> 
> Pierre-Marie

I occasionally do something similiar for running large test suites. a tmpfs 
mount for the data dirs, and /dev/null for log4j. Keep in mind zk is already 
keeping data in memory, so ram usage is at least 2x data set size with this.

cheers,

Kapil