You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Metin Akat <ak...@gmail.com> on 2010/09/16 11:40:28 UTC
Which filesystem is best for deploying couchdb and why?
It's hard to write the body of this e-mail, because I don't know what
else to add to the question, so I'll leave it like this :)
Re: Which filesystem is best for deploying couchdb and why?
Posted by Wout Mertens <wo...@gmail.com>.
On Sep 19, 2010, at 9:42 , Metin Akat wrote:
> Hmm, then I start thinking that btrfs is really better than ext4,
> especially if I want to do things like copy the files in order to
> deploy (fast) another instance with already build view indexes.
Well, ext4 is considered more stable than btrfs, that has to count for something in your reasoning :-)
How would btrfs help with fast copies? You would still need to transfer the files to another system, no?
> And would the couchdb files compress well?
Yup...
Wout.
>
> On Sat, Sep 18, 2010 at 12:23 PM, Wout Mertens <wo...@gmail.com> wrote:
>> Everything except the 4KB blocksize I would say. While ZFS and Btrfs are very different on-disk, they are both Copy-On-Write filesystems with extents, compression and cheap snapshots. I don't know how the 4KB blocksize settings translates onto Btrfs.
>>
>> They are both really suited for the append-only workload CouchDB presents.
>>
>> Wout.
>>
>> On Sep 18, 2010, at 9:49 , Metin Akat wrote:
>>
>>> What part of this blog post is relevant to btrfs?
>>>
>>> On Fri, Sep 17, 2010 at 10:22 PM, Chris Anderson <jc...@apache.org> wrote:
>>>> On Thu, Sep 16, 2010 at 6:22 PM, Tyler Gillies <ty...@pdxbrain.com> wrote:
>>>>> Wow, thanks for the thought out writeup!
>>>>>
>>>>
>>>> here's a blog post http://letsgetdugg.com/2010/06/25/couchdb-on-zfs/
>>>>
>>>>> On Thu, Sep 16, 2010 at 12:47 PM, Randall Leeds <ra...@gmail.com>wrote:
>>>>>
>>>>>> Disclaimer: I'm no file systems expert.
>>>>>>
>>>>>> I recommend something with extents otherwise you might take a big
>>>>>> performance hit while couch deletes old db files after compaction.
>>>>>> Compression sounds cool as long as you can do it really fast (are
>>>>>> there setups where this happens in hardware?).
>>>>>>
>>>>>> reiserfs:
>>>>>> According to wikipedia it "still uses the big kernel lock (BKL) — a
>>>>>> global kernel-wide lock" which makes performance on multiple cores
>>>>>> suffer.
>>>>>> It's big benefit, as I always understood it, is being able to pack
>>>>>> smile files together into single blocks. You will likely not have lots
>>>>>> of small files with Couch :-P
>>>>>>
>>>>>> xfs:
>>>>>> Delayed allocation might be a big performance win with a Couch. Since
>>>>>> outstanding writes are committed together in chunks and then fsync'd
>>>>>> all together I bet this feature would do good things for Couch
>>>>>> performance.
>>>>>>
>>>>>> ext(3|4)
>>>>>> I'd recommend ext4 over ext3. Delayed allocation like xfs as well as
>>>>>> the multiblock allocator should make it much better than ext3. You
>>>>>> also get extents.
>>>>>>
>>>>>> btrfs/zfs:
>>>>>> Some of the features of each sound interesting, but nothing that
>>>>>> stands out to me as "great for CouchDB". Snapshots and backups are
>>>>>> cool, but Couch is doing this for you already in a sense due to the
>>>>>> way the btree is appended: CouchDB documents are, in a sense,
>>>>>> copy-on-write. Checksumming is cool if you think it's important for
>>>>>> your data integrity. If you want snapshots for backup you can always
>>>>>> use CouchDB replication.
>>>>>>
>>>>>> If you run any tests I'd be very, very interested in seeing your results.
>>>>>>
>>>>>> -Randall
>>>>>>
>>>>>> On Thu, Sep 16, 2010 at 03:11, Metin Akat <ak...@gmail.com> wrote:
>>>>>>> I'm sure almost everybody out there is using ext4/3 (including me),
>>>>>>> but what about filesystems like btrfs, zfs, reiserfs, xfs. Some of
>>>>>>> them have very appealing feature-sets (like compression for example,
>>>>>>> and we all know how greedy is couchdb for disk space).
>>>>>>> And I know that for example btrfs is not yet "recommended for
>>>>>>> production". But its time is coming. From what I see, Ubuntu 10.10
>>>>>>> works flawlessly on btrfs.
>>>>>>> So I'd be happy if we have some discussion on the topic, instead of
>>>>>>> "everybody uses ext4, just use it" kind of stuff :).
>>>>>>> Couchdb was "alpha software" for years, and we all used it in
>>>>>>> production, so we are not afraid of alpha/beta software, as long as
>>>>>>> it's good :)
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> http://www.readwriteweb.com/about#tyler
>>>>>
>>>>> Ask me anything <http://tumble.pdxbrain.com/ask>!
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Chris Anderson
>>>> http://jchrisa.net
>>>> http://couch.io
>>>>
>>
>>
Re: Which filesystem is best for deploying couchdb and why?
Posted by Mike Leddy <mi...@loop.com.br>.
With zfs I get very good compression ratios:
canoa:~# zfs get all zfs/couchdb
NAME PROPERTY VALUE SOURCE
zfs/couchdb type filesystem -
zfs/couchdb creation Tue Aug 3 16:52 2010 -
zfs/couchdb used 53.1G -
zfs/couchdb available 36.0G -
zfs/couchdb referenced 53.1G -
zfs/couchdb compressratio 5.87x -
zfs/couchdb mounted yes -
zfs/couchdb quota none default
zfs/couchdb reservation none default
zfs/couchdb recordsize 128K default
zfs/couchdb mountpoint /zfs/couchdb default
zfs/couchdb sharenfs off default
zfs/couchdb checksum on default
zfs/couchdb compression gzip inherited from zfs
zfs/couchdb atime off inherited from zfs
zfs/couchdb devices on default
......
That's 5.87x using gzip (level 6 which is the default). I wouldn't be able
to use my rather small (128Gb) solid state disks without it :-)
Mike
On Sun, 2010-09-19 at 10:42 +0300, Metin Akat wrote:
> Hmm, then I start thinking that btrfs is really better than ext4,
> especially if I want to do things like copy the files in order to
> deploy (fast) another instance with already build view indexes.
>
> And would the couchdb files compress well?
>
> On Sat, Sep 18, 2010 at 12:23 PM, Wout Mertens <wo...@gmail.com> wrote:
> > Everything except the 4KB blocksize I would say. While ZFS and Btrfs are very different on-disk, they are both Copy-On-Write filesystems with extents, compression and cheap snapshots. I don't know how the 4KB blocksize settings translates onto Btrfs.
> >
> > They are both really suited for the append-only workload CouchDB presents.
> >
> > Wout.
> >
> > On Sep 18, 2010, at 9:49 , Metin Akat wrote:
> >
> >> What part of this blog post is relevant to btrfs?
> >>
> >> On Fri, Sep 17, 2010 at 10:22 PM, Chris Anderson <jc...@apache.org> wrote:
> >>> On Thu, Sep 16, 2010 at 6:22 PM, Tyler Gillies <ty...@pdxbrain.com> wrote:
> >>>> Wow, thanks for the thought out writeup!
> >>>>
> >>>
> >>> here's a blog post http://letsgetdugg.com/2010/06/25/couchdb-on-zfs/
> >>>
> >>>> On Thu, Sep 16, 2010 at 12:47 PM, Randall Leeds <ra...@gmail.com>wrote:
> >>>>
> >>>>> Disclaimer: I'm no file systems expert.
> >>>>>
> >>>>> I recommend something with extents otherwise you might take a big
> >>>>> performance hit while couch deletes old db files after compaction.
> >>>>> Compression sounds cool as long as you can do it really fast (are
> >>>>> there setups where this happens in hardware?).
> >>>>>
> >>>>> reiserfs:
> >>>>> According to wikipedia it "still uses the big kernel lock (BKL) — a
> >>>>> global kernel-wide lock" which makes performance on multiple cores
> >>>>> suffer.
> >>>>> It's big benefit, as I always understood it, is being able to pack
> >>>>> smile files together into single blocks. You will likely not have lots
> >>>>> of small files with Couch :-P
> >>>>>
> >>>>> xfs:
> >>>>> Delayed allocation might be a big performance win with a Couch. Since
> >>>>> outstanding writes are committed together in chunks and then fsync'd
> >>>>> all together I bet this feature would do good things for Couch
> >>>>> performance.
> >>>>>
> >>>>> ext(3|4)
> >>>>> I'd recommend ext4 over ext3. Delayed allocation like xfs as well as
> >>>>> the multiblock allocator should make it much better than ext3. You
> >>>>> also get extents.
> >>>>>
> >>>>> btrfs/zfs:
> >>>>> Some of the features of each sound interesting, but nothing that
> >>>>> stands out to me as "great for CouchDB". Snapshots and backups are
> >>>>> cool, but Couch is doing this for you already in a sense due to the
> >>>>> way the btree is appended: CouchDB documents are, in a sense,
> >>>>> copy-on-write. Checksumming is cool if you think it's important for
> >>>>> your data integrity. If you want snapshots for backup you can always
> >>>>> use CouchDB replication.
> >>>>>
> >>>>> If you run any tests I'd be very, very interested in seeing your results.
> >>>>>
> >>>>> -Randall
> >>>>>
> >>>>> On Thu, Sep 16, 2010 at 03:11, Metin Akat <ak...@gmail.com> wrote:
> >>>>>> I'm sure almost everybody out there is using ext4/3 (including me),
> >>>>>> but what about filesystems like btrfs, zfs, reiserfs, xfs. Some of
> >>>>>> them have very appealing feature-sets (like compression for example,
> >>>>>> and we all know how greedy is couchdb for disk space).
> >>>>>> And I know that for example btrfs is not yet "recommended for
> >>>>>> production". But its time is coming. From what I see, Ubuntu 10.10
> >>>>>> works flawlessly on btrfs.
> >>>>>> So I'd be happy if we have some discussion on the topic, instead of
> >>>>>> "everybody uses ext4, just use it" kind of stuff :).
> >>>>>> Couchdb was "alpha software" for years, and we all used it in
> >>>>>> production, so we are not afraid of alpha/beta software, as long as
> >>>>>> it's good :)
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> http://www.readwriteweb.com/about#tyler
> >>>>
> >>>> Ask me anything <http://tumble.pdxbrain.com/ask>!
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Chris Anderson
> >>> http://jchrisa.net
> >>> http://couch.io
> >>>
> >
> >
>
Re: Which filesystem is best for deploying couchdb and why?
Posted by Metin Akat <ak...@gmail.com>.
Hmm, then I start thinking that btrfs is really better than ext4,
especially if I want to do things like copy the files in order to
deploy (fast) another instance with already build view indexes.
And would the couchdb files compress well?
On Sat, Sep 18, 2010 at 12:23 PM, Wout Mertens <wo...@gmail.com> wrote:
> Everything except the 4KB blocksize I would say. While ZFS and Btrfs are very different on-disk, they are both Copy-On-Write filesystems with extents, compression and cheap snapshots. I don't know how the 4KB blocksize settings translates onto Btrfs.
>
> They are both really suited for the append-only workload CouchDB presents.
>
> Wout.
>
> On Sep 18, 2010, at 9:49 , Metin Akat wrote:
>
>> What part of this blog post is relevant to btrfs?
>>
>> On Fri, Sep 17, 2010 at 10:22 PM, Chris Anderson <jc...@apache.org> wrote:
>>> On Thu, Sep 16, 2010 at 6:22 PM, Tyler Gillies <ty...@pdxbrain.com> wrote:
>>>> Wow, thanks for the thought out writeup!
>>>>
>>>
>>> here's a blog post http://letsgetdugg.com/2010/06/25/couchdb-on-zfs/
>>>
>>>> On Thu, Sep 16, 2010 at 12:47 PM, Randall Leeds <ra...@gmail.com>wrote:
>>>>
>>>>> Disclaimer: I'm no file systems expert.
>>>>>
>>>>> I recommend something with extents otherwise you might take a big
>>>>> performance hit while couch deletes old db files after compaction.
>>>>> Compression sounds cool as long as you can do it really fast (are
>>>>> there setups where this happens in hardware?).
>>>>>
>>>>> reiserfs:
>>>>> According to wikipedia it "still uses the big kernel lock (BKL) — a
>>>>> global kernel-wide lock" which makes performance on multiple cores
>>>>> suffer.
>>>>> It's big benefit, as I always understood it, is being able to pack
>>>>> smile files together into single blocks. You will likely not have lots
>>>>> of small files with Couch :-P
>>>>>
>>>>> xfs:
>>>>> Delayed allocation might be a big performance win with a Couch. Since
>>>>> outstanding writes are committed together in chunks and then fsync'd
>>>>> all together I bet this feature would do good things for Couch
>>>>> performance.
>>>>>
>>>>> ext(3|4)
>>>>> I'd recommend ext4 over ext3. Delayed allocation like xfs as well as
>>>>> the multiblock allocator should make it much better than ext3. You
>>>>> also get extents.
>>>>>
>>>>> btrfs/zfs:
>>>>> Some of the features of each sound interesting, but nothing that
>>>>> stands out to me as "great for CouchDB". Snapshots and backups are
>>>>> cool, but Couch is doing this for you already in a sense due to the
>>>>> way the btree is appended: CouchDB documents are, in a sense,
>>>>> copy-on-write. Checksumming is cool if you think it's important for
>>>>> your data integrity. If you want snapshots for backup you can always
>>>>> use CouchDB replication.
>>>>>
>>>>> If you run any tests I'd be very, very interested in seeing your results.
>>>>>
>>>>> -Randall
>>>>>
>>>>> On Thu, Sep 16, 2010 at 03:11, Metin Akat <ak...@gmail.com> wrote:
>>>>>> I'm sure almost everybody out there is using ext4/3 (including me),
>>>>>> but what about filesystems like btrfs, zfs, reiserfs, xfs. Some of
>>>>>> them have very appealing feature-sets (like compression for example,
>>>>>> and we all know how greedy is couchdb for disk space).
>>>>>> And I know that for example btrfs is not yet "recommended for
>>>>>> production". But its time is coming. From what I see, Ubuntu 10.10
>>>>>> works flawlessly on btrfs.
>>>>>> So I'd be happy if we have some discussion on the topic, instead of
>>>>>> "everybody uses ext4, just use it" kind of stuff :).
>>>>>> Couchdb was "alpha software" for years, and we all used it in
>>>>>> production, so we are not afraid of alpha/beta software, as long as
>>>>>> it's good :)
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> http://www.readwriteweb.com/about#tyler
>>>>
>>>> Ask me anything <http://tumble.pdxbrain.com/ask>!
>>>>
>>>
>>>
>>>
>>> --
>>> Chris Anderson
>>> http://jchrisa.net
>>> http://couch.io
>>>
>
>
Re: Which filesystem is best for deploying couchdb and why?
Posted by Wout Mertens <wo...@gmail.com>.
Everything except the 4KB blocksize I would say. While ZFS and Btrfs are very different on-disk, they are both Copy-On-Write filesystems with extents, compression and cheap snapshots. I don't know how the 4KB blocksize settings translates onto Btrfs.
They are both really suited for the append-only workload CouchDB presents.
Wout.
On Sep 18, 2010, at 9:49 , Metin Akat wrote:
> What part of this blog post is relevant to btrfs?
>
> On Fri, Sep 17, 2010 at 10:22 PM, Chris Anderson <jc...@apache.org> wrote:
>> On Thu, Sep 16, 2010 at 6:22 PM, Tyler Gillies <ty...@pdxbrain.com> wrote:
>>> Wow, thanks for the thought out writeup!
>>>
>>
>> here's a blog post http://letsgetdugg.com/2010/06/25/couchdb-on-zfs/
>>
>>> On Thu, Sep 16, 2010 at 12:47 PM, Randall Leeds <ra...@gmail.com>wrote:
>>>
>>>> Disclaimer: I'm no file systems expert.
>>>>
>>>> I recommend something with extents otherwise you might take a big
>>>> performance hit while couch deletes old db files after compaction.
>>>> Compression sounds cool as long as you can do it really fast (are
>>>> there setups where this happens in hardware?).
>>>>
>>>> reiserfs:
>>>> According to wikipedia it "still uses the big kernel lock (BKL) — a
>>>> global kernel-wide lock" which makes performance on multiple cores
>>>> suffer.
>>>> It's big benefit, as I always understood it, is being able to pack
>>>> smile files together into single blocks. You will likely not have lots
>>>> of small files with Couch :-P
>>>>
>>>> xfs:
>>>> Delayed allocation might be a big performance win with a Couch. Since
>>>> outstanding writes are committed together in chunks and then fsync'd
>>>> all together I bet this feature would do good things for Couch
>>>> performance.
>>>>
>>>> ext(3|4)
>>>> I'd recommend ext4 over ext3. Delayed allocation like xfs as well as
>>>> the multiblock allocator should make it much better than ext3. You
>>>> also get extents.
>>>>
>>>> btrfs/zfs:
>>>> Some of the features of each sound interesting, but nothing that
>>>> stands out to me as "great for CouchDB". Snapshots and backups are
>>>> cool, but Couch is doing this for you already in a sense due to the
>>>> way the btree is appended: CouchDB documents are, in a sense,
>>>> copy-on-write. Checksumming is cool if you think it's important for
>>>> your data integrity. If you want snapshots for backup you can always
>>>> use CouchDB replication.
>>>>
>>>> If you run any tests I'd be very, very interested in seeing your results.
>>>>
>>>> -Randall
>>>>
>>>> On Thu, Sep 16, 2010 at 03:11, Metin Akat <ak...@gmail.com> wrote:
>>>>> I'm sure almost everybody out there is using ext4/3 (including me),
>>>>> but what about filesystems like btrfs, zfs, reiserfs, xfs. Some of
>>>>> them have very appealing feature-sets (like compression for example,
>>>>> and we all know how greedy is couchdb for disk space).
>>>>> And I know that for example btrfs is not yet "recommended for
>>>>> production". But its time is coming. From what I see, Ubuntu 10.10
>>>>> works flawlessly on btrfs.
>>>>> So I'd be happy if we have some discussion on the topic, instead of
>>>>> "everybody uses ext4, just use it" kind of stuff :).
>>>>> Couchdb was "alpha software" for years, and we all used it in
>>>>> production, so we are not afraid of alpha/beta software, as long as
>>>>> it's good :)
>>>>
>>>
>>>
>>>
>>> --
>>> http://www.readwriteweb.com/about#tyler
>>>
>>> Ask me anything <http://tumble.pdxbrain.com/ask>!
>>>
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchrisa.net
>> http://couch.io
>>
Re: Which filesystem is best for deploying couchdb and why?
Posted by Metin Akat <ak...@gmail.com>.
What part of this blog post is relevant to btrfs?
On Fri, Sep 17, 2010 at 10:22 PM, Chris Anderson <jc...@apache.org> wrote:
> On Thu, Sep 16, 2010 at 6:22 PM, Tyler Gillies <ty...@pdxbrain.com> wrote:
>> Wow, thanks for the thought out writeup!
>>
>
> here's a blog post http://letsgetdugg.com/2010/06/25/couchdb-on-zfs/
>
>> On Thu, Sep 16, 2010 at 12:47 PM, Randall Leeds <ra...@gmail.com>wrote:
>>
>>> Disclaimer: I'm no file systems expert.
>>>
>>> I recommend something with extents otherwise you might take a big
>>> performance hit while couch deletes old db files after compaction.
>>> Compression sounds cool as long as you can do it really fast (are
>>> there setups where this happens in hardware?).
>>>
>>> reiserfs:
>>> According to wikipedia it "still uses the big kernel lock (BKL) — a
>>> global kernel-wide lock" which makes performance on multiple cores
>>> suffer.
>>> It's big benefit, as I always understood it, is being able to pack
>>> smile files together into single blocks. You will likely not have lots
>>> of small files with Couch :-P
>>>
>>> xfs:
>>> Delayed allocation might be a big performance win with a Couch. Since
>>> outstanding writes are committed together in chunks and then fsync'd
>>> all together I bet this feature would do good things for Couch
>>> performance.
>>>
>>> ext(3|4)
>>> I'd recommend ext4 over ext3. Delayed allocation like xfs as well as
>>> the multiblock allocator should make it much better than ext3. You
>>> also get extents.
>>>
>>> btrfs/zfs:
>>> Some of the features of each sound interesting, but nothing that
>>> stands out to me as "great for CouchDB". Snapshots and backups are
>>> cool, but Couch is doing this for you already in a sense due to the
>>> way the btree is appended: CouchDB documents are, in a sense,
>>> copy-on-write. Checksumming is cool if you think it's important for
>>> your data integrity. If you want snapshots for backup you can always
>>> use CouchDB replication.
>>>
>>> If you run any tests I'd be very, very interested in seeing your results.
>>>
>>> -Randall
>>>
>>> On Thu, Sep 16, 2010 at 03:11, Metin Akat <ak...@gmail.com> wrote:
>>> > I'm sure almost everybody out there is using ext4/3 (including me),
>>> > but what about filesystems like btrfs, zfs, reiserfs, xfs. Some of
>>> > them have very appealing feature-sets (like compression for example,
>>> > and we all know how greedy is couchdb for disk space).
>>> > And I know that for example btrfs is not yet "recommended for
>>> > production". But its time is coming. From what I see, Ubuntu 10.10
>>> > works flawlessly on btrfs.
>>> > So I'd be happy if we have some discussion on the topic, instead of
>>> > "everybody uses ext4, just use it" kind of stuff :).
>>> > Couchdb was "alpha software" for years, and we all used it in
>>> > production, so we are not afraid of alpha/beta software, as long as
>>> > it's good :)
>>>
>>
>>
>>
>> --
>> http://www.readwriteweb.com/about#tyler
>>
>> Ask me anything <http://tumble.pdxbrain.com/ask>!
>>
>
>
>
> --
> Chris Anderson
> http://jchrisa.net
> http://couch.io
>
Re: Which filesystem is best for deploying couchdb and why?
Posted by Chris Anderson <jc...@apache.org>.
On Thu, Sep 16, 2010 at 6:22 PM, Tyler Gillies <ty...@pdxbrain.com> wrote:
> Wow, thanks for the thought out writeup!
>
here's a blog post http://letsgetdugg.com/2010/06/25/couchdb-on-zfs/
> On Thu, Sep 16, 2010 at 12:47 PM, Randall Leeds <ra...@gmail.com>wrote:
>
>> Disclaimer: I'm no file systems expert.
>>
>> I recommend something with extents otherwise you might take a big
>> performance hit while couch deletes old db files after compaction.
>> Compression sounds cool as long as you can do it really fast (are
>> there setups where this happens in hardware?).
>>
>> reiserfs:
>> According to wikipedia it "still uses the big kernel lock (BKL) — a
>> global kernel-wide lock" which makes performance on multiple cores
>> suffer.
>> It's big benefit, as I always understood it, is being able to pack
>> smile files together into single blocks. You will likely not have lots
>> of small files with Couch :-P
>>
>> xfs:
>> Delayed allocation might be a big performance win with a Couch. Since
>> outstanding writes are committed together in chunks and then fsync'd
>> all together I bet this feature would do good things for Couch
>> performance.
>>
>> ext(3|4)
>> I'd recommend ext4 over ext3. Delayed allocation like xfs as well as
>> the multiblock allocator should make it much better than ext3. You
>> also get extents.
>>
>> btrfs/zfs:
>> Some of the features of each sound interesting, but nothing that
>> stands out to me as "great for CouchDB". Snapshots and backups are
>> cool, but Couch is doing this for you already in a sense due to the
>> way the btree is appended: CouchDB documents are, in a sense,
>> copy-on-write. Checksumming is cool if you think it's important for
>> your data integrity. If you want snapshots for backup you can always
>> use CouchDB replication.
>>
>> If you run any tests I'd be very, very interested in seeing your results.
>>
>> -Randall
>>
>> On Thu, Sep 16, 2010 at 03:11, Metin Akat <ak...@gmail.com> wrote:
>> > I'm sure almost everybody out there is using ext4/3 (including me),
>> > but what about filesystems like btrfs, zfs, reiserfs, xfs. Some of
>> > them have very appealing feature-sets (like compression for example,
>> > and we all know how greedy is couchdb for disk space).
>> > And I know that for example btrfs is not yet "recommended for
>> > production". But its time is coming. From what I see, Ubuntu 10.10
>> > works flawlessly on btrfs.
>> > So I'd be happy if we have some discussion on the topic, instead of
>> > "everybody uses ext4, just use it" kind of stuff :).
>> > Couchdb was "alpha software" for years, and we all used it in
>> > production, so we are not afraid of alpha/beta software, as long as
>> > it's good :)
>>
>
>
>
> --
> http://www.readwriteweb.com/about#tyler
>
> Ask me anything <http://tumble.pdxbrain.com/ask>!
>
--
Chris Anderson
http://jchrisa.net
http://couch.io
Re: Which filesystem is best for deploying couchdb and why?
Posted by Tyler Gillies <ty...@pdxbrain.com>.
Wow, thanks for the thought out writeup!
On Thu, Sep 16, 2010 at 12:47 PM, Randall Leeds <ra...@gmail.com>wrote:
> Disclaimer: I'm no file systems expert.
>
> I recommend something with extents otherwise you might take a big
> performance hit while couch deletes old db files after compaction.
> Compression sounds cool as long as you can do it really fast (are
> there setups where this happens in hardware?).
>
> reiserfs:
> According to wikipedia it "still uses the big kernel lock (BKL) — a
> global kernel-wide lock" which makes performance on multiple cores
> suffer.
> It's big benefit, as I always understood it, is being able to pack
> smile files together into single blocks. You will likely not have lots
> of small files with Couch :-P
>
> xfs:
> Delayed allocation might be a big performance win with a Couch. Since
> outstanding writes are committed together in chunks and then fsync'd
> all together I bet this feature would do good things for Couch
> performance.
>
> ext(3|4)
> I'd recommend ext4 over ext3. Delayed allocation like xfs as well as
> the multiblock allocator should make it much better than ext3. You
> also get extents.
>
> btrfs/zfs:
> Some of the features of each sound interesting, but nothing that
> stands out to me as "great for CouchDB". Snapshots and backups are
> cool, but Couch is doing this for you already in a sense due to the
> way the btree is appended: CouchDB documents are, in a sense,
> copy-on-write. Checksumming is cool if you think it's important for
> your data integrity. If you want snapshots for backup you can always
> use CouchDB replication.
>
> If you run any tests I'd be very, very interested in seeing your results.
>
> -Randall
>
> On Thu, Sep 16, 2010 at 03:11, Metin Akat <ak...@gmail.com> wrote:
> > I'm sure almost everybody out there is using ext4/3 (including me),
> > but what about filesystems like btrfs, zfs, reiserfs, xfs. Some of
> > them have very appealing feature-sets (like compression for example,
> > and we all know how greedy is couchdb for disk space).
> > And I know that for example btrfs is not yet "recommended for
> > production". But its time is coming. From what I see, Ubuntu 10.10
> > works flawlessly on btrfs.
> > So I'd be happy if we have some discussion on the topic, instead of
> > "everybody uses ext4, just use it" kind of stuff :).
> > Couchdb was "alpha software" for years, and we all used it in
> > production, so we are not afraid of alpha/beta software, as long as
> > it's good :)
>
--
http://www.readwriteweb.com/about#tyler
Ask me anything <http://tumble.pdxbrain.com/ask>!
Re: Which filesystem is best for deploying couchdb and why?
Posted by Randall Leeds <ra...@gmail.com>.
Disclaimer: I'm no file systems expert.
I recommend something with extents otherwise you might take a big
performance hit while couch deletes old db files after compaction.
Compression sounds cool as long as you can do it really fast (are
there setups where this happens in hardware?).
reiserfs:
According to wikipedia it "still uses the big kernel lock (BKL) — a
global kernel-wide lock" which makes performance on multiple cores
suffer.
It's big benefit, as I always understood it, is being able to pack
smile files together into single blocks. You will likely not have lots
of small files with Couch :-P
xfs:
Delayed allocation might be a big performance win with a Couch. Since
outstanding writes are committed together in chunks and then fsync'd
all together I bet this feature would do good things for Couch
performance.
ext(3|4)
I'd recommend ext4 over ext3. Delayed allocation like xfs as well as
the multiblock allocator should make it much better than ext3. You
also get extents.
btrfs/zfs:
Some of the features of each sound interesting, but nothing that
stands out to me as "great for CouchDB". Snapshots and backups are
cool, but Couch is doing this for you already in a sense due to the
way the btree is appended: CouchDB documents are, in a sense,
copy-on-write. Checksumming is cool if you think it's important for
your data integrity. If you want snapshots for backup you can always
use CouchDB replication.
If you run any tests I'd be very, very interested in seeing your results.
-Randall
On Thu, Sep 16, 2010 at 03:11, Metin Akat <ak...@gmail.com> wrote:
> I'm sure almost everybody out there is using ext4/3 (including me),
> but what about filesystems like btrfs, zfs, reiserfs, xfs. Some of
> them have very appealing feature-sets (like compression for example,
> and we all know how greedy is couchdb for disk space).
> And I know that for example btrfs is not yet "recommended for
> production". But its time is coming. From what I see, Ubuntu 10.10
> works flawlessly on btrfs.
> So I'd be happy if we have some discussion on the topic, instead of
> "everybody uses ext4, just use it" kind of stuff :).
> Couchdb was "alpha software" for years, and we all used it in
> production, so we are not afraid of alpha/beta software, as long as
> it's good :)
Re: Which filesystem is best for deploying couchdb and why?
Posted by Metin Akat <ak...@gmail.com>.
I'm sure almost everybody out there is using ext4/3 (including me),
but what about filesystems like btrfs, zfs, reiserfs, xfs. Some of
them have very appealing feature-sets (like compression for example,
and we all know how greedy is couchdb for disk space).
And I know that for example btrfs is not yet "recommended for
production". But its time is coming. From what I see, Ubuntu 10.10
works flawlessly on btrfs.
So I'd be happy if we have some discussion on the topic, instead of
"everybody uses ext4, just use it" kind of stuff :).
Couchdb was "alpha software" for years, and we all used it in
production, so we are not afraid of alpha/beta software, as long as
it's good :)
On Thu, Sep 16, 2010 at 12:54 PM, Tyler Gillies <ty...@pdxbrain.com> wrote:
> ext4
>
> On Thu, Sep 16, 2010 at 2:40 AM, Metin Akat <ak...@gmail.com> wrote:
>
>> It's hard to write the body of this e-mail, because I don't know what
>> else to add to the question, so I'll leave it like this :)
>>
>
>
>
> --
> http://www.readwriteweb.com/about#tyler
>
> Ask me anything <http://tumble.pdxbrain.com/ask>!
>
Re: Which filesystem is best for deploying couchdb and why?
Posted by Tyler Gillies <ty...@pdxbrain.com>.
ext4
On Thu, Sep 16, 2010 at 2:40 AM, Metin Akat <ak...@gmail.com> wrote:
> It's hard to write the body of this e-mail, because I don't know what
> else to add to the question, so I'll leave it like this :)
>
--
http://www.readwriteweb.com/about#tyler
Ask me anything <http://tumble.pdxbrain.com/ask>!