ext3 vs ext4 vs xfs for solr....recommendations needed...

[William Pierce <ev...@hotmail.com>]

Folks:

For those of your experienced linux-solr hands, I am seeking recommendations for which file system you think would work best with solr.  We are currently running with Ubuntu 9.04 on an amazon ec2 instance.  The default file system I think is ext3.  

 I am of course seeking, of course, to ensure good performance with stability.  What I have been reading is that ext4 may be a little too "bleeding edge"...but I defer to those of you who know more about this...

Thanks,

- Bill

Re: ext3 vs ext4 vs xfs for solr....recommendations needed...

[Mark Miller <ma...@gmail.com>]

William Pierce wrote:
> Folks:
>
> For those of your experienced linux-solr hands, I am seeking recommendations for which file system you think would work best with solr.  We are currently running with Ubuntu 9.04 on an amazon ec2 instance.  The default file system I think is ext3.  
>
>  I am of course seeking, of course, to ensure good performance with stability.  What I have been reading is that ext4 may be a little too "bleeding edge"...but I defer to those of you who know more about this...
>
> Thanks,
>
> - Bill
>   
I'd prob stick to ext3 - there appear to be quite a few wins in terms of
access speed, but ext4 has some sort of issue with writes - I think it
involves fsync, which lucene/solr uses for an index commit. If you have
Lucene's autocommit turned on (off by default, and removed in Lucene
3.0), the speed on ext4 is just hammered for indexing. Its not so bad
without autocommit (fewer fsyncs, as they should only occur on Solr
commits), but it makes the upgrade less compelling certainly.

You can see the hit in this sqllite insert test - I'm guessing its the
same issue:

http://www.phoronix.com/scan.php?page=article&item=ext4_btrfs_nilfs2&num=2


-- 
- Mark

http://www.lucidimagination.com