You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Dragon Fly <dr...@hotmail.com> on 2012/07/18 17:50:52 UTC
RAM or SSD...
Hi,
If I want to improve performance, which of the following is better and why?
1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
2. Put the index on a solid state drive.
By the way, my index is about 30 GB. Thank you.
Re: RAM or SSD...
Posted by Dawid Weiss <da...@gmail.com>.
> Rum is an essential ingredient in all software systems :-)
You probably meant "social systems".
D.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: RAM or SSD...
Posted by Simon Willnauer <si...@gmail.com>.
On Wed, Jul 18, 2012 at 9:05 PM, Tim Eck <ti...@gmail.com> wrote:
> Rum is an essential ingredient in all software systems :-)
Absolutely! :)
simon
>
> -----Original Message-----
> From: Simon Willnauer [mailto:simon.willnauer@gmail.com]
> Sent: Wednesday, July 18, 2012 11:49 AM
> To: java-user@lucene.apache.org
> Subject: Re: RAM or SSD...
>
> 1. use mmap directory
> 2. buy rum
> 3. get an SSD
>
> simon :)
>
> On Wed, Jul 18, 2012 at 8:36 PM, Vitaly Funstein <vf...@gmail.com> wrote:
>> You do not want to store 30 G of data in the JVM heap, no matter what
>> library does this.
>>
>> On Wed, Jul 18, 2012 at 10:44 AM, Paul Jakubik <pa...@purediscovery.com> wrote:
>>> If only 30GB, go with RAM and MMAPDirectory (as long as you have the budget
>>> for that hardware).
>>>
>>> My understanding is that RAMDirectory is intended for unit tests, not for
>>> production indexes.
>>>
>>> On Wed, Jul 18, 2012 at 10:50 AM, Dragon Fly <dr...@hotmail.com>wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> If I want to improve performance, which of the following is better and why?
>>>>
>>>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
>>>> 2. Put the index on a solid state drive.
>>>>
>>>> By the way, my index is about 30 GB. Thank you.
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: RAM or SSD...
Posted by Tim Eck <ti...@gmail.com>.
Rum is an essential ingredient in all software systems :-)
-----Original Message-----
From: Simon Willnauer [mailto:simon.willnauer@gmail.com]
Sent: Wednesday, July 18, 2012 11:49 AM
To: java-user@lucene.apache.org
Subject: Re: RAM or SSD...
1. use mmap directory
2. buy rum
3. get an SSD
simon :)
On Wed, Jul 18, 2012 at 8:36 PM, Vitaly Funstein <vf...@gmail.com> wrote:
> You do not want to store 30 G of data in the JVM heap, no matter what
> library does this.
>
> On Wed, Jul 18, 2012 at 10:44 AM, Paul Jakubik <pa...@purediscovery.com> wrote:
>> If only 30GB, go with RAM and MMAPDirectory (as long as you have the budget
>> for that hardware).
>>
>> My understanding is that RAMDirectory is intended for unit tests, not for
>> production indexes.
>>
>> On Wed, Jul 18, 2012 at 10:50 AM, Dragon Fly <dr...@hotmail.com>wrote:
>>
>>>
>>> Hi,
>>>
>>> If I want to improve performance, which of the following is better and why?
>>>
>>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
>>> 2. Put the index on a solid state drive.
>>>
>>> By the way, my index is about 30 GB. Thank you.
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: RAM or SSD...
Posted by Simon Willnauer <si...@gmail.com>.
1. use mmap directory
2. buy rum
3. get an SSD
simon :)
On Wed, Jul 18, 2012 at 8:36 PM, Vitaly Funstein <vf...@gmail.com> wrote:
> You do not want to store 30 G of data in the JVM heap, no matter what
> library does this.
>
> On Wed, Jul 18, 2012 at 10:44 AM, Paul Jakubik <pa...@purediscovery.com> wrote:
>> If only 30GB, go with RAM and MMAPDirectory (as long as you have the budget
>> for that hardware).
>>
>> My understanding is that RAMDirectory is intended for unit tests, not for
>> production indexes.
>>
>> On Wed, Jul 18, 2012 at 10:50 AM, Dragon Fly <dr...@hotmail.com>wrote:
>>
>>>
>>> Hi,
>>>
>>> If I want to improve performance, which of the following is better and why?
>>>
>>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
>>> 2. Put the index on a solid state drive.
>>>
>>> By the way, my index is about 30 GB. Thank you.
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: RAM or SSD...
Posted by Vitaly Funstein <vf...@gmail.com>.
You do not want to store 30 G of data in the JVM heap, no matter what
library does this.
On Wed, Jul 18, 2012 at 10:44 AM, Paul Jakubik <pa...@purediscovery.com> wrote:
> If only 30GB, go with RAM and MMAPDirectory (as long as you have the budget
> for that hardware).
>
> My understanding is that RAMDirectory is intended for unit tests, not for
> production indexes.
>
> On Wed, Jul 18, 2012 at 10:50 AM, Dragon Fly <dr...@hotmail.com>wrote:
>
>>
>> Hi,
>>
>> If I want to improve performance, which of the following is better and why?
>>
>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
>> 2. Put the index on a solid state drive.
>>
>> By the way, my index is about 30 GB. Thank you.
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: RAM or SSD...
Posted by Paul Jakubik <pa...@purediscovery.com>.
If only 30GB, go with RAM and MMAPDirectory (as long as you have the budget
for that hardware).
My understanding is that RAMDirectory is intended for unit tests, not for
production indexes.
On Wed, Jul 18, 2012 at 10:50 AM, Dragon Fly <dr...@hotmail.com>wrote:
>
> Hi,
>
> If I want to improve performance, which of the following is better and why?
>
> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
> 2. Put the index on a solid state drive.
>
> By the way, my index is about 30 GB. Thank you.
>
Re: RAM or SSD...
Posted by Dawid Weiss <da...@gmail.com>.
> Why anyone buys computers without SSD's is a mystery to me. Use SSDs for
On topic and highly recommended:
http://www.youtube.com/watch?v=H7PJ1oeEyGg
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: RAM or SSD...
Posted by Denis Bazhenov <do...@gmail.com>.
Another option is to use tmpfs (memory driven file system) as a backing storage for index. This allows to minimize disc access latency. In average case this doesn't make sense, because OS will cache file system access quite effective if you have enough memory. But in worst case (if you measure response time 99 percentile or higher, for example) this method can give you more stable performance.
On Jul 20, 2012, at 10:32 PM, Dragon Fly <dr...@hotmail.com> wrote:
>
> Thank you.
>
>> From: dawid.weiss@gmail.com
>> Date: Thu, 19 Jul 2012 13:34:26 +0200
>> Subject: Re: RAM or SSD...
>> To: java-user@lucene.apache.org
>>
>> Read this:
>> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
>>
>> Dawid
>>
>> On Thu, Jul 19, 2012 at 1:32 PM, Dragon Fly <dr...@hotmail.com> wrote:
>>>
>>> The slowest part of my application is to read the search hits from disk. I was hoping that using an SSD or RAMDirectory/MMapDirectory would speed that up. I read the JavaDoc for MMapDirectory but didn't really understand how that differs from RAMDirectory. Could someone please explain?
>>>
>>>> Date: Wed, 18 Jul 2012 23:46:51 -0700
>>>> Subject: Re: RAM or SSD...
>>>> From: vfunstein@gmail.com
>>>> To: java-user@lucene.apache.org
>>>>
>>>> I was referring to *RAMDirectory*.
>>>>
>>>> On Wed, Jul 18, 2012 at 11:04 PM, Lance Norskog <go...@gmail.com> wrote:
>>>>>> You do not want to store 30 G of data in the JVM heap, no matter what
>>>> library does this.
>>>>> MMapDirectory does not store data in the JVM heap. It lets the
>>>>> operating system manage the disk buffer space. Even if the JVM says "I
>>>>> have 30G of memory space", it really does not. It only has address
>>>>> space allocated by the OS but no memory.
>>>>>
>>>>> On Wed, Jul 18, 2012 at 10:39 PM, Toke Eskildsen <te...@statsbiblioteket.dk>
>>>> wrote:
>>>>>> On Wed, 2012-07-18 at 17:50 +0200, Dragon Fly wrote:
>>>>>>> If I want to improve performance, which of the following is better and
>>>> why?
>>>>>>>
>>>>>>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
>>>>>>
>>>>>> As others has pointed out, MMapDirectory should work better than
>>>>>> RAMDirectory. I am sure it will work fine with a relative small index
>>>>>> such as yours. However, it does not scale that well with index size.
>>>>>>
>>>>>>> 2. Put the index on a solid state drive.
>>>>>>
>>>>>> Why anyone buys computers without SSD's is a mystery to me. Use SSDs for
>>>>>> the small low-latency stuff and a secondary spinning drive for the large
>>>>>> slow stuff. Nowadays, a 30GB index (or 100GB for that matter) falls into
>>>>>> the small low-latency bucket. SSDs speeds up almost everything, saves
>>>>>> RAM and spares a lot of work hours optimizing I/O-speed.
>>>>>>
>>>>>> Regards,
>>>>>> Toke Eskildsen
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lance Norskog
>>>>> goksron@gmail.com
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
---
Denis Bazhenov <do...@gmail.com>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: RAM or SSD...
Posted by Dragon Fly <dr...@hotmail.com>.
Thank you.
> From: dawid.weiss@gmail.com
> Date: Thu, 19 Jul 2012 13:34:26 +0200
> Subject: Re: RAM or SSD...
> To: java-user@lucene.apache.org
>
> Read this:
> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
>
> Dawid
>
> On Thu, Jul 19, 2012 at 1:32 PM, Dragon Fly <dr...@hotmail.com> wrote:
> >
> > The slowest part of my application is to read the search hits from disk. I was hoping that using an SSD or RAMDirectory/MMapDirectory would speed that up. I read the JavaDoc for MMapDirectory but didn't really understand how that differs from RAMDirectory. Could someone please explain?
> >
> >> Date: Wed, 18 Jul 2012 23:46:51 -0700
> >> Subject: Re: RAM or SSD...
> >> From: vfunstein@gmail.com
> >> To: java-user@lucene.apache.org
> >>
> >> I was referring to *RAMDirectory*.
> >>
> >> On Wed, Jul 18, 2012 at 11:04 PM, Lance Norskog <go...@gmail.com> wrote:
> >> >> You do not want to store 30 G of data in the JVM heap, no matter what
> >> library does this.
> >> > MMapDirectory does not store data in the JVM heap. It lets the
> >> > operating system manage the disk buffer space. Even if the JVM says "I
> >> > have 30G of memory space", it really does not. It only has address
> >> > space allocated by the OS but no memory.
> >> >
> >> > On Wed, Jul 18, 2012 at 10:39 PM, Toke Eskildsen <te...@statsbiblioteket.dk>
> >> wrote:
> >> >> On Wed, 2012-07-18 at 17:50 +0200, Dragon Fly wrote:
> >> >>> If I want to improve performance, which of the following is better and
> >> why?
> >> >>>
> >> >>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
> >> >>
> >> >> As others has pointed out, MMapDirectory should work better than
> >> >> RAMDirectory. I am sure it will work fine with a relative small index
> >> >> such as yours. However, it does not scale that well with index size.
> >> >>
> >> >>> 2. Put the index on a solid state drive.
> >> >>
> >> >> Why anyone buys computers without SSD's is a mystery to me. Use SSDs for
> >> >> the small low-latency stuff and a secondary spinning drive for the large
> >> >> slow stuff. Nowadays, a 30GB index (or 100GB for that matter) falls into
> >> >> the small low-latency bucket. SSDs speeds up almost everything, saves
> >> >> RAM and spares a lot of work hours optimizing I/O-speed.
> >> >>
> >> >> Regards,
> >> >> Toke Eskildsen
> >> >>
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Lance Norskog
> >> > goksron@gmail.com
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
Re: RAM or SSD...
Posted by Dawid Weiss <da...@gmail.com>.
Read this:
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
Dawid
On Thu, Jul 19, 2012 at 1:32 PM, Dragon Fly <dr...@hotmail.com> wrote:
>
> The slowest part of my application is to read the search hits from disk. I was hoping that using an SSD or RAMDirectory/MMapDirectory would speed that up. I read the JavaDoc for MMapDirectory but didn't really understand how that differs from RAMDirectory. Could someone please explain?
>
>> Date: Wed, 18 Jul 2012 23:46:51 -0700
>> Subject: Re: RAM or SSD...
>> From: vfunstein@gmail.com
>> To: java-user@lucene.apache.org
>>
>> I was referring to *RAMDirectory*.
>>
>> On Wed, Jul 18, 2012 at 11:04 PM, Lance Norskog <go...@gmail.com> wrote:
>> >> You do not want to store 30 G of data in the JVM heap, no matter what
>> library does this.
>> > MMapDirectory does not store data in the JVM heap. It lets the
>> > operating system manage the disk buffer space. Even if the JVM says "I
>> > have 30G of memory space", it really does not. It only has address
>> > space allocated by the OS but no memory.
>> >
>> > On Wed, Jul 18, 2012 at 10:39 PM, Toke Eskildsen <te...@statsbiblioteket.dk>
>> wrote:
>> >> On Wed, 2012-07-18 at 17:50 +0200, Dragon Fly wrote:
>> >>> If I want to improve performance, which of the following is better and
>> why?
>> >>>
>> >>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
>> >>
>> >> As others has pointed out, MMapDirectory should work better than
>> >> RAMDirectory. I am sure it will work fine with a relative small index
>> >> such as yours. However, it does not scale that well with index size.
>> >>
>> >>> 2. Put the index on a solid state drive.
>> >>
>> >> Why anyone buys computers without SSD's is a mystery to me. Use SSDs for
>> >> the small low-latency stuff and a secondary spinning drive for the large
>> >> slow stuff. Nowadays, a 30GB index (or 100GB for that matter) falls into
>> >> the small low-latency bucket. SSDs speeds up almost everything, saves
>> >> RAM and spares a lot of work hours optimizing I/O-speed.
>> >>
>> >> Regards,
>> >> Toke Eskildsen
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>
>> >
>> >
>> >
>> > --
>> > Lance Norskog
>> > goksron@gmail.com
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: RAM or SSD...
Posted by Dragon Fly <dr...@hotmail.com>.
The slowest part of my application is to read the search hits from disk. I was hoping that using an SSD or RAMDirectory/MMapDirectory would speed that up. I read the JavaDoc for MMapDirectory but didn't really understand how that differs from RAMDirectory. Could someone please explain?
> Date: Wed, 18 Jul 2012 23:46:51 -0700
> Subject: Re: RAM or SSD...
> From: vfunstein@gmail.com
> To: java-user@lucene.apache.org
>
> I was referring to *RAMDirectory*.
>
> On Wed, Jul 18, 2012 at 11:04 PM, Lance Norskog <go...@gmail.com> wrote:
> >> You do not want to store 30 G of data in the JVM heap, no matter what
> library does this.
> > MMapDirectory does not store data in the JVM heap. It lets the
> > operating system manage the disk buffer space. Even if the JVM says "I
> > have 30G of memory space", it really does not. It only has address
> > space allocated by the OS but no memory.
> >
> > On Wed, Jul 18, 2012 at 10:39 PM, Toke Eskildsen <te...@statsbiblioteket.dk>
> wrote:
> >> On Wed, 2012-07-18 at 17:50 +0200, Dragon Fly wrote:
> >>> If I want to improve performance, which of the following is better and
> why?
> >>>
> >>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
> >>
> >> As others has pointed out, MMapDirectory should work better than
> >> RAMDirectory. I am sure it will work fine with a relative small index
> >> such as yours. However, it does not scale that well with index size.
> >>
> >>> 2. Put the index on a solid state drive.
> >>
> >> Why anyone buys computers without SSD's is a mystery to me. Use SSDs for
> >> the small low-latency stuff and a secondary spinning drive for the large
> >> slow stuff. Nowadays, a 30GB index (or 100GB for that matter) falls into
> >> the small low-latency bucket. SSDs speeds up almost everything, saves
> >> RAM and spares a lot of work hours optimizing I/O-speed.
> >>
> >> Regards,
> >> Toke Eskildsen
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >
> >
> >
> > --
> > Lance Norskog
> > goksron@gmail.com
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
Re: RAM or SSD...
Posted by Vitaly Funstein <vf...@gmail.com>.
I was referring to *RAMDirectory*.
On Wed, Jul 18, 2012 at 11:04 PM, Lance Norskog <go...@gmail.com> wrote:
>> You do not want to store 30 G of data in the JVM heap, no matter what
library does this.
> MMapDirectory does not store data in the JVM heap. It lets the
> operating system manage the disk buffer space. Even if the JVM says "I
> have 30G of memory space", it really does not. It only has address
> space allocated by the OS but no memory.
>
> On Wed, Jul 18, 2012 at 10:39 PM, Toke Eskildsen <te...@statsbiblioteket.dk>
wrote:
>> On Wed, 2012-07-18 at 17:50 +0200, Dragon Fly wrote:
>>> If I want to improve performance, which of the following is better and
why?
>>>
>>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
>>
>> As others has pointed out, MMapDirectory should work better than
>> RAMDirectory. I am sure it will work fine with a relative small index
>> such as yours. However, it does not scale that well with index size.
>>
>>> 2. Put the index on a solid state drive.
>>
>> Why anyone buys computers without SSD's is a mystery to me. Use SSDs for
>> the small low-latency stuff and a secondary spinning drive for the large
>> slow stuff. Nowadays, a 30GB index (or 100GB for that matter) falls into
>> the small low-latency bucket. SSDs speeds up almost everything, saves
>> RAM and spares a lot of work hours optimizing I/O-speed.
>>
>> Regards,
>> Toke Eskildsen
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
Re: RAM or SSD...
Posted by Lance Norskog <go...@gmail.com>.
> You do not want to store 30 G of data in the JVM heap, no matter what library does this.
MMapDirectory does not store data in the JVM heap. It lets the
operating system manage the disk buffer space. Even if the JVM says "I
have 30G of memory space", it really does not. It only has address
space allocated by the OS but no memory.
On Wed, Jul 18, 2012 at 10:39 PM, Toke Eskildsen <te...@statsbiblioteket.dk> wrote:
> On Wed, 2012-07-18 at 17:50 +0200, Dragon Fly wrote:
>> If I want to improve performance, which of the following is better and why?
>>
>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
>
> As others has pointed out, MMapDirectory should work better than
> RAMDirectory. I am sure it will work fine with a relative small index
> such as yours. However, it does not scale that well with index size.
>
>> 2. Put the index on a solid state drive.
>
> Why anyone buys computers without SSD's is a mystery to me. Use SSDs for
> the small low-latency stuff and a secondary spinning drive for the large
> slow stuff. Nowadays, a 30GB index (or 100GB for that matter) falls into
> the small low-latency bucket. SSDs speeds up almost everything, saves
> RAM and spares a lot of work hours optimizing I/O-speed.
>
> Regards,
> Toke Eskildsen
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
--
Lance Norskog
goksron@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: RAM or SSD...
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
On Wed, 2012-07-18 at 17:50 +0200, Dragon Fly wrote:
> If I want to improve performance, which of the following is better and why?
>
> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
As others has pointed out, MMapDirectory should work better than
RAMDirectory. I am sure it will work fine with a relative small index
such as yours. However, it does not scale that well with index size.
> 2. Put the index on a solid state drive.
Why anyone buys computers without SSD's is a mystery to me. Use SSDs for
the small low-latency stuff and a secondary spinning drive for the large
slow stuff. Nowadays, a 30GB index (or 100GB for that matter) falls into
the small low-latency bucket. SSDs speeds up almost everything, saves
RAM and spares a lot of work hours optimizing I/O-speed.
Regards,
Toke Eskildsen
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: RAM or SSD...
Posted by Stephen Howe <si...@gmail.com>.
What metrics are you measuring performance by? Also, what is your current
setup? You might be able to speed up your current setup by tweaking
configuration settings without needing more hardware.
On Wed, Jul 18, 2012 at 11:50 AM, Dragon Fly <dr...@hotmail.com>wrote:
>
> Hi,
>
> If I want to improve performance, which of the following is better and why?
>
> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index.
> 2. Put the index on a solid state drive.
>
> By the way, my index is about 30 GB. Thank you.
>