You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Hiren Shah <hi...@gmail.com> on 2012/04/10 22:40:26 UTC
Query for "cache" mechanism to used
I have a requirement to get results of search query within 1 second for a
database table.The database table is returning results slowly at this
point.A table has to be moved to a cache and searched from there so that
search results come fast.I want to do google type page refresh on my
existing search page -- which means the page should refresh as the user
types.
In order to acheive this the search results should return within one
second.My database is teradata.Its queries are taking 2 to 3 seconds at
least.Hence i want to look for other options like caching.I want to use
cache so that the resuls come fast.
Columns are
*company , Id , Industry, parent ...4 more*
Its a search page.So if user types "ja" all items starting from ja like
*company ------------- Id ------------- Industry --------------parent*
jaico ------------- 222 -------------paints ------------- Jaico asia
Jammy fruits------------- 232-------------food------------- jammy
International
The table contains 3.2 million rows and there are 8 columns that are
present.The search data need to return all 8 columns.Considering byte wise
there are 150 chars per row.So total bytes are 3.2 million * 150 chars =
480 Megabytes .I need to store this much data in cache and then fire search
queries like sql (grouping ,like ,order by) across them.What would be the
best option to use in this case
1. ehcache
2. jboss cache
3. Inifinispan
4. Apache Lucene
Please suggest which option is good .Is it better to do caching in memory
or to use lucene?
*What need to be cached?-->* It is a table of 3.2 million rows with 8
columns.
*Why it is to be cached?-->* It is to be cached so that search results come
*faster* than sql query.If i use sql query it takes very long time.Hence i
want to move towards caching data.
Re: Query for "cache" mechanism to used
Posted by Hiren Shah <hi...@gmail.com>.
Hello Ivan
Thanks for the reply.
1. I tried to use lucene .It stored the data on indexes on the hard
disk.The search works well.How of the ehcache above lucene will help
performane.
2. Will Lucene used alone give better performance. *OR *Will ehcache
used above Lucene give better performance?
3. How to make ehcache above Lucene?Any help on it would be
appreciated.I am using sql table to be stored as cache as i need to fire
sql queries like on "multiple" fields.
4. One proble, i came across ehcahce was that for searchable API it
does not allow disk overflow.API fails if i try to do it.So if my data is
500Mb and jvm memory is configured for it then it works fine.But if the
data exceeds 500Mb then it cannot be cached.Is this not a major
problem.is there any workaround on this?
Thanks
Hiren
On Thu, Apr 12, 2012 at 4:12 AM, Ivan Brusic <iv...@brusic.com> wrote:
> A cache should be independent of the data store. Ehcache works well in
> front of Lucene as well as a (relational) database. However, caches
> work great for key/value data, so the cache value would be a result
> set. Is caching the grouped result good enough?
>
> --
> Ivan
>
> On Tue, Apr 10, 2012 at 1:40 PM, Hiren Shah <hi...@gmail.com>
> wrote:
> > I have a requirement to get results of search query within 1 second for a
> > database table.The database table is returning results slowly at this
> > point.A table has to be moved to a cache and searched from there so that
> > search results come fast.I want to do google type page refresh on my
> > existing search page -- which means the page should refresh as the user
> > types.
> > In order to acheive this the search results should return within one
> > second.My database is teradata.Its queries are taking 2 to 3 seconds at
> > least.Hence i want to look for other options like caching.I want to use
> > cache so that the resuls come fast.
> >
> > Columns are
> > *company , Id , Industry, parent ...4 more*
> > Its a search page.So if user types "ja" all items starting from ja like
> >
> > *company ------------- Id ------------- Industry --------------parent*
> > jaico ------------- 222 -------------paints ------------- Jaico asia
> > Jammy fruits------------- 232-------------food------------- jammy
> > International
> >
> > The table contains 3.2 million rows and there are 8 columns that are
> > present.The search data need to return all 8 columns.Considering byte
> wise
> > there are 150 chars per row.So total bytes are 3.2 million * 150 chars =
> > 480 Megabytes .I need to store this much data in cache and then fire
> search
> > queries like sql (grouping ,like ,order by) across them.What would be the
> > best option to use in this case
> >
> > 1. ehcache
> > 2. jboss cache
> > 3. Inifinispan
> > 4. Apache Lucene
> >
> > Please suggest which option is good .Is it better to do caching in memory
> > or to use lucene?
> > *What need to be cached?-->* It is a table of 3.2 million rows with 8
> > columns.
> > *Why it is to be cached?-->* It is to be cached so that search results
> come
> > *faster* than sql query.If i use sql query it takes very long time.Hence
> i
> > want to move towards caching data.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Query for "cache" mechanism to used
Posted by Ivan Brusic <iv...@brusic.com>.
A cache should be independent of the data store. Ehcache works well in
front of Lucene as well as a (relational) database. However, caches
work great for key/value data, so the cache value would be a result
set. Is caching the grouped result good enough?
--
Ivan
On Tue, Apr 10, 2012 at 1:40 PM, Hiren Shah <hi...@gmail.com> wrote:
> I have a requirement to get results of search query within 1 second for a
> database table.The database table is returning results slowly at this
> point.A table has to be moved to a cache and searched from there so that
> search results come fast.I want to do google type page refresh on my
> existing search page -- which means the page should refresh as the user
> types.
> In order to acheive this the search results should return within one
> second.My database is teradata.Its queries are taking 2 to 3 seconds at
> least.Hence i want to look for other options like caching.I want to use
> cache so that the resuls come fast.
>
> Columns are
> *company , Id , Industry, parent ...4 more*
> Its a search page.So if user types "ja" all items starting from ja like
>
> *company ------------- Id ------------- Industry --------------parent*
> jaico ------------- 222 -------------paints ------------- Jaico asia
> Jammy fruits------------- 232-------------food------------- jammy
> International
>
> The table contains 3.2 million rows and there are 8 columns that are
> present.The search data need to return all 8 columns.Considering byte wise
> there are 150 chars per row.So total bytes are 3.2 million * 150 chars =
> 480 Megabytes .I need to store this much data in cache and then fire search
> queries like sql (grouping ,like ,order by) across them.What would be the
> best option to use in this case
>
> 1. ehcache
> 2. jboss cache
> 3. Inifinispan
> 4. Apache Lucene
>
> Please suggest which option is good .Is it better to do caching in memory
> or to use lucene?
> *What need to be cached?-->* It is a table of 3.2 million rows with 8
> columns.
> *Why it is to be cached?-->* It is to be cached so that search results come
> *faster* than sql query.If i use sql query it takes very long time.Hence i
> want to move towards caching data.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org