You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Ganesh - yahoo <em...@yahoo.co.in> on 2008/08/01 07:44:40 UTC

Re: Using lucene as a database... good idea or bad idea?

Thanks Andy and Karsten.

----- Original Message ----- 
From: "Andy Liu" <an...@gmail.com>
To: <ja...@lucene.apache.org>
Sent: Thursday, July 31, 2008 8:16 PM
Subject: Re: Using lucene as a database... good idea or bad idea?


> If essentially all you need is key-value storage, Berkeley DB for Java 
> works
> well.  Lookup by ID is fast, can iterate through documents, supports
> secondary keys, updates, etc.
>
> Lucene would work relatively well for this, although inserting documents
> might not be as fast, because segments need to be merged and data ends up
> getting copied over again at certain points.  So if you're running a batch
> process with a lot of inserts, you might get better throughput with BDB as
> opposed to Lucene, but, of course, benchmark to confirm ;)
>
> Andy
>
> On Thu, Jul 31, 2008 at 9:12 AM, Karsten F.
> <ka...@fiz-technik.de>wrote:
>
>>
>> Hi Ganesh,
>>
>> in this Thread nobody said, that lucene is a good storage server.
>> Only "it could be used as storage server" (Grant: Connect data storage 
>> with
>> simple, fast lookup and Lucene..)
>>
>> I don't now about automatic rentention.
>> But for the rest in your list of features I suggest to take a deep look 
>> to
>>  - Jackrabbit (Standard jcr jsr170 implemention, I like the webDAV 
>> support)
>>  - dSpace (real working content repository software, with good 
>> permissions
>> management)
>>
>> Both use lucene for searching
>>
>> Best regards
>>    Karsten
>>
>>
>> Ganesh - yahoo wrote:
>> >
>> > which one will be the best to use as storage server. Lucene or
>> Jackrabbit.
>> >
>> > My requirement is to provide support to
>> > 1) Archive the documents
>> > 2) Do full text search on the documents.
>> > 3) Do backup the index store and archive store. [periodical basis]
>> > 4) Remove the documents after certain period [rentention policy]
>> >
>> > Whether Lucene could be used as archival store. Most of them in this
>> > mailing
>> > list said 'yes'. If so going for separate database to archive the data
>> and
>> > separate database to index it, will be better option or one database to
>> be
>> > used as archive and index.
>> >
>> > One more idea from this list is to use Jackrabbit / JDBM / My SQL to
>> > archive
>> > the data. Which will be the best?
>> >
>> > I am in desiging phase and i have time to explore and prototype any 
>> > other
>> > products. Please do suggest me a good one.
>> >
>> > Regards
>> > Ganesh
>> >
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Using-lucene-as-a-database...-good-idea-or-bad-idea--tp18703473p18754258.html
>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
> 

Send instant messages to your online friends http://in.messenger.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Using lucene as a database... good idea or bad idea?

Posted by Marcus Herou <ma...@tailsweep.com>.
Ah well some test classes could be appropriate:
http://dev.tailsweep.com/svn/abstractcache/trunk/src/test/java/org/tailsweep/abstractcache/test/

On Sun, Aug 3, 2008 at 5:07 PM, Marcus Herou <ma...@tailsweep.com>wrote:

> And for the heck of it I implemented a berkeleydb "java.util.Map" storage
> as well.
>
>
> http://dev.tailsweep.com/svn/abstractcache/trunk/src/main/java/org/tailsweep/abstractcache/disk/sleepycat/BerkelyDbCache.java
>
> Kindly
>
> //Marcus
>
>
> On Fri, Aug 1, 2008 at 7:44 AM, Ganesh - yahoo <em...@yahoo.co.in>wrote:
>
>> Thanks Andy and Karsten.
>>
>> ----- Original Message ----- From: "Andy Liu" <an...@gmail.com>
>> To: <ja...@lucene.apache.org>
>> Sent: Thursday, July 31, 2008 8:16 PM
>> Subject: Re: Using lucene as a database... good idea or bad idea?
>>
>>
>>
>>  If essentially all you need is key-value storage, Berkeley DB for Java
>>> works
>>> well.  Lookup by ID is fast, can iterate through documents, supports
>>> secondary keys, updates, etc.
>>>
>>> Lucene would work relatively well for this, although inserting documents
>>> might not be as fast, because segments need to be merged and data ends up
>>> getting copied over again at certain points.  So if you're running a
>>> batch
>>> process with a lot of inserts, you might get better throughput with BDB
>>> as
>>> opposed to Lucene, but, of course, benchmark to confirm ;)
>>>
>>> Andy
>>>
>>> On Thu, Jul 31, 2008 at 9:12 AM, Karsten F.
>>> <ka...@fiz-technik.de>wrote:
>>>
>>>
>>>> Hi Ganesh,
>>>>
>>>> in this Thread nobody said, that lucene is a good storage server.
>>>> Only "it could be used as storage server" (Grant: Connect data storage
>>>> with
>>>> simple, fast lookup and Lucene..)
>>>>
>>>> I don't now about automatic rentention.
>>>> But for the rest in your list of features I suggest to take a deep look
>>>> to
>>>>  - Jackrabbit (Standard jcr jsr170 implemention, I like the webDAV
>>>> support)
>>>>  - dSpace (real working content repository software, with good
>>>> permissions
>>>> management)
>>>>
>>>> Both use lucene for searching
>>>>
>>>> Best regards
>>>>   Karsten
>>>>
>>>>
>>>> Ganesh - yahoo wrote:
>>>> >
>>>> > which one will be the best to use as storage server. Lucene or
>>>> Jackrabbit.
>>>> >
>>>> > My requirement is to provide support to
>>>> > 1) Archive the documents
>>>> > 2) Do full text search on the documents.
>>>> > 3) Do backup the index store and archive store. [periodical basis]
>>>> > 4) Remove the documents after certain period [rentention policy]
>>>> >
>>>> > Whether Lucene could be used as archival store. Most of them in this
>>>> > mailing
>>>> > list said 'yes'. If so going for separate database to archive the data
>>>> and
>>>> > separate database to index it, will be better option or one database
>>>> to
>>>> be
>>>> > used as archive and index.
>>>> >
>>>> > One more idea from this list is to use Jackrabbit / JDBM / My SQL to
>>>> > archive
>>>> > the data. Which will be the best?
>>>> >
>>>> > I am in desiging phase and i have time to explore and prototype any >
>>>> other
>>>> > products. Please do suggest me a good one.
>>>> >
>>>> > Regards
>>>> > Ganesh
>>>> >
>>>> >
>>>> >
>>>>
>>>> --
>>>> View this message in context:
>>>>
>>>> http://www.nabble.com/Using-lucene-as-a-database...-good-idea-or-bad-idea--tp18703473p18754258.html
>>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>>
>>>
>> Send instant messages to your online friends
>> http://in.messenger.yahoo.com
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
> --
> Marcus Herou CTO and co-founder Tailsweep AB
> +46702561312
> marcus.herou@tailsweep.com
> http://www.tailsweep.com/
> http://blogg.tailsweep.com/
>



-- 
Marcus Herou CTO and co-founder Tailsweep AB
+46702561312
marcus.herou@tailsweep.com
http://www.tailsweep.com/
http://blogg.tailsweep.com/

Re: Using lucene as a database... good idea or bad idea?

Posted by Marcus Herou <ma...@tailsweep.com>.
And for the heck of it I implemented a berkeleydb "java.util.Map" storage as
well.

http://dev.tailsweep.com/svn/abstractcache/trunk/src/main/java/org/tailsweep/abstractcache/disk/sleepycat/BerkelyDbCache.java

Kindly

//Marcus

On Fri, Aug 1, 2008 at 7:44 AM, Ganesh - yahoo <em...@yahoo.co.in>wrote:

> Thanks Andy and Karsten.
>
> ----- Original Message ----- From: "Andy Liu" <an...@gmail.com>
> To: <ja...@lucene.apache.org>
> Sent: Thursday, July 31, 2008 8:16 PM
> Subject: Re: Using lucene as a database... good idea or bad idea?
>
>
>
>  If essentially all you need is key-value storage, Berkeley DB for Java
>> works
>> well.  Lookup by ID is fast, can iterate through documents, supports
>> secondary keys, updates, etc.
>>
>> Lucene would work relatively well for this, although inserting documents
>> might not be as fast, because segments need to be merged and data ends up
>> getting copied over again at certain points.  So if you're running a batch
>> process with a lot of inserts, you might get better throughput with BDB as
>> opposed to Lucene, but, of course, benchmark to confirm ;)
>>
>> Andy
>>
>> On Thu, Jul 31, 2008 at 9:12 AM, Karsten F.
>> <ka...@fiz-technik.de>wrote:
>>
>>
>>> Hi Ganesh,
>>>
>>> in this Thread nobody said, that lucene is a good storage server.
>>> Only "it could be used as storage server" (Grant: Connect data storage
>>> with
>>> simple, fast lookup and Lucene..)
>>>
>>> I don't now about automatic rentention.
>>> But for the rest in your list of features I suggest to take a deep look
>>> to
>>>  - Jackrabbit (Standard jcr jsr170 implemention, I like the webDAV
>>> support)
>>>  - dSpace (real working content repository software, with good
>>> permissions
>>> management)
>>>
>>> Both use lucene for searching
>>>
>>> Best regards
>>>   Karsten
>>>
>>>
>>> Ganesh - yahoo wrote:
>>> >
>>> > which one will be the best to use as storage server. Lucene or
>>> Jackrabbit.
>>> >
>>> > My requirement is to provide support to
>>> > 1) Archive the documents
>>> > 2) Do full text search on the documents.
>>> > 3) Do backup the index store and archive store. [periodical basis]
>>> > 4) Remove the documents after certain period [rentention policy]
>>> >
>>> > Whether Lucene could be used as archival store. Most of them in this
>>> > mailing
>>> > list said 'yes'. If so going for separate database to archive the data
>>> and
>>> > separate database to index it, will be better option or one database to
>>> be
>>> > used as archive and index.
>>> >
>>> > One more idea from this list is to use Jackrabbit / JDBM / My SQL to
>>> > archive
>>> > the data. Which will be the best?
>>> >
>>> > I am in desiging phase and i have time to explore and prototype any >
>>> other
>>> > products. Please do suggest me a good one.
>>> >
>>> > Regards
>>> > Ganesh
>>> >
>>> >
>>> >
>>>
>>> --
>>> View this message in context:
>>>
>>> http://www.nabble.com/Using-lucene-as-a-database...-good-idea-or-bad-idea--tp18703473p18754258.html
>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>
> Send instant messages to your online friends http://in.messenger.yahoo.com
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


-- 
Marcus Herou CTO and co-founder Tailsweep AB
+46702561312
marcus.herou@tailsweep.com
http://www.tailsweep.com/
http://blogg.tailsweep.com/